Excellent

The feature freeze for R 2.7.0 is in place with an expected April 22nd release. I've noticed some of the locale stuff has changed recently (LC_CTYPE and the like) so I'll need to fix those warning messages. Right now I'm setting up the IKImageBrowserView-backed graphics device interface for the GUI, though the built-in quartz device has been serving me well so far. There are a couple of reasons for the view set up, which uses bitmaps to represent the graphics device on the backend. After the page is completed, the bitmap is converted to a more efficient representation (e.g. PNG) for temporary storage and the display list recorded into the device. We'll also continue to cheat and provide a Cover Flow mode (no, I don't care that it's a private API).

Another interesting feature is that, unlike the current Mac GUI, the graphics device is considered a View on the same Model (the interpreter). We'll see how it plays out, but that might help with some of the fancy footwork usually required when dealing with the NSDocumentController.

Implementing XCode 3 parenthesis highlighting

One of the things I like about XCode 3 is the parenthesis matching. It seems very obtrusive at first, but I find it works better than the more subtle Emacs-style matching once you get used to it. For the R GUI I wanted to implement the same thing, which turned out to be fairly simple.

The highlighter is implemented using the new [NSTextView showFindIndicatorInRange:] so there's literally no work to do there.

The first thing you need to know is that adding this functionality in textView:shouldChangeTextInRange:replacementString: won't work because the text is added after the method executes and that removes the find indicator. Fortunately, we just implemented a custom keybinding in the last post so all we need to do is bind the various close delimiters to special selectors (fancyCloseParen: for example). Then we can do things in doCommandBySelector:

// ...
} else if(@selector(fancyCloseParen:) == aSelector) {
  [self insertDelimiter:')' withMatch:'(' inTextView:aTextView atPosition:aRange.location];
  return YES;
} // ...

next all we have to do is find the delimiter (which we do with a category on NSString) and then insert the find indicator:

- (void)insertDelimiter:(unichar)aDelim withMatch:(unichar)aMatch inTextView:(NSTextView*)aTextView atPosition:(NSInteger)aPos {
  [aTextView insertText:[NSString stringWithFormat:@"%c",aDelim]];
  NSRange where = [[aTextView string] rangeOfDelimiter:aMatch matching:aDelim inRange:NSMakeRange(lastPrompt,aPos-lastPrompt)];
  if(where.location == NSNotFound)
   NSBeep();
  else
   [aTextView showFindIndicatorForRange:where];
}

Easy as pie.

On Key Bindings

For my R GUI I want user customizable key bindings. I know for a fact that there are customizable key bindings built into the Cocoa text system, but they aren't user accessible and the only response from Apple was a message posted to a mailing list about 2 years ago to the effect of "I have something." Two years later, well, surprise, surprise. So what to do?

Well, for the first pass (found in the R-Multi git repository) I class-dumped AppKit and found NSKeyBindingManager. It turns out that this class has a shared initializer and a means for setting a new key binding dictionary. So, I did the simple thing and just changed that dictionary during app initialization to support some new keybindings similar to the ones found in TextMate (which doesn't use NSTextView as far as I know):

NSMutableDictionary *newBindings = [[NSMutableDictionary alloc] 
    initWithDictionary:origBindings];
[[NSKeyBindingManager sharedKeyBindingManager] setDictionary:newBindings];
[newBindings release];

About as simple as you can possibly get. Unfortunately, this approach has several drawbacks primarily due to the fact that it is a global change to the entire application (which is more useful that the SYSTEM global version mostly recommended). Can we do better? Let's find out.

For a clue, let's look at one of the R GUI's own crash traces:

[NSTextView keyDown:]
...[NSView interpretKeyEvents:]
......[NSTSMInputContext interpretKeyEvents:]
.........[NSKeyBindingManager(NSKeyBindingManager_MultiClients) flushTextForClient:]

Excellent. So it looks like we'll be needing to subclass and hook into interpretKeyEvents: to get things to work properly in our system. However, we cannot simply use an NSKeyBindingManager because the manager doesn't return a BOOL so we can't tell when we haven't handled a binding. Or does it?

So, it turns out there is a MultiClient category on NSKeyBindingManager that lets us do just that using interpretEventAsCommand:forClient:

So, my interpretKeyEvents now looks like:

- (void)interpretKeyEvents:(id)sender {
 if([(NSArray*)sender count] == 1) {
  if(YES == [manager interpretEventAsCommand:[(NSArray*)sender objectAtIndex:0] 
                   forClient:self]) return;
 }
 [super interpretKeyEvents:sender];
}

So far it seems to work in testing, but I'll have to do more testing with it over time. For example, I don't know if I'll ever see more than one event in that array.

R 2.7.0 Quartz graphics device clipboard support

Tom Elliot recently posted a request to the R-SIG-Mac list looking for a way to programmatically put graphics onto the clipboard under OS X. Right now this is pretty hard, but I was inspired to patch R-devel's (the future 2.7.0) new Quartz device to allow for clipboard output.

After applying the patch, you can specify file="clipboard://" and when the device is closed (yes, you MUST close the graphics device to see output. It would probably be a good idea to write the file on a NewPage as well, come to think of it). Then you can wrap up a simple function to get copying of the current device to the clipboard (or you can simply open it directly).

copy.to.clipboard = function(dpi=300) { dev.copy(device=quartz,type="png",file="clipboard://",dpi=dpi);dev.close(); }

I've attached the patch for the brave souls willing to try it out:

Index: qdBitmap.c
===================================================================
--- qdBitmap.c  (revision 43081)
+++ qdBitmap.c  (working copy)
@@ -49,16 +49,45 @@
         /* On 10.4+ we can employ the CGImageDestination API to create a
            variety of different bitmap formats */
 #if MAC_OS_X_VERSION_MAX_ALLOWED >= MAC_OS_X_VERSION_10_4
-        CFURLRef    path  = CFURLCreateFromFileSystemRepresentation(kCFAllocatorDefault,(const UInt8*)qbd->path,strlen(qbd->path),FALSE);
-        CFStringRef type  = CFStringCreateWithBytes(kCFAllocatorDefault,(UInt8*)qbd->uti,strlen(qbd->uti),kCFStringEncodingUTF8,FALSE);
-        CGImageDestinationRef dest = CGImageDestinationCreateWithURL(path,type,1,NULL);
-        CGImageRef image = CGBitmapContextCreateImage(qbd->bitmap);
-        CGImageDestinationAddImage(dest,image,NULL);
-        CGImageDestinationFinalize(dest);
-        CFRelease(image);
-        CFRelease(dest);
-        CFRelease(type);
+        CFStringRef pathString = CFStringCreateWithBytes(kCFAllocatorDefault,(UInt8*)qbd->path,strlen(qbd->path),kCFStringEncodingUTF8,FALSE);
+               CFURLRef path;
+               if(CFStringFind(pathString,CFSTR("://"),0).location != kCFNotFound) {
+               CFStringRef pathEscaped= CFURLCreateStringByAddingPercentEscapes(kCFAllocatorDefault,pathString,NULL,NULL,kCFStringEncodingUTF8);
+                       path = CFURLCreateWithString(kCFAllocatorDefault,pathEscaped,NULL);
+                       CFRelease(pathEscaped);
+               } else {
+                       path = CFURLCreateFromFileSystemRepresentation(kCFAllocatorDefault,(const UInt8*)qbd->path,strlen(qbd->path),FALSE);
+               }
+               CFRelease(pathString);
+               
+               CFStringRef scheme = CFURLCopyScheme(path);
+               CFStringRef type  = CFStringCreateWithBytes(kCFAllocatorDefault,(UInt8*)qbd->uti,strlen(qbd->uti),kCFStringEncodingUTF8,FALSE);
+       CGImageRef image = CGBitmapContextCreateImage(qbd->bitmap);
+               if(CFStringCompare(scheme,CFSTR("file"),0) == 0) {
+               CGImageDestinationRef dest = CGImageDestinationCreateWithURL(path,type,1,NULL);
+               CGImageDestinationAddImage(dest,image,NULL);
+               CGImageDestinationFinalize(dest);
+               CFRelease(dest);
+               } else if(CFStringCompare(scheme,CFSTR("clipboard"),0) == 0) {
+                       //Copy our image into data
+                       CFMutableDataRef      data = CFDataCreateMutable(kCFAllocatorDefault,0);
+                       CGImageDestinationRef dest = CGImageDestinationCreateWithData(data,type,1,NULL);
+                       CGImageDestinationAddImage(dest,image,NULL);
+                       CGImageDestinationFinalize(dest);
+                       CFRelease(dest);
+                       PasteboardRef pb = NULL;
+                       if(noErr == PasteboardCreate(kPasteboardClipboard,&pb)) {
+                               PasteboardClear(pb);
+                               PasteboardSyncFlags syncFlags = PasteboardSynchronize(pb);
+                               PasteboardPutItemFlavor(pb,(PasteboardItemID)1,type,data,0);
+                       }
+                       CFRelease(data);
+               } else
+                       warning("Not a supported scheme, no image data written.");
+               CFRelease(scheme);
+               CFRelease(type);
         CFRelease(path);
+               CFRelease(image);
 #endif
     }
     /* Free ourselves */

Cover Flow!

Yup. Figured it out. One of two ways to browse display plots. There is a more useful mode, using an iPhoto-like Browser that also lets you edit the list of plots. The arrow keys also let you move between plots.

iWork '08 Spelunking.

Fiddling around with iWork '08 a bit and Numbers support is trivial to add to the R GUI. Basically, it exports a tab delimited version as text, much like Excel, though it uses the more modern pasteboard type (where Excel exports a much less useful string). We can detect that it is numbers from the metadata and "native" pasteboard types and throw up the same dialog as for the Excel pasting. Yay!

I'm thinking, for symmetry, that I should let you right click on a data frame in the workspace and copy as a tab delimited type for pasting into Excel/Numbers as well.

Oh, yeah.

I did figure out one thing in an idle moment on vacation last week. How to paste from Excel to the R console. Thought y'all might like something like that. I should probably do another featurecast soon to show some of the things that have been fleshed out a bit. These include...

• Multi-level hinting. If you have "plot(foo()," the hinting mechanism will show you "plot," rather than nothing


• Drag-n-drop from the workspace to the console gets a summary() on those objects


• Excel (or Excel-like things I suppose. Anything that posts VALU or NSTabularPboardType) pasting


• Device recording, with the option to deactivate as well as clear plot listing


• Clear the console


• Copying text from the console optionally does so in a "source()-able" manner. i.e. prompts and output are removed.


• Keybindings for popular things like Command-= to give you "<-"


• TextMate-style brace handling. i.e. select some text and type "(" (or ",`,',{,[,Command-[) and it will put the appropriate bracket around the selected text.


• An editor. :-)


• Preferences :-)

Any requests?

I'm sure there's other stuff I've forgotten at this point, but I'm still unburying myself otherwise and getting ready to head to BioC'07 in a couple of days.

RExecServer git repository update

The master branch of the git repository is now up-to-date with the leopard branch, minus any leopard specific code (of which is there none at present). R-devel is required to get things to compile. The GUI code isn't available yet, I'd like to get things to a truly usable stage first. I haven't had as much time this week, other things have taken priority, but some minor stuff was added such as the beginnings of the preference panes (and the infrastructure to support them) as well as the start of the action menus. I also changed the graphics device plot selector to take advantage of available space. Perhaps I'll post a screencast of that soon.

In the meantime, people who aren't using OS X, but want to consider a similar GUI type of set up might want to look at the Omegahat CORBA stuff, which did something very similar a long time ago. The page hasn't even been updated since before OS X 10.0 was released. :-)

Just a couple minor updates

Doing some other things today so only a couple of minor updates watching The Tour this evening:

• Removed the mouse-sensitive pages menu. This seems to be pretty disruptive without putting some extra delay into the transition, but the delay makes it too slow. Instead, I'm now using the toolbar lozenge to show and hide. I think I'll also hook it up to middle-click.


• Implemented some display transformers for the Workspace. There's a (localized) "n objects" on the bottom now. The class and size information for objects is now shown (could probably be prettier!) as well.
  
• Updated: Oh, yeah. You can also clear a console with Cmd-L now (the current prompt is restored). I saw that feature request on R-SIG-Mac a couple of weeks ago and it was something like 5 lines of code so I tossed it in.
  

R GUI Screencast

Some of the new GUI features are hard to get from screenshots so I decided to make a little screencast to show off a couple of the new features.

Starting to think about help...

You know what would be awesome? If the HTML manpages generated by R were actually microformat-enabled HTML. I was just thinking about how it would be nice if you could identify example code and whatnot from the eventual help page. I wonder how scary that code is? IIRC the HTML generated by R's manpage generator was pretty much HTML3 (so not even easy to style) so it could be pretty scary. It may even be one of the Perl bits.

Okay, I need to get some sleep now.

Making Progress on the Console

Making some progress on the GUI's Console implementation. You can see that we now indicate WHICH server we're talking to at the moment. I've added an attached "workspace" inspector in the form of a source list such as the ones you see in Mail and iTunes. This holds information about the objects for inspection or dragging and dropping between servers. The thin black line is a splitter bar and is user selectable (double clicking collapses it entirely).

Both sides now have a status bar (of course I made sure the shading matched the action button... why do you ask?) with an action menu. As well, the hint is back and is now being powered from a new object inspection infrastructure in RExecServer (I have some other code to work on, but I will hopefully mind a few minutes to finish things and do a git push). We're now parsing out the function information rather than capturing string output so I think we can do completion in the style of Xcode or TextMate where, say, lm completion would result in:

lm(<#formula>,<#data>,<#subset>,<#weights>,<#na.action>)

I think Ctrl-/ should advance and that TAB (perhaps, I'm open to suggestion) should complete the editing. In other words, imagine we had...

lm(X ~ Y,mydata,<#subset>,<#weights>,<#na.action>)

then TAB would clean the other three arguments and jump us out of the function. You can't see it, but the console is actually doing a lot of invisible markup that we can use to detect the completion regions. We're presently also using text attributes to track input, prompt, error and output regions. This means we can easily cut and paste text WITHOUT prompts (which has been a major feature requests) and without ambiguity. We can also save a console script as a sourceable .R file for later use (sort of like dribble) saving the previous output as comments or not as all.

Ha Ha! Success!

After several hours of missing one very important line of code in RExecServer, which was preventing it from operating in purely vended mode---it was a delegate problem and the TerminalDelegate would replace it---I've gotten RExecServer executing and communicating with an R GUI.





The GUI is Leopard specific, but I can offer some details so far:

• We use a normal NSDocument with no modifications to NSDocumentController or NSApplication or any of the questionable things I had to do for the pre-RExecServer GUI implementation


• Devices are considered to be views of the document so devices and the console are explicitly intertwined now. In much the same way that the main Interface Builder window and the associated UI views are linked. Hopefully this means an end to the infamous "hanging" windows.


• Scripts won't be, but we'll need a UI for specifying their target console...

Apparently I'm on a tear...

Rob Goedman gets the Official Tester Award for RExecServer, reporting several issues with the graphics subsystem and my inability to appropriately use version control systems. There are a bunch of changes in the git repository:

• All the files you need are actually there


• Clipping rects are restored on subsequent updates so some things are working better


• Terminal support has been refactored into a pseudo-GUI to help make sure I have coverage on the things I'll need for a true front end. Its probably fractionally slower than a true Terminal version of R, but it has some more flexibility


• Device windows close when devices go away. A good thing too because they'd crash if you resized them after the device disappeared.

I've been playing with making the RGUI_Type not be AQUA, which restores all of the help file functionality, but causes an annoying (and untrue) complaint from quartz() among other things like trying to use X11 for select.list(). I wish there was a way to selectively deactivate things like that in R instead of the blanket situation we have now. There are a number of places where we have

if(.Platform$GUI == "windows" | .Platform$GUI == "AQUA") ... else ...

that are just irritating. Personally, I think we should be doing dispatch (S3, S4, I don't care) where .Platform$GUI becomes an _object_ so that I can define functions for my particular GUI and where .default is the stuff on the RHS of the else.

RExecServer object vending

The RExecServer started getting basic object vending today. To test it out I implemented some simple object copying using Distributed Objects. This is checked into the public git repository, but will require R-devel to run because of changes to function export under R-devel. Behold!

First we start ourselves a couple of copies of R. Then,

> x = 1:100
> .Call("RES_CopyObject","x","R Execution Server 2")
NULL
> .Call("RES_ServerName")
[1] "R Execution Server 1"
> 

Afterwards, we can take a look at the other execution server:

> ls()
[1] "x"
> .Call("RES_ServerName")
[1] "R Execution Server 2"
> x
  [1]   1   2   3   4   5   6   7   8   9  10  11  12  13  14  15  16  17  18
 [19]  19  20  21  22  23  24  25  26  27  28  29  30  31  32  33  34  35  36
 [37]  37  38  39  40  41  42  43  44  45  46  47  48  49  50  51  52  53  54
 [55]  55  56  57  58  59  60  61  62  63  64  65  66  67  68  69  70  71  72
 [73]  73  74  75  76  77  78  79  80  81  82  83  84  85  86  87  88  89  90
 [91]  91  92  93  94  95  96  97  98  99 100
> 

Ta Da! There's no big trick to it--we're just using the standard serialization routines to read and write NSData objects on the Cocoa side and then using the usual NSDistantObject routines to actually transmit the data. There's no real error checking at the moment, but that will come.

Also, with the help of some scripts from Rob Goedman I think I've tracked down the last of the clipping and state stacking errors in this latest checkin.

RExecServer available as a git repository

The RExecServer source is now available as a git repository. Once I figure out what exactly they mean I will be "mobbing" it to allow for patch submission. The binary hasn't been updated, but the source fixes a few reported problems with the pager() and some graphics issues (clipping mostly). It also adds the ability to specify the usual command-line options except for the gui related ones (for obvious reasons).

Old Things New Again: Multiple Evaluators for R Under OS X

This post is mostly the result of a conversation I had with Stefano Iacus a couple of weeks ago at WWDC. He was making the observation that a) he would like to be able to run multiple copies of R from the R GUI and b) that he would really really love to run R evaluators over XGrid.

The second one might be harder, but I think the first one can be solved. Ideally, we would simply be able to spawn off multiple R evaluators in separate threads within the R GUI and apart from synchronization problems in the GUI we would be good to go. However, I rate the chances of R becoming thread-safe (let alone supporting multiple evaluators) any time soon as "slim to none." Of course, I'm not on R Core so I could be wrong, but from what it would take (every function in every package would need an extra argument for starters) it seems unlikely. The way around this? Spawn off separate R processes and connect them up within the R GUI. This is basically what people do when they use R from Terminal so there is no real disadvantage compared to the current methods and a lot of potential advantages.

So, the plan:

1. Implement RExecServer as an LSUIElement application. As much as I'd like to use Leopard-specific features here (garbage collection in particular), people using Tiger have multiple processors too. So, we're stuck with autorelease pools for now.
a. Vend an interface as a NSDistantObject that can be picked up by the GUI
b. Provide a threaded stdin reader (if TERM is set) using the "traditional" R reader. This is to provide ESS support. I think we can actually vend the object and allow the stdin reader at the same time. Er, this could be a cool feature for something we'll talk about in October (if all goes well, this isn't my day job so it only gets implemented when I have some spare (hah) time) :-).
c. Theoretically, we could vend to/from different machines. Using Bonjour you could publish your R session. We'd have to work out some sort of security model. Not sure how that's normally handled by NSDistantObject.

2. Change the graphics device a bit. Mostly I don't think we want to ship around the graphics list. In general, we can ship a bitmap that is appropriate for the display. We can also ship back a PDF if so desired, but performance with a bitmap is likely to be higher with no discernible quality difference except in special circumstances. We can just have a [Device dataInFormat:] that sends back an NSData of the appropriate format (RGBA bitmap, PDF, etc). The nice part is that both the client and server are running OS X and both have access to the Font metrics so there doesn't need to be communication there.
a. On the ESS thing again, we can provide a simple shim device window to give ESS users a decent graphics device with full interaction. It won't be as cool as the one in the GUI, but that's what you get for using ESS ;-).

3. The GUI now maintains connections to any number of GUI console/device windows.
a. Stefano suggested being able to copy and paste between environments. I think this can be done using private Pasteboards and serializing objects to RAWSXP types, converting them to NSData and then transferring them over.
b. The GUI itself never need become unresponsive. You could even force kill a runaway server.

Now, certain things get more difficult. Certain GUI toolkits, like my own Mojave, won't be running in the GUI process anymore making them difficult to write GUIs. Of course, nobody that I know of writes GUIs using Mojave (and you'd think I'd know), but this also rules out everything else. Personally, I think the way around this is some sort of Dashboard-style interface where the front-end is implemented in HTML and Javascript with hooks back into R using a special protocol handler (which is apparently an SDK these days...) or a Javascript proxy to allow execution. I tend to favor the protocol handler.