Fascinating Captain...

I recently learned that Snow Leopard includes a (public!) framework called IOSurface that allows for the sharing of basically OpenGL textures between processes. That neatly solves the problem with having your graphics device running in a separate process for the GUI. Previously, I've shipped bitmaps around between the GUI and the backing process (which is headless) but this could definitely be a performance improvement both in update speed and in memory. I'll have to look into it.

I suspect even RGL could be made to operate in this mode though you'd need some help from the package since it expects to run an event loop for handling mouse interaction with the scene graph.

Aw, nuts.

Upgraded to Snow Leopard and so it is time once again to make some changes to my personal R GUI--if for no other reason than to test cool stuff like Blocks and Grand Central Dispatch. It also looks like there's some interesting developments in Pasteboards and a few other under the hood places.

First off are Blocks. They look like function pointers but they definitely aren't. They're actually Objective-C classes that you can treat like functions. Very Smalltalk, complete with similar tricks to stuff their data onto the stack you explicitly move them to the heap (they're objects so you can copy them).

Unfortunately, they are not interchangeable with C function pointers so you can't just use them as callback functions. A shame really, since right now I have a libffi-based binding between the R callback functions (ReadConsole, WriteConsole, etc) to thunk to Objective-C methods. All hope is not lost, the R binding is not so complicated so writing a Block compatible version shouldn't be too hard.

Implementing XCode 3 parenthesis highlighting

One of the things I like about XCode 3 is the parenthesis matching. It seems very obtrusive at first, but I find it works better than the more subtle Emacs-style matching once you get used to it. For the R GUI I wanted to implement the same thing, which turned out to be fairly simple.

The highlighter is implemented using the new [NSTextView showFindIndicatorInRange:] so there's literally no work to do there.

The first thing you need to know is that adding this functionality in textView:shouldChangeTextInRange:replacementString: won't work because the text is added after the method executes and that removes the find indicator. Fortunately, we just implemented a custom keybinding in the last post so all we need to do is bind the various close delimiters to special selectors (fancyCloseParen: for example). Then we can do things in doCommandBySelector:

// ...
} else if(@selector(fancyCloseParen:) == aSelector) {
  [self insertDelimiter:')' withMatch:'(' inTextView:aTextView atPosition:aRange.location];
  return YES;
} // ...

next all we have to do is find the delimiter (which we do with a category on NSString) and then insert the find indicator:

- (void)insertDelimiter:(unichar)aDelim withMatch:(unichar)aMatch inTextView:(NSTextView*)aTextView atPosition:(NSInteger)aPos {
  [aTextView insertText:[NSString stringWithFormat:@"%c",aDelim]];
  NSRange where = [[aTextView string] rangeOfDelimiter:aMatch matching:aDelim inRange:NSMakeRange(lastPrompt,aPos-lastPrompt)];
  if(where.location == NSNotFound)
   NSBeep();
  else
   [aTextView showFindIndicatorForRange:where];
}

Easy as pie.

On Key Bindings

For my R GUI I want user customizable key bindings. I know for a fact that there are customizable key bindings built into the Cocoa text system, but they aren't user accessible and the only response from Apple was a message posted to a mailing list about 2 years ago to the effect of "I have something." Two years later, well, surprise, surprise. So what to do?

Well, for the first pass (found in the R-Multi git repository) I class-dumped AppKit and found NSKeyBindingManager. It turns out that this class has a shared initializer and a means for setting a new key binding dictionary. So, I did the simple thing and just changed that dictionary during app initialization to support some new keybindings similar to the ones found in TextMate (which doesn't use NSTextView as far as I know):

NSMutableDictionary *newBindings = [[NSMutableDictionary alloc] 
    initWithDictionary:origBindings];
[[NSKeyBindingManager sharedKeyBindingManager] setDictionary:newBindings];
[newBindings release];

About as simple as you can possibly get. Unfortunately, this approach has several drawbacks primarily due to the fact that it is a global change to the entire application (which is more useful that the SYSTEM global version mostly recommended). Can we do better? Let's find out.

For a clue, let's look at one of the R GUI's own crash traces:

[NSTextView keyDown:]
...[NSView interpretKeyEvents:]
......[NSTSMInputContext interpretKeyEvents:]
.........[NSKeyBindingManager(NSKeyBindingManager_MultiClients) flushTextForClient:]

Excellent. So it looks like we'll be needing to subclass and hook into interpretKeyEvents: to get things to work properly in our system. However, we cannot simply use an NSKeyBindingManager because the manager doesn't return a BOOL so we can't tell when we haven't handled a binding. Or does it?

So, it turns out there is a MultiClient category on NSKeyBindingManager that lets us do just that using interpretEventAsCommand:forClient:

So, my interpretKeyEvents now looks like:

- (void)interpretKeyEvents:(id)sender {
 if([(NSArray*)sender count] == 1) {
  if(YES == [manager interpretEventAsCommand:[(NSArray*)sender objectAtIndex:0] 
                   forClient:self]) return;
 }
 [super interpretKeyEvents:sender];
}

So far it seems to work in testing, but I'll have to do more testing with it over time. For example, I don't know if I'll ever see more than one event in that array.

Old Things New Again: Multiple Evaluators for R Under OS X

This post is mostly the result of a conversation I had with Stefano Iacus a couple of weeks ago at WWDC. He was making the observation that a) he would like to be able to run multiple copies of R from the R GUI and b) that he would really really love to run R evaluators over XGrid.

The second one might be harder, but I think the first one can be solved. Ideally, we would simply be able to spawn off multiple R evaluators in separate threads within the R GUI and apart from synchronization problems in the GUI we would be good to go. However, I rate the chances of R becoming thread-safe (let alone supporting multiple evaluators) any time soon as "slim to none." Of course, I'm not on R Core so I could be wrong, but from what it would take (every function in every package would need an extra argument for starters) it seems unlikely. The way around this? Spawn off separate R processes and connect them up within the R GUI. This is basically what people do when they use R from Terminal so there is no real disadvantage compared to the current methods and a lot of potential advantages.

So, the plan:

1. Implement RExecServer as an LSUIElement application. As much as I'd like to use Leopard-specific features here (garbage collection in particular), people using Tiger have multiple processors too. So, we're stuck with autorelease pools for now.
a. Vend an interface as a NSDistantObject that can be picked up by the GUI
b. Provide a threaded stdin reader (if TERM is set) using the "traditional" R reader. This is to provide ESS support. I think we can actually vend the object and allow the stdin reader at the same time. Er, this could be a cool feature for something we'll talk about in October (if all goes well, this isn't my day job so it only gets implemented when I have some spare (hah) time) :-).
c. Theoretically, we could vend to/from different machines. Using Bonjour you could publish your R session. We'd have to work out some sort of security model. Not sure how that's normally handled by NSDistantObject.

2. Change the graphics device a bit. Mostly I don't think we want to ship around the graphics list. In general, we can ship a bitmap that is appropriate for the display. We can also ship back a PDF if so desired, but performance with a bitmap is likely to be higher with no discernible quality difference except in special circumstances. We can just have a [Device dataInFormat:] that sends back an NSData of the appropriate format (RGBA bitmap, PDF, etc). The nice part is that both the client and server are running OS X and both have access to the Font metrics so there doesn't need to be communication there.
a. On the ESS thing again, we can provide a simple shim device window to give ESS users a decent graphics device with full interaction. It won't be as cool as the one in the GUI, but that's what you get for using ESS ;-).

3. The GUI now maintains connections to any number of GUI console/device windows.
a. Stefano suggested being able to copy and paste between environments. I think this can be done using private Pasteboards and serializing objects to RAWSXP types, converting them to NSData and then transferring them over.
b. The GUI itself never need become unresponsive. You could even force kill a runaway server.

Now, certain things get more difficult. Certain GUI toolkits, like my own Mojave, won't be running in the GUI process anymore making them difficult to write GUIs. Of course, nobody that I know of writes GUIs using Mojave (and you'd think I'd know), but this also rules out everything else. Personally, I think the way around this is some sort of Dashboard-style interface where the front-end is implemented in HTML and Javascript with hooks back into R using a special protocol handler (which is apparently an SDK these days...) or a Javascript proxy to allow execution. I tend to favor the protocol handler.