Thoughts on an iPad R (or similar device)

This has come up a few times on various lists I read and I have a strong opinion, so I figured I'd jot down some notes. The topic of discussion is the notion of R running on an Apple iPad, the inevitable Android Pads or the HP Slate (assuming the thing ever makes it out the door). Maybe even some sort of Windows 7 Mobile device.

There are a couple of immediate hurdles that come to mind:

* Limited RAM. The iPad has 256MB, the iPhone 4 has 512MB. A standard R compute box sports, what, 4GB at a minimum these days? Sure, back when I started using R I had maybe 512MB or 1GB of ram on my laptop, but I also had a lot less data back then.

* Battery. It doesn't really matter how fast the processors in these things are, the big problem is battery. Batteries aren't really getting better, the 10 hour battery life in modern laptops comes from a) bigger batteries (this is real reason you can't remove the battery from an Apple laptop anymore---doing that lets you wedge a bigger battery into the box without weight gain) and b) aggressive power management for the CPU.

* Apple prohibits interpreters outside of very specific use cases (games, Javascript). This is an Apple specific thing, but it's worth mentioning as there are now a lot of iPads out there. (Also Fortran is an issue under the new rules.)

For those reasons, I don't think there should be an R that runs directly on your Pad device the same way it does on your desktop. You don't have enough RAM and if you crank up the processor enough you'll destroy your battery life. Just look at what your favorite game does to your battery and remember that it gets to use specialized hardware for its processing.

So what should we do instead? Those who've known me for a while (they probably don't read this, but whatever) will remember StatPaper and the "Kernel" R hobby projects I've had at various times. Both operated on the premise that the R engine itself should run in a separate process from the UI and interact with the user through some sort of remote connection. This was inspired by Mathematica in the case of StatPaper and by the desire to use the multiple cores found in most machines through a single GUI in the case of Kernel R (which let you spin up multiple R consoles and transfer data between them).

I don't really work on either anymore, I have a day job and don't really program recreationally anymore (not that I dislike programming, it's just that I have work problems that provide a focus for learning new technologies and noodling around so I tend to go there first). But I do still like the idea of them and I think they could work for R. Here's I think what looks like a plan.


* Move away from the notion of R on a filesystem. R should really be living in something like a database, probably an object database. One option might be some of the new-ish document-oriented databases like CouchDB or MongoDB. Both would let you serialize R objects into documents using something that looked a lot like a standard environment.

* These databases should also live in a cloud environment (Cloudy-R! Eh? I know you like it). Having some sort of standard serialization that wasn't R specific could also let you have other applications updating data and using results. This lets R focus one what it's good at without being forced to do something it isn't good at (like web UIs).

* Use the same serialization format as the command format for a remote R UI. The serialization format should ALSO encode the graphics device.


Once you have those things you have the core of what you need for your Pad R. You have an R that exists in an always running state (because you could serialize the entire R environment to your database when the environment is idle. If you build it carefully you could probably even eliminate the need for a sticky session but that would likely involve some rummaging about in the internals).

On the client side, you can start simple with the familiar console interface then you can start to experiment with other interface modalities. Perhaps some sort of direct manipulation of a graphics object for exploratory data analysis.

Fascinating Captain...

I recently learned that Snow Leopard includes a (public!) framework called IOSurface that allows for the sharing of basically OpenGL textures between processes. That neatly solves the problem with having your graphics device running in a separate process for the GUI. Previously, I've shipped bitmaps around between the GUI and the backing process (which is headless) but this could definitely be a performance improvement both in update speed and in memory. I'll have to look into it.

I suspect even RGL could be made to operate in this mode though you'd need some help from the package since it expects to run an event loop for handling mouse interaction with the scene graph.

Aw, nuts.

Upgraded to Snow Leopard and so it is time once again to make some changes to my personal R GUI--if for no other reason than to test cool stuff like Blocks and Grand Central Dispatch. It also looks like there's some interesting developments in Pasteboards and a few other under the hood places.

First off are Blocks. They look like function pointers but they definitely aren't. They're actually Objective-C classes that you can treat like functions. Very Smalltalk, complete with similar tricks to stuff their data onto the stack you explicitly move them to the heap (they're objects so you can copy them).

Unfortunately, they are not interchangeable with C function pointers so you can't just use them as callback functions. A shame really, since right now I have a libffi-based binding between the R callback functions (ReadConsole, WriteConsole, etc) to thunk to Objective-C methods. All hope is not lost, the R binding is not so complicated so writing a Block compatible version shouldn't be too hard.

Seen in a report about the G1...

Says Tim Bray about the G1's browser (compared to the iPhone's Safari browser):

All they need to catch up is some fit-and-finish and that little pinch/unpinch trick for zooming.

Yes, well... Horseshoes, hand grenades and thermonuclear devices and all that. Seriously, the only thing that really sets one operating system apart from another IS fit and finish. Something the Linux On The Desktop (or laptop for that matter) have never really figured out. Aping pieces of Windows and OS X (I notice modern distros seem to have BOTH a start bar AND a global menu bar... why?) will never get you fit and finish. Some poor bastard (bastards most likely) sitting down and smoothing the hell out of the rough interface edges is what's going to get you there.

Ah, interesting....

It looks like someone is finally doing a commercial R of some sort (and congrats on the series A to anyone who may work there) over at REvolution Computing (and thanks to Ben for sending me Google alert). I always sort of wondered why Insightful didn't try to wire up their GUI stuff from S-PLUS to the R engine, aside from maybe some sort of licensing concern. Though it looks like a lot of people involved are ex-Insightful the website seems to indicate more of a focus on support rather than front end--I'll be very curious to see how that plays out.

No official Mac support though, so it's unlikely to affect my life for the time being. ;-)

Excellent

The feature freeze for R 2.7.0 is in place with an expected April 22nd release. I've noticed some of the locale stuff has changed recently (LC_CTYPE and the like) so I'll need to fix those warning messages. Right now I'm setting up the IKImageBrowserView-backed graphics device interface for the GUI, though the built-in quartz device has been serving me well so far. There are a couple of reasons for the view set up, which uses bitmaps to represent the graphics device on the backend. After the page is completed, the bitmap is converted to a more efficient representation (e.g. PNG) for temporary storage and the display list recorded into the device. We'll also continue to cheat and provide a Cover Flow mode (no, I don't care that it's a private API).

Another interesting feature is that, unlike the current Mac GUI, the graphics device is considered a View on the same Model (the interpreter). We'll see how it plays out, but that might help with some of the fancy footwork usually required when dealing with the NSDocumentController.

Implementing XCode 3 parenthesis highlighting

One of the things I like about XCode 3 is the parenthesis matching. It seems very obtrusive at first, but I find it works better than the more subtle Emacs-style matching once you get used to it. For the R GUI I wanted to implement the same thing, which turned out to be fairly simple.

The highlighter is implemented using the new [NSTextView showFindIndicatorInRange:] so there's literally no work to do there.

The first thing you need to know is that adding this functionality in textView:shouldChangeTextInRange:replacementString: won't work because the text is added after the method executes and that removes the find indicator. Fortunately, we just implemented a custom keybinding in the last post so all we need to do is bind the various close delimiters to special selectors (fancyCloseParen: for example). Then we can do things in doCommandBySelector:

// ...
} else if(@selector(fancyCloseParen:) == aSelector) {
  [self insertDelimiter:')' withMatch:'(' inTextView:aTextView atPosition:aRange.location];
  return YES;
} // ...

next all we have to do is find the delimiter (which we do with a category on NSString) and then insert the find indicator:

- (void)insertDelimiter:(unichar)aDelim withMatch:(unichar)aMatch inTextView:(NSTextView*)aTextView atPosition:(NSInteger)aPos {
  [aTextView insertText:[NSString stringWithFormat:@"%c",aDelim]];
  NSRange where = [[aTextView string] rangeOfDelimiter:aMatch matching:aDelim inRange:NSMakeRange(lastPrompt,aPos-lastPrompt)];
  if(where.location == NSNotFound)
   NSBeep();
  else
   [aTextView showFindIndicatorForRange:where];
}

Easy as pie.

On Key Bindings

For my R GUI I want user customizable key bindings. I know for a fact that there are customizable key bindings built into the Cocoa text system, but they aren't user accessible and the only response from Apple was a message posted to a mailing list about 2 years ago to the effect of "I have something." Two years later, well, surprise, surprise. So what to do?

Well, for the first pass (found in the R-Multi git repository) I class-dumped AppKit and found NSKeyBindingManager. It turns out that this class has a shared initializer and a means for setting a new key binding dictionary. So, I did the simple thing and just changed that dictionary during app initialization to support some new keybindings similar to the ones found in TextMate (which doesn't use NSTextView as far as I know):

NSMutableDictionary *newBindings = [[NSMutableDictionary alloc] 
    initWithDictionary:origBindings];
[[NSKeyBindingManager sharedKeyBindingManager] setDictionary:newBindings];
[newBindings release];

About as simple as you can possibly get. Unfortunately, this approach has several drawbacks primarily due to the fact that it is a global change to the entire application (which is more useful that the SYSTEM global version mostly recommended). Can we do better? Let's find out.

For a clue, let's look at one of the R GUI's own crash traces:

[NSTextView keyDown:]
...[NSView interpretKeyEvents:]
......[NSTSMInputContext interpretKeyEvents:]
.........[NSKeyBindingManager(NSKeyBindingManager_MultiClients) flushTextForClient:]

Excellent. So it looks like we'll be needing to subclass and hook into interpretKeyEvents: to get things to work properly in our system. However, we cannot simply use an NSKeyBindingManager because the manager doesn't return a BOOL so we can't tell when we haven't handled a binding. Or does it?

So, it turns out there is a MultiClient category on NSKeyBindingManager that lets us do just that using interpretEventAsCommand:forClient:

So, my interpretKeyEvents now looks like:

- (void)interpretKeyEvents:(id)sender {
 if([(NSArray*)sender count] == 1) {
  if(YES == [manager interpretEventAsCommand:[(NSArray*)sender objectAtIndex:0] 
                   forClient:self]) return;
 }
 [super interpretKeyEvents:sender];
}

So far it seems to work in testing, but I'll have to do more testing with it over time. For example, I don't know if I'll ever see more than one event in that array.

Dear iPhone SDK haters,

Nobody is forcing you to develop for the iPhone. If you don't like it, by all means, go develop for 'Android' and be sure to let me know how that works out for you.

A little FFI goes a long way

As my two faithful readers have surely noted, one of my little projects is a Leopard-specific GUI for R that tries to leverage as many of the new features as possible. As it happens, one of those features is Garbage Collection, which I can't use because of the way the R framework is built. Perhaps I'll be able to use Simon's method for building self-contained R applications with the framework built in the appropriate way.

What I'd really like to talk about though is the fact that having PyObjC and whatever the Ruby bridge is called means that I can be positive that a copy of libffi is available on every Leopard system. This let me finally build something I've wanted for a long time: an elegant C<->Cocoa thunk for the R function pointers. To wit:

ptr_R_WriteConsoleEx = ffi_bind(self,@selector(writeConsole:length:type:),&ffi_type_void,
  4,&ffi_type_pointer,&ffi_type_sint,&ffi_type_sint);

So, how do we do it? Well, first we need a little bit of context to keep a binding between a function pointer and a specific object/method combination. We also keep the types of arguments around as well as the return type. If we were doing a complete libffi bridge we would introspect this from the class at runtime rather than the static method I use here. In the case of R this would be overkill since the R function pointers are very much fixed entities. In any case, here's the little structure I use:

typedef struct {
 id  obj;
 SEL sel;
 IMP fn;
 int nargs;
 ffi_type *retval;
 ffi_cif  *cif;
 ffi_type *types[0];
} ffi_call_struct;

Next, we need to write a little function to use as a closure and move the C function call to the ObjC function call. All libffi closure handlers look have the same function declaration:

void handler(ffi_cif *cif,void *retval,void *args[],void *user_data) {

First we build a place to hold our arguments

 int i;
 ffi_call_struct *call = (ffi_call_struct*)user_data;
 void **values = (void**)malloc(sizeof(void*)*(call->nargs+2));

and simply transfer the arguments accross, making sure that the first two are the target object and target selector (Obj-C methods have two implicit arguments):

 for(i=0;inargs;i++) { values[2+i] = args[i]; }
 values[0] = &(call->obj);
 values[1] = &(call->sel); 

Then all we do is use ffi_call to make a call. Note that we use the NSAutoreleasePool method, which wouldn't be necessary if we could enable garbage collection:

 NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];
 ffi_call(call->cif,(void (*)(void))call->fn,retval,values);
 [pool release];
 free(values);
}

Next we need a little function to build an appropriate closure function to use as the R function pointer:

void *ffi_bind(id obj,SEL selector,ffi_type *retval,int nargs,...) {
 int i;
 ffi_cif *cif;
 ffi_closure *closure;
 va_list ap;

We use vargs so that we can define the argument lists inline. First, we allocate our context structure and transfer the argument types

 //Allocate the structure and copy the call information into the structure.
 ffi_call_struct *call = (ffi_call_struct*)malloc(sizeof(ffi_call_struct)+(nargs+2)*sizeof(ffi_type*));
 call->obj = obj;
 call->sel = selector;
 call->fn  = [obj methodForSelector:selector];
 call->retval = retval;
 call->nargs = nargs;

 call->cif = (ffi_cif*)malloc(sizeof(ffi_cif));
 cif = (ffi_cif*)malloc(sizeof(ffi_cif));
 
 call->types[0] = &ffi_type_pointer;
 call->types[1] = &ffi_type_pointer;
 va_start(ap,nargs);
 for(i=0;itypes[2+i] = va_arg(ap,ffi_type*);

Next we define the two ffi_cifs. The first one is the ffi_cif of the Obj-C method call and the second is the one for the function pointer we're trying to create. Finally, we create the closure and return it

 ffi_prep_cif(call->cif,FFI_DEFAULT_ABI,nargs+2,call->retval,call->types); //The CIF for the ObjC Method
 ffi_prep_cif(cif,FFI_DEFAULT_ABI,nargs,call->retval,&(call->types[2]));   //The CIF for the function call
 closure = (ffi_closure*)malloc(sizeof(ffi_closure));
 ffi_prep_closure(closure,cif,handler,call);
 return closure;
}

and that's pretty much all there is to it. There are two minor caveats, the first is that anything that employs method swizzling on this class is likely to fail since we cache the function pointer at bind time. Fortunately, swizzling is relatively rare these days so it generally doesn't happen. It may also technically be slower than the hand-coded method, but there are a couple of areas where we could probably speed up the thunking operation---the allocation of

values

in particular could be moved to the context object.

And the bloggers chime in...

So, the iPhone SDK was announced (good luck with actually downloading it) and I'm looking forward to playing around with it (no, I'm not going to try to get R running. Let's face it, the iPhone UI is more like, say, Data Desk than anything else). I am getting annoyed with assorted bloggers and pundits bitching about perceived shortcomings.

First, the 70/30 split. That seems to be the biggest complaint. In the words of Dr. Evil, "quite typical, really." Jobs is probably right when he says that it's just enough to cover expenses. Credit card processing ain't free, kids. They might see a little net revenue, but it's in the noise relative to the rest of their business. Basically, apps are a way to sell iPhones, and it'll work.

I also saw someone complain about not being able to change the iPhone UI L&F. Well, yeah. Duh. Apple has seen X11 and knows exactly what happens when you allow that sort of control. Basically, rip-offs of the Windows 95 UI (No, don't expect me to be impressed that you can map Windows 95 on to the sides of a cube and spin it around. At the end of the day you've still got Windows 95 with uglier controls).

Speaking of UI and complete non sequiturs (well, not completely since Salesforce presented in the iPhone thing). I ran across an O'Reilly book this evening at Barnes & Noble called "Designing Dashboards," except that the title was longer. For statisticians with at least a mild interest in presenting results there's nothing there that comes as a surprise (and several areas that could have used some help, color palette selection for example). Clearly the author has read Tufte's books (sparklines even made an appearance). Mostly, though, I'm struck by how absolutely horrible the types of dashboard produced by things like Business Objects and Salesforce (there are others, but these are really popular). The outputs are very expensive in terms of real estate (lots of round things, pie charts and dials and such. Usually in some godawful 3d rendering) and deliver very little, usually a single number. Even when they become more traditional/less junky, the little things really start to stand out. There's a lot of "stop light color" usage, which seems like a good idea but really just induces fatigue (I tried this recently in my own workflow tool and could only stand to use the tool for about 10 minutes. The same dimension is now represented by a much more subtle change in glyph size, which is surprisingly easy to assess at a glance). Even when they move away from stop-light colors there's no rhyme or reason to the color choices nor any apparent internal consistency.

Anyway, I'll probably write more on what I'm learning about visual display of complex information from the workflow tool I've been developing as a place to hang my modeling hat for the last couple of months. I've got a stable core set of users and I use the tool myself literally daily (usually the development version which varies somewhat from the 'production' version). I've also got some things to say about R and databases and workflow in that environment as well.

Long time, no post.

So, a comment to my Cover Flow post yesterday reminded me that I haven't posted anything here in quite some time despite making a lot of progress on an R GUI over the Summer and the release of Leopard at the end of October. I have an excuse though! See, in mid-October I left academia and started working for a company in San Francisco called AdBrite. I'm pretty happy with things so far, I get to do statistics and program basically as much as I want. There were a bunch of factors that contributed to the switch from academics, most of which I'm not going to discuss on a public blog, though I'm happy to correspond via email. Long story short, AdBrite made me an offer I couldn't refuse (and it had nothing to do with the Nolan Lab, Garry and all the folks in the lab are great). The whole NIH obligation thing could become a problem (this is an area where cross discipline researchers really get screwed), but I'll deal with it as it comes, I try to think of it as a really ill-advised student loan.

Anyway, this means I haven't had much time for the R Mac GUI in the last several months as most of the work I've been doing has been more server-side rather than client-side so I haven't seen much of the R GUI and when I do see it, it happens to be on a Windows box---we have SAS and SAS notably is not available for OS X. I suppose I could Parallels it, but I'd still be running Windows and the Win64 box was already here.

I'd like to get back to it, but we'll have to see. I think at the very least I'll get the code off my backup drive and put the source code up somewhere so it isn't lost to the ages. On the bright side, a lot of the graphics drawing code made it in the core R release for 2.7.0 and the API Simon chose to adopt is fairly similar so a bunch of the graphics code can actually be removed. Simon also put paging into the core graphics device, though it's not as fun as Cover Flow or the Image Browser version (though those are Leopard specific so he can't really put it into the core R).

I think there's some useful stuff in the Console implementation, particularly the clipboard support and the improved completion/hinting support. I like Simon's idea of having a built-in console with the ability to spawn external consoles at will, there are some things (RGL comes to mind) that didn't like the RExecServer setup very much.

I'll try to post more often and get that source up somewhere. :-)

R 2.7.0 Quartz graphics device clipboard support

Tom Elliot recently posted a request to the R-SIG-Mac list looking for a way to programmatically put graphics onto the clipboard under OS X. Right now this is pretty hard, but I was inspired to patch R-devel's (the future 2.7.0) new Quartz device to allow for clipboard output.

After applying the patch, you can specify file="clipboard://" and when the device is closed (yes, you MUST close the graphics device to see output. It would probably be a good idea to write the file on a NewPage as well, come to think of it). Then you can wrap up a simple function to get copying of the current device to the clipboard (or you can simply open it directly).

copy.to.clipboard = function(dpi=300) { dev.copy(device=quartz,type="png",file="clipboard://",dpi=dpi);dev.close(); }

I've attached the patch for the brave souls willing to try it out:

Index: qdBitmap.c
===================================================================
--- qdBitmap.c  (revision 43081)
+++ qdBitmap.c  (working copy)
@@ -49,16 +49,45 @@
         /* On 10.4+ we can employ the CGImageDestination API to create a
            variety of different bitmap formats */
 #if MAC_OS_X_VERSION_MAX_ALLOWED >= MAC_OS_X_VERSION_10_4
-        CFURLRef    path  = CFURLCreateFromFileSystemRepresentation(kCFAllocatorDefault,(const UInt8*)qbd->path,strlen(qbd->path),FALSE);
-        CFStringRef type  = CFStringCreateWithBytes(kCFAllocatorDefault,(UInt8*)qbd->uti,strlen(qbd->uti),kCFStringEncodingUTF8,FALSE);
-        CGImageDestinationRef dest = CGImageDestinationCreateWithURL(path,type,1,NULL);
-        CGImageRef image = CGBitmapContextCreateImage(qbd->bitmap);
-        CGImageDestinationAddImage(dest,image,NULL);
-        CGImageDestinationFinalize(dest);
-        CFRelease(image);
-        CFRelease(dest);
-        CFRelease(type);
+        CFStringRef pathString = CFStringCreateWithBytes(kCFAllocatorDefault,(UInt8*)qbd->path,strlen(qbd->path),kCFStringEncodingUTF8,FALSE);
+               CFURLRef path;
+               if(CFStringFind(pathString,CFSTR("://"),0).location != kCFNotFound) {
+               CFStringRef pathEscaped= CFURLCreateStringByAddingPercentEscapes(kCFAllocatorDefault,pathString,NULL,NULL,kCFStringEncodingUTF8);
+                       path = CFURLCreateWithString(kCFAllocatorDefault,pathEscaped,NULL);
+                       CFRelease(pathEscaped);
+               } else {
+                       path = CFURLCreateFromFileSystemRepresentation(kCFAllocatorDefault,(const UInt8*)qbd->path,strlen(qbd->path),FALSE);
+               }
+               CFRelease(pathString);
+               
+               CFStringRef scheme = CFURLCopyScheme(path);
+               CFStringRef type  = CFStringCreateWithBytes(kCFAllocatorDefault,(UInt8*)qbd->uti,strlen(qbd->uti),kCFStringEncodingUTF8,FALSE);
+       CGImageRef image = CGBitmapContextCreateImage(qbd->bitmap);
+               if(CFStringCompare(scheme,CFSTR("file"),0) == 0) {
+               CGImageDestinationRef dest = CGImageDestinationCreateWithURL(path,type,1,NULL);
+               CGImageDestinationAddImage(dest,image,NULL);
+               CGImageDestinationFinalize(dest);
+               CFRelease(dest);
+               } else if(CFStringCompare(scheme,CFSTR("clipboard"),0) == 0) {
+                       //Copy our image into data
+                       CFMutableDataRef      data = CFDataCreateMutable(kCFAllocatorDefault,0);
+                       CGImageDestinationRef dest = CGImageDestinationCreateWithData(data,type,1,NULL);
+                       CGImageDestinationAddImage(dest,image,NULL);
+                       CGImageDestinationFinalize(dest);
+                       CFRelease(dest);
+                       PasteboardRef pb = NULL;
+                       if(noErr == PasteboardCreate(kPasteboardClipboard,&pb)) {
+                               PasteboardClear(pb);
+                               PasteboardSyncFlags syncFlags = PasteboardSynchronize(pb);
+                               PasteboardPutItemFlavor(pb,(PasteboardItemID)1,type,data,0);
+                       }
+                       CFRelease(data);
+               } else
+                       warning("Not a supported scheme, no image data written.");
+               CFRelease(scheme);
+               CFRelease(type);
         CFRelease(path);
+               CFRelease(image);
 #endif
     }
     /* Free ourselves */

Those of your tracking R-devel may have noticed...

...that there was recently a big change to the way the Quartz device works. Turns out, Simon and I had roughly the same idea about a Quartz 2D implementation of the Quartz device for reasons of speed and the combined result (I ran "patch" on my R-devel tree and then Simon did all of the hard work :-) ) is in there now. This means the RExecServer and other R GUIs can now all use the same drawing backend for a variety of sources. I believe, for people who don't want to use RExecServer, that Simon also put in a CGLayer-based Cocoa target that runs an event loop in much the same way as RExecServer to give you interactive graphics natively in v2.7 (I'm guessing, since the feature freeze has passed). It also means native access to all of the Quartz 2D bitmap generation goodness and a lot more speed for ALL of the graphics device implementations.

Simon also went through and added some other features as well, including support for non-square pixels and some scaling things that I never got working correctly.

So, things have been moving along even though it may appear that I dropped off the face of the planet for a while. I'm going to be updating RExecServer soon, which means a dependence on R-devel, to use the converged device and it's new API (which is also available to any GUI developer).

Not Dead

Just busy trying to get some software sorted out before feature freeze. Turns out to be a lot of work when you break R. :-)

Cover Flow!

Yup. Figured it out. One of two ways to browse display plots. There is a more useful mode, using an iPhoto-like Browser that also lets you edit the list of plots. The arrow keys also let you move between plots.

iWork '08 Spelunking.

Fiddling around with iWork '08 a bit and Numbers support is trivial to add to the R GUI. Basically, it exports a tab delimited version as text, much like Excel, though it uses the more modern pasteboard type (where Excel exports a much less useful string). We can detect that it is numbers from the metadata and "native" pasteboard types and throw up the same dialog as for the Excel pasting. Yay!

I'm thinking, for symmetry, that I should let you right click on a data frame in the workspace and copy as a tab delimited type for pasting into Excel/Numbers as well.

OLPC Musing

So, I've been playing with the OLPC a bit (I got a chance to play with a prototype in person at a party a while back) and I wonder: where's the science/mathematics? Authoring languages and EToys, which seem to be the focus presently (well, that and Tetris of course) are all well and good, but it would be really nice to see some more focus on the physical sciences, which leads naturally into topics like data analysis and statistics (collecting data is just the first step after all).

The minimal implementation would seem to be to start with something like the TILE/StatDocs projects that Duncan Temple Lang (Duncan, I hearby dub you DTL for the remainder of my posts. Sort of like DHH, the Ruby on Rails guy) and Deb Nolan have been doing for a while now (there used to be a StatDocs.org website, but it seems to have disappeared). They give you an organizational unit of a "Lab" or an "Experiment," with some data and ways of interacting with the data (plotting and so on). These'd probably be hosted on the school's server and brought across the network dynamically. Another interesting tack would be to let the server also COLLECT data from the kids' experiments to be combined together into larger datasets via surveys or actual data collection. An ad hoc version of this is pretty common in introductory statistics courses for example.

This leads to the question of what to use as the analytical environment. Being biased, I would tend towards using R + GTK + Gecko, with a UI designed for the more modal experience of the OLPC. Alternatively, Python could be used for the implementation, though I'm not convinced that Python would be a particularly good interactive data language (things like R/S's formula DSL are really quite powerful for example). I tend to dislike point-n-click on principal, especially in an environment that stressed authoring languages.

Beyond that, you want kids to be able to collect their own data. I remember having this thing for my Atari 800 when I was a kid that had a number of different probe options for things like temperature and pH and rainfall and such. The software it had was pretty primitive, but it was lots of fun (for me anyway) playing with the temperature probe and such. It seems like the USB ports could be used to build data acquisition systems pretty cheaply these days---you wouldn't need really high performance for the most part and that lets kids collect their own data, which will always be more compelling than some prepackaged dataset since they "own" the data. There's at least one, possibly more, Open Source hardware project for building lots of these sorts of sensors for DIY weather monitoring that can probably be leveraged for that sort of thing.

RExecServer can load help

It's not the best implementation in the world---I still think that should take place at the R level not in the GUI implementation---but on the plane to Seattle I gave RExecServer the ability to load help via the pager instead of just complaining.

Using RExecServer from ESS

In the long term I hope that most of this can be configured by an Installer of some sort, but for those wanting to use RExecServer with ESS, here's how my setup is currently configured:

When I use Emacs at all (which is very rarely these days, I mostly use TextMate) I use Aquamacs, which ships with ESS installed. Hopefully the instructions won't be all that different for people using Carbon Emacs and what have you (and if they are, perhaps people would be so kind as to post their changes). I'm also not a super-sophisticated ESS user so there might be better ways of doing the ESS side of things.

First, though, we need to get ourselves easy access to RExecServer from the command line. I like to do this via a symbolic link in /usr/local/bin (/usr/bin would also work). You'd think that we could just symlink RExecServer.app/Contents/MacOS/RExecServer and be done with it, but the way OS X starts applications is... uh... strange and doing that will cause all sorts of problems because the bundle loader won't be able to find any resources and promptly crash out. It's not pretty. What we need to do is actually set a special environment variable called CFProcessPath to point to the application bundle rather than /usr/local or whatever.

Fortunately, I made this into a little script so you won't have to do it yourself. It lives in the Resources folder of the application bundle. So, from Terminal we would do something like:

$ cd /usr/local/bin
$ sudo ln -s /Applications/RExecServer.app/Contents/Resources/RExecServer.sh R-exec

if you have put RExecServer.app somewhere other than Applications, use that path instead of "/Applications." That's pretty much all there is to it. Of course, "/usr/local/bin" should be in your path

$ R-exec

R version 2.6.0 Under development (unstable) (2007-07-14 r42234)
Copyright (C) 2007 The R Foundation for Statistical Computing
ISBN 3-900051-07-0

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

  Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> 

You should be able to use R-exec from Terminal just like normal R. For ESS, I have a line in my .emacs file:

(require 'ess-site)
(setq inferior-R-program-name "R-exec")

that starts my RExecServer version of R instead of the normal version. Once upon a time, I think I read that ESS is smart enough to detect multiple installed versions of R with specially named symlinks, but I don't know how to get it to work (if an ESS expert knows how to do this, please chime in).

In any case, that's pretty much all there is to it. Hopefully that helps, but if you have questions please email me or (better yet so others can see) leave a comment.