Spring cleaning and the blog

To try and improve readability and reach, I made a few changes today:

1) Re-enabled SyntaxHighlighter for code snippets
2) Widened the width of the main content to 600px (for code blocks)
3) removed Twitter and added Google Friend Connect
4) enabled CAPTCHA to reduce comment span
5) tweak some colors so the subsections are easier to read.

If you have any other advice for improving the Look and Feel, leave a comment.

-Ray

Structural Typing for GWT and Javascript

In a previous post I described a type of impedance mismatch between Javascript and Java idioms that makes the GWT Exporter still less than ideal for supporting Javascript programmers. A short illustrative (but contrived) example:

public class Customer implements Exportable {
   private String firstName, lastName;
   @Export String getFirstName() { 
      return firstName; 
    }

   @Export void setFirstName(String fn) {
      firstName = fn;
   }

   @Export String getLastName() { 
      return lastName; 
    }

   @Export void setLastName(String ln) {
      lastName = fn;
   }
}

Today, you may use this export in Javascript like this:

var cust = new Customer();
cust.setFirstName("Ray");
cust.setLastName("Cromwell");
Processor.doSomething(cust);

The example is contrived because you could use a constructor, but with more complex objects with nested types, you wouldn't use a constructor, but either a builder pattern, or inject the types after construction.

Javascript developers however don't work in the world of Javabean interfaces, they prefer easy construction of configuration/builder information via object literals:

Processor.doSomething({firstName: "Ray", lastName: "Cromwell"})

Moreover, when passing in say, a bind of event callbacks, they'd prefer to write:

foo.addEvents({
    click: function(e) { ... }
    move: function(e) { ... }
    drop: function(e) { ... }
});

The challenge is to seemlessly bridge this idiomatic mismatch between JS and Java GWT code without actually having to write much bridging code or adapters.

Structural Typing

Java is a manifestly typed language. All types have to be declared, and type checking is done by explicit hierarchy. Javascript is a dynamically typed language, with essentially no type checking at all. Orthogonal to this is the concept of Structural Typing. Haskell, ML, and Scala are all examples of languages which support structural typing. Structural typing was also planned for Javascript 2 before it got killed.

So what's a structural type? Recall the Customer example from the previous section. It was a class with two fields, firstName, and lastName, both Strings. If Java supported a structural type system, I could declare a method in two ways:

public void process(Customer cust) { ... }

public void process({firstName: String, lastName: String} cust) { ... } 

In this invented syntax, the cust parameter to the second process() function is an anonymous type, we don't know its real name. However, we are stating that as long as it consists of two fields named "firstName", and "lastName", and the types are both Strings, then we can access these fields and treat it like a Customer (although it may not be one)

Hmm....I smell an idea...

Structural Type Exports

What if I rewrite the Customer POJO class with a @StructuralType annotation:

@Export
@StructuralType
public class Customer implements Exportable { ... }

The dispatch code for an exported process() function could then look like this:

$wnd.Processor.prototype.process = function(cust) {
  if(cust instanceof $wnd.Customer && isExportedInstance(cust)) {
    // JSNI dispatch to @Processor::process(LCustomer;)(unwrap(cust));
   }
  else if(typeof(cust) == 'object') {
      // cust is not an instance of the Customer POJO, but an object
      if(cust.firstName != undefined && cust.lastName != undefined) {
          var scust = new Customer();
          scust.setFirstName(cust.firstName);
          scust.setLastName(cust.lastName); 
          @Processor::process(unwrap(scust));
      }
   }
}

The GWT compiler would auto-inject this structural type check and initialization code simply by annotating a parameter or return type involved with @StructuralType

Refinements to the idea

Taking this a step further, one could override the expected type literal field names to be checked

@SType("fn")
public void setFirstName(String firstName) { ... }
@SType("ln")
public void setLastName(String lastName) { ... }

which would allow the object literal to be specified as {fn: "Ray",ln: "Cromwell"}. Another extension would allow partial matches to succeed with default values supplied:

@Optional("Ray")
public void setFirstName(String firstName) { ... }

which would reduce the structural type to just a lastName field for matching purposes, but would allow the specification of firstName to be supplied and injected into the setter as "Ray" if it wasn't present.

Structurally Typed Interfaces

GWT Exporter already supports closure conversion for single-method Java interfaces. That is, if the following interface:

public interface MyCallback {
     public void go(Object arg);
}

occurs as a type parameter in an exported method, the Javascript developer may supply function(arg) { ... } and the GWT generated bridge code will automatically convert this into a GWT object instance that implements the MyCallback interface type. What it cannot support is the example given earlier:

foo.addEvents({
    click: function(e) { ... }
    move: function(e) { ... }
    drop: function(e) { ... }
});

which would represent something like:

public interface MyEventHandler {
    public void onClick(Event e);
    public void onMove(Event e);
    public void onDrop(Event e);
}

However, using structural typing conventions, and an @StructuralType annotation on an interface, we can auto-convert object literals containing multiple function closures into a Java interface for GWT.

I am still working out the details of the full overload resolution algorithm for overloaded function types, but some version of this proposal will make it into GWT Exporter 3.0.

-Ray

GWT Exporter 2.05 released

The release adds two new features:

1) arrays are return types or parameters supported, that is:

@Export public double[] getFoo() { }
@Export public SomeExportable[] getBar() {}
@Export public void setFoo(double[] foo) { }
@Export public void setBar(SomeExportable[] bar) { }

now works, in both Web mode and Hosted Mode.

2) Export Overlays implemented. This means you can now Export classes you don't own.

@ExportPackage("gwt")
public interface WindowExport extends ExportOverlay<Window> {
   @Export void alert(String s);
}

will export the GWT Window class into the namespace $wnd.gwt.Window, and export the alert() method of this class.

The JARs are downloadable from: http://timefire-repository.googlecode.com/svn/mavenrepo/org/timepedia/exporter/gwtexporter/2.05/

What's next for GWT Exporter?

I have learned over the past year that a lot of Chronoscope users like to use the Javascript API, but the current API is unfortunately, a small sliver of the full power the GWT API offers. In dealing with increasing demand for more exported functions, I have run into some basic limitations that GWT-Exporter has that make it less than ideal for exporting idiomatic Javascript APIs.

Exporting Classes you don't control

Since Chronoscope made the switch to the new GWT Incubator/GWT 1.6 Event handling system, I have been unable to export event handler functions, because the HandlerRegistration type returned by the GWT event system is not an interface, making it difficult to wrap. In general, if you don't own a class, you can't add Exporter annotations to it, which is problematic because there are lots of types you don't control.

A new mechanism will be added in an upcoming GWT Exporter release, I dub Export Overlays, which will work something like this:

@ExportPackage("gwt")
public interface WindowExport extends ExportOverlay<Window> {
    @Export void alert(String message);
}

In the above code, we are exporting the GWT Window class in the Javascript package "gwt", but only exposing the alert() method. This will result in an exported wrapper in $wnd.gwt.Window with the method published on $wnd.gwt.Window.prototype.alert.

Fine control over Javascript->Java type conversions

Javascript doesn't support overloaded method dispatch as a language feature, but most Javascript libraries do use heavily overloaded functions with variable arguments and variable argument types. Currently, GWT Exporter supports Javascript overloading by arity, but not by type, so two Java methods of the same name and identical arity cannot be exported without overriding the exported name.

Some of this will be mitigated by adding in automatic type checking for handling overloaded dispatch, but more is needed. Consider the following Javascript API:

plot.addHandler({
    onMove: function(event) {  }
    onHover: function(event) { }
    onFocus: function(event) { }
});

This is somewhat idiomatic Javascript, leveraging object literals to pass in multiple bindings. The current approach is to have 3 methods, addMoveHander, addHoverHandler, addFocusHandler, but this doesn't feel quite right to some Javascript programmers. Another problem is illustrated below:

@Export
public class Foo implements Exportable {
   public void setFoo(Date date) { }
}

A Javascript developer would naturally want to write:

var date = new Date();
foo.setFoo(date);

But this won't work, because there is no automatic conversion in GWT Exporter between JS Date and java.util.Date. I could add this (as was done with JS String and java.lang.String), but that still leaves open the problem of the inability to pass Javascript types from third party Javascript libraries into GWT methods.

Type Converter injection

What I'm currently looking at is a mechanism to declare Javascript fragments on Java methods which can type-convert arguments between the exported wrapper, before they invoke the underlying GWT method. Something like this:

@Export
public class Foo implements Exportable {
  public void setFoo(@Convert(JsType.DATE, "new java.util.Date($_arg.getTime()$)") Date foo) { }   
}

The syntax is still subject to change of course, but the idea is that you supply an annotation on each param, using an enum (STRING, FUNCTION, ARRAY, OBJECT, etc) which corresponds to the return value of the 'typeof' operator, or you use a special INSTANCE version which allows you to specify a match according to 'instanceof'. Next, you supply a Java expression), including inline JSNI code (here shown in between $ $). The GWT Exporter then arranges to generate code to check the type, and inject your fragment before calling the underlying method.

Another possibility is to add JSNI methods to Export Overlays which allow arbitrary conversion code to be specified and injected into the wrapper prior to invocation of methods.

By combining this with the Export Overlay concept, it might be possible to deal with the idiomatic JS Literal pattern, while keeping JSNI methods out of classes that may not want them, or that you do not control.

Primitive arrays an oversight

Finally, GWT Exporter currently does not support export of parameters or return types which are native Java arrays, which is a hole in the export type system. This will be added in a near future release.

-Ray

Small demo of Timelord

Timefire is building a set of social collaboration and visualization tools to help people understand data better, and part of this has been the development of Chronoscope/Timescope, but an even larger part of this effort has been the development of an end user application that permits users to play with data, annotate it, and share their activity with friends.

Timelord is the codename of our application, which runs as an embeddable widget, a Google Gadget, a Spreadsheet Gadget, or as a native Android application. It leverages OpenSocial when possible, to allow users to share activities, but also integrates with other contact sources, like GData Contacts API, Android address book, etc.

One of the more important features of Timelord is the ability for users to create their own "mini-presentations" which script tours of the data. These mini-presentations are simply a special comment syntax, like Wiki-text or BBCode, to instruct the chart to perform actions which would normally be done programmatically, like adding markers, or zooming.

A cool feature, I think, is the ability to do 2-way synchronization between these mini-presentations and YouTube video, which is demonstrated in the screencast below, courtesy of our friend Al Gore, and the YouTube Player API.

Take it away, Mr Gore (best if watched in HD):


Watch it in HD here

UPDATE: YouTube seems to have disappeared my HD version, so there is an HD version mirrored at http://vimeo.com/3292149

I don't think I have to mention that this is all done in GWT? :)

-Ray

GWT Exporter now generates Javascript documentation

I added a rough implementation of a Javadoc Doclet (my first!) to the exporter trunk. It is an extension of the Standard Doclet, so it not only generates Javadoc for your GWT classes, but automatically emits a file called "jsdoc.html" that includes documentation for every @Export in your source base. This allows you to automatically produce documentation for Javascript programmers from GWT source. I use the Maven Javadoc Plugin to execute this, but you can use ant tasks or the command line as well.

It's quite ugly at the moment, no CSS stylesheet, but I'm working on a basic one. I'm happing to take any CSS donations from those who want to check out the trunk and play with it.

Code project is : http://code.google.com/p/gwt-exporter/

-Ray

Screencast of Timescope, the commercial version of Chronoscope

Hi Folks,
I've uploaded a screencast of Timescope, which is the commercial version of Chronoscope, to Vimeo. This is a sneak peak at some of the features in the upcoming release of Timescope, as well as Chronoscope 1.0. The chief differences between Timescope and Chronoscope, are a more accurate multiresolution algorithm that preserves data better when zooming out, new marker and renderer types (CandleStick charts for example), new marker types like Image, Axis, and Callout markers, iPhone support, a native Android version, as well as a Chart Server.

The screencast is at: http://vimeo.com/2983154, I'd embed it here, but it has to be watched in HD in order to see what's going on.


-Ray