Chronoscope on iPhone: It works! Sorta...

Rendering works fine, but navigation doesn't. The iPhone doesn't seem to give mouse move DOM events to elements, focused or not, so no matter how you drag in the browser window, you can't capture the mouse drag/move event before the iPhone Safari handles it. Bummer. I tried Chronoscope, Dojo Toolkit, GWT demos, Google Maps (maps.google.com), all have the same issues with mouse dragging.


Update: Here's code go enable mousemove events on iPhone

function init() {
var drag = document.getElementById("drag");
drag.addEventListener("mousemove", moveme, true);
drag.addEventListener("click", function(evt)
{ evt.preventDefault(); }, false);
}

<span id="drag" style="-khtml-user-drag:element;">Drag Me</span>

I don't know if the style attribute is needed, this is just what I was able to get working. The mousemove event appears, but only as a result of a click, not a drag.

-Ray

GWT Demystified: Generators Part 3, Meet the Oracle

In the Matrix movies, the Oracle told Neo exactly what he needed to hear. In this GWT Demystified, we'll explore TypeOracle(s) and what they need to tell Generators.

Recall from Part 2 of this series, that we created an interface called Exportable, and told GWT that whenever GWT.create(Exportable) is called, to execute our generator on the class.

Let's look at the guts of the generator code:

package org.timepedia.exporter.rebind;

import com.google.gwt.core.ext.Generator;
import com.google.gwt.core.ext.GeneratorContext;
import com.google.gwt.core.ext.TreeLogger;
import com.google.gwt.core.ext.UnableToCompleteException;
public class ExporterGenerator extends Generator {

public String generate(TreeLogger logger, GeneratorContext ctx,
String requestedClass)
throws UnableToCompleteException {

ClassExporter classExporter = new ClassExporter(logger, ctx);
String generatedClass = classExporter.exportClass(requestedClass);
return generatedClass;

}
}

Generators by convention are not placed in the .client package, nor .public or .server, but in a package called .rebind. They are compile time Java code, not client side Javascript, nor run-time servlet code. Generators must extend Generator and override the generate(logger, context, requestedClass) method.

TreeLogger is just a logging interface for compiler diagnostics, GeneratorContext allows access to compile time state, and the requested class is the fully qualified classname of the class that was passed to GWT.create().

In my implementation, I store this call state on another class for convenience, let's take a look at my ClassExporter.

package org.timepedia.exporter.rebind;

import com.google.gwt.core.ext.TreeLogger;
import com.google.gwt.core.ext.GeneratorContext;
import com.google.gwt.core.ext.UnableToCompleteException;
import com.google.gwt.user.rebind.SourceWriter;
import com.google.gwt.user.rebind.ClassSourceFileComposerFactory;

import java.util.ArrayList;
import java.util.HashMap;
import java.util.HashSet;
import java.io.PrintWriter;

/**
* This class performs the generation of export methods for a single class
*
* @author Ray Cromwell <ray@timepedia.org>
*/
public class ClassExporter {
private TreeLogger logger;
private GeneratorContext ctx;
private ExportableTypeOracle xTypeOracle;
private SourceWriter sw;
private ArrayList<JExportableClassType> exported;
private HashSet<String> visited;
private static final String ARG_PREFIX = "arg";

public ClassExporter(TreeLogger logger, GeneratorContext ctx) {
this(logger, ctx, new HashSet<String>());
}

public ClassExporter(TreeLogger logger, GeneratorContext ctx,
HashSet<String> visited) {
this.logger = logger;
this.ctx = ctx;
// a type oracle that can answer questions about whether types are
// exportable
xTypeOracle = new ExportableTypeOracle(ctx.getTypeOracle(), logger);

// globally, classes that have already been exported by the current generator
this.visited = visited;

// classes exported by *this* ClassExporter instance
exported = new ArrayList<JExportableClassType>();

}

Now, this looks more meaty. Before I can start to explain this code, I need to
explain the GeneratorContext.

Interfacing to the compiler

In order to get anything done with generators, you need to access some of the compiler's internal state, and you need to be able to generate source code. The GeneratorContext entry provides the ability to get Oracles, as well as create source code files on disk.

What's an Oracle? Think of it as the compile time analog of Java Reflection or Sun's Mirror API. In particular, the TypeOracle returned by GeneratorContext.getTypeOracle() is a pretty close analog to java.lang.Reflect introspection. You can query for classes, their fields, methods, return values, parameters, superclasses, subclasses, etc.

The GeneratorContext also exposes a tryCreate() method that allows one to create Java source files on disk, giving a PrintWriter in return. It returns null if the code has already been generated (generators may be invoked more than once), which is your signal to exit early. In practice, you usually don't use tryCreate() directly, but use helper classes which I'll explain shortly.

We create a class called ExportableTypeOracle whose job it is to answer questions about Exportable types, such as which methods are exportable, which parameters are exportable, what's the JavaScript name of a particular method going to be, etc.

Creating source files

Let's look at the main function, paired down a little bit:

/**
* This method generates an implementation class that implements Exporter
* and returns the fully qualified name of the class.
*
* @param requestedClass
* @return qualified name of generated Exporter class
* @throws UnableToCompleteException
*/
public String exportClass(String requestedClass)
throws UnableToCompleteException {

// JExportableClassType is a wrapper around JClassType
// which provides only the information and logic neccessary for
// the generator
JExportableClassType requestedType =
xTypeOracle.findExportableClassType(requestedClass);

// add this so we don't try to recursively reexport ourselves later
exported.add(requestedType);
visited.add(requestedType.getQualifiedSourceName());

if (requestedType == null) {
logger.log(
TreeLogger.ERROR, "Type '"
+ requestedClass + "' does not implement Exportable", null
);
throw new UnableToCompleteException();
}

// get the name of the Java class implementing Exporter
String genName = requestedType.getExporterImplementationName();

// get the package name of the Exporter implementation
String packageName = requestedType.getPackageName();

// get a fully qualified reference to the Exporter implementation
String qualName =
requestedType.getQualifiedExporterImplementationName();


sw = getSourceWriter(
logger, ctx, packageName,
genName, "Exporter");
if (sw == null) {
return qualName; // null, already generated
}

So we look up our class in the ExportableTypeOracle. If you get null, it means the class isn't Exportable. We then ask for an implementation class name ("Foo" yields class "FooExporterImpl implements Exporter") We then ask getSourceWriter to
create such a class for us on disk and return a Writer. Let's look at that method:

/**
* Get SourceWriter for following class and preamble
* package packageName;
* import com.google.gwt.core.client.GWT;
* import org.timepedia.exporter.client.Exporter;
* public class className implements interfaceName (usually Exporter) {
*
* }
*
* @param logger
* @param context
* @param packageName
* @param className
* @param interfaceNames vararg list of interfaces
* @return
*/
protected SourceWriter getSourceWriter(
TreeLogger logger, GeneratorContext context,
String packageName, String className, String... interfaceNames) {
PrintWriter printWriter = context.tryCreate(
logger, packageName, className
);
if (printWriter == null) {
return null;
}
ClassSourceFileComposerFactory composerFactory =
new ClassSourceFileComposerFactory(packageName, className);
composerFactory.addImport("com.google.gwt.core.client.GWT");
for (String interfaceName : interfaceNames) {
composerFactory.addImplementedInterface(interfaceName);
}

composerFactory.addImport("org.timepedia.exporter.client.Exporter");
return composerFactory.createSourceWriter(context, printWriter);
}

Here we use helper classes provided by GWT to create a class with a given packageName and className. We add imports, add interfaces, and then create a SourceWriter to write to the PrintWriter. SourceWriter differs primarily from PrintWriter in that it can keep track of indent level, and indent generated code.

The rest of the exportClass method looks like this:

sw.indent();

// here we define a JSNI Javascript method called export0()
sw.println("public native void export0() /*-{");
sw.indent();

// if not defined, we create a Javascript package hierarchy
// foo.bar.baz to hold the Javascript bridge
declarePackages(requestedType);

// export Javascript constructors
exportConstructor(requestedType);

// export all static fields
exportFields(requestedType);

// export all exportable methods
exportMethods(requestedType);

// add map from TypeName to JS constructor in ExporterBase
registerTypeMap(requestedType);

sw.outdent();
sw.println("}-*/;");

sw.println();

// the Javascript constructors refer to static factory methods
// on the Exporter implementation, referenced via JSNI
// We generate them here
exportStaticFactoryConstructors(requestedType);

// finally, generate the Exporter.export() method
// which invokes recursively, via GWT.create(),
// every other Exportable type we encountered in the exported ArrayList
// ending with a call to export0()

genExportMethod(requestedType, exported);
sw.outdent();

sw.commit(logger);

// return the name of the generated Exporter implementation class
return qualName;
}

This article is too big to delve into every one of this methods, so I will explain just one: exportFields, but first, you will finally have to meet the TypeOracle.

The Oracle that tells you exactly what you need

I created a helper class called ExportableTypeOracle that answers only the questions relevent for my generator:

package org.timepedia.exporter.rebind;

import com.google.gwt.core.ext.typeinfo.*;
import com.google.gwt.core.ext.TreeLogger;


public class ExportableTypeOracle {
private TypeOracle typeOracle;
private TreeLogger log;
static final String EXPORTER_CLASS =
"org.timepedia.exporter.client.Exporter";
static final String EXPORTABLE_CLASS =
"org.timepedia.exporter.client.Exportable";
private JClassType exportableType = null;
private JClassType jsoType = null;
private JClassType stringType = null;

public static final String GWT_EXPORT_MD = "gwt.export";
private static final String GWT_NOEXPORT_MD = "gwt.noexport";
public static final String JSO_CLASS =
"com.google.gwt.core.client.JavaScriptObject";
private static final String STRING_CLASS = "java.lang.String";
private static final String GWT_EXPORTCLOSURE = "gwt.exportClosure";

public ExportableTypeOracle(TypeOracle typeOracle, TreeLogger log) {
this.typeOracle = typeOracle;
this.log = log;
exportableType = typeOracle.findType(EXPORTABLE_CLASS);
jsoType = typeOracle.findType(JSO_CLASS);
stringType = typeOracle.findType(STRING_CLASS);
assert exportableType != null;
}

public JExportableClassType findExportableClassType(String requestedClass) {
JClassType requestedType = typeOracle.findType(requestedClass);
if (requestedType != null) {

if (requestedType.isAssignableTo(exportableType)) {
return new JExportableClassType(this, requestedType);
}
}
return null;
}

The first method needed is findExportableClassType. Given a request like "org.foo.bar.MyClass", it looks this type up in the underlying TypeOracle via findType(), and if found, it checks to see if the JClassType that is returned can be assigned to org.timepedia.exporter.client.Exportable. If so, this class is deemed Exportable, and it returns a wrapper around JClassType that can answer more questions about fields, methods, etc.

package org.timepedia.exporter.rebind;

import com.google.gwt.core.ext.typeinfo.JClassType;
import com.google.gwt.core.ext.typeinfo.JField;
import com.google.gwt.core.ext.typeinfo.JMethod;
import com.google.gwt.core.ext.typeinfo.JConstructor;

import java.util.ArrayList;


public class JExportableClassType implements JExportable, JExportableType {
private ExportableTypeOracle exportableTypeOracle;
private JClassType type;
private static final String IMPL_EXTENSION = "ExporterImpl";

public JExportableClassType(ExportableTypeOracle exportableTypeOracle,
JClassType type) {
this.exportableTypeOracle = exportableTypeOracle;
this.type = type;
}



public JExportableField[] getExportableFields() {
ArrayList<JExportableField> exportableFields =
new ArrayList<JExportableField>();

for (JField field : type.getFields()) {
if (ExportableTypeOracle.isExportable(field)) {
exportableFields.add(new JExportableField(this, field));
}

}
return exportableFields.toArray(new JExportableField[0]);
}

For brevity, I only show the getExportableFields() method. This method enumerates over the real ClassType.getFields() method, and for each JField entry, asks the ExportableTypeOracle if it is exportable or not. If so, it creates another wrapper around JField called JExportableField and adds it to the returned results.

But what makes a field exportable?

public static boolean isExportable(JField field) {
return field.isStatic() &&
field.isPublic() &&
field.isFinal() &&
( isExportable(field.getMetaData(GWT_EXPORT_MD)) ||
( isExportable(field.getEnclosingType()) &&
!isNotExportable(
field.getMetaData(GWT_NOEXPORT_MD))));
}

A field is exportable if and only if it is: 1) "public static final", 2) it has a @gwt.export annotation OR its class has a @gwt.export annotation and the field does not have a @gwt.noexport annotation.

Finally, let's look at the exportFields() method.

/**
* For each exportable field Foo, we generate the following Javascript:
* $wnd.package.className.Foo = JSNI Reference to Foo
*
* @param requestedType
* @throws UnableToCompleteException
*/
private void exportFields(JExportableClassType requestedType)
throws UnableToCompleteException {
for (JExportableField field : requestedType.getExportableFields()) {
sw.print("$wnd." + field.getJSQualifiedExportName() + " = ");
sw.println("@" + field.getJSNIReference() + ";");
}
}

Self-explanatory except for the getJSQualifiedExportName() and getJSNIRefernece(). The former method will return the JavaScript package + JavaScript constructor + exported name of the field. So for a field "FOO" on class "org.foo.Bar", this string might be "org.foo.Bar.FOO" if none of the exported names were overriden with an annotation.

The getJSNIReference() function will return a string like "org.foo.Bar::FOO" to reference to the public static final field of that class.

Finally, when done generating, you call SourceWriter.commit(logger), and return the name of the class you generated.

That about wraps up this edition of GWT Demystified. The GWT Exporter source and module is now available on http://code.google.com/p/gwt-exporter

-Ray

GWT Demystified: Generators Part Deux

In the last part, I described what generators are and what they can be used for. In this post, I'll be explaining how I used them to build the GWT Exporter.

GWT Exporter

The first step is to decide what your generator is going to do. In my case, I want the generator to introspect one of my GWT classes, and generate an exported JS API with bridge methods.

What's a bridge method?

Imagine you have the following GWT Class:

package org.foo.bar;
public class Util {
public static String doSomethingUseful(int x) {
return "Hello "+x;
}
}

And you want to allow JS users to call the doSomethingUseful() function?

Today, you would use JSNI to export 'bridge methods' like so:

package org.foo.bar;
public class Util {
public static String doSomethingUseful(int x) {
return "Hello "+x;
}
public native void export() /*-{
$wnd.doSomethingUseful = function(x) {
@org.foo.bar.Util::doSomethingUseful(I)(x);
}-*/;
}
}

$wnd stands for the top-level window object, and by assigning to its doSomethingUseful property, you ensure GWT won't obfuscate it. The JSNI call to Util.doSomethingUseful will be obfuscated however, thus the bridge method is neccessary to export the obfuscated symbol.

It gets more complicated if you want to bridge a non-static function, but here's an example:

public class Employee {
private String firstName, lastName;
public Employee(String firstName, String lastName) {
this.firstName = firstName;
this.lastName = lastName;
}

public String getFirstName() {
return firstName;
}

public String getLastName() {
return getLastName;
}

public static Employee createEmployee(String firstName,
String lastName) {
return new Employee(firstName, lastName);
}

public native void export() /*-{
$wnd.Employee = function(firstName, lastName) {
// call factory method and store GWT reference
this.instance =
@org.foo.bar.Employee::createEmployee(Ljava/lang/String;
Ljava/lang/String)(firstName, lastName);
}

var _=$wnd.Employee.prototype;
_.getFirstName = function() {
this.instance.@org.foo.bar.Employee::getFirstName()();
}
_.getLastName = function() {
this.instance.@org.foo.bar.Employee::getLastName()();
}
}-*/;
}

Which you may use as

x = new Employee('Ray', 'Cromwell');
alert(x.getFirstName());

As you can see, manual bridging gets tedious!

Generators to the rescue!
The first step in implementing a generator is to decide on a type that will be used to trigger the generator. So let's introduce a new marker interface called 'Exportable'. We also want the generated class to implement the Exporter interface, primarily, the export() method.

Here they are:

package org.timepedia.exporter.client.Exportable;
public interface Exportable {
}

public interface Exporter {
public void export();
}

Simple eh? Next we'll add the following line to our GwtExporter.gwt.xml module file:

<generate-with
class="org.timepedia.exporter.rebind.ExporterGenerator">
<when-type-assignable
class="org.timepedia.exporter.client.Exportable"/>
</generate-with>

What does this mean?

It means that when GWT.create() is invoked with a type that can be assigned to an Exportable, invoke the generator. That is, we want to write

public class Employee implements Exportable { ... }

Exporter x=(Exporter)GWT.create(Employee.class);
x.export();

and have it invoke the generator.

Specifying what gets exported

Next we have to decide on the rules for exporting. Which methods of an Exportable get exported? How do we control the generated JS namespace? etc. For now, let's settle on the following logic -- a method is exportable IF and ONLY IF:

1. The class enclosing the method implements Exportable
2. Metadata has determined it's ok to export (more on this later)

Also, we need error checking. It is an error to export a method if any parameter or return type is not one of:

1. a primitive type (int, float, etc)
2. another Exportable
3. an immutable JRE type (String, Integer, Double, etc)
4. JavaScriptObject

Metadata

GWT has its own form of annotations similar to JavaDoc/XDocLet tags. We will use this to control export policy. We will support two forms of export policy:

1. White List
1. By default, nothing exported.
   
2. Each method to be exported must have a "@gwt.export" metadata annotation
2. Black List
1. Place "@gwt.export" on class itself (in JavaDoc for class)
2. By default, all public methods exported
3. Each method to be removed from export consideration tagged with "@gwt.noexport"

Finally, by default, the GWT Class's package will be used as the JS's namespace, (e.g. new $wnd.org.foo.bar.Employee). If you wish to use another package for the JS export, place "@gwt.exportPackage [package1.package2...]" on the class JavaDoc.

As an example:

package org.foo.bar;
/**
* @gwt.export
* @gwt.exportPackage examples
*/
public class Employee implements Exportable {
private String firstName, lastName;
public Employee(String firstName, String lastName) {
this.firstName = firstName;
this.lastName = lastName;
}

public String getFirstName() {
return firstName;
}
/**
* @gwt.noexport
*/
public String getLastName() {
return getLastName;
}

and uses black-list policy to export getFirstName() but supress the export of getLastName(); Using white-list policy, you would write:

package org.foo.bar;
/**
* @gwt.exportPackage examples
*/
public class Employee implements Exportable {
private String firstName, lastName;
/**
* @gwt.export
*/
public Employee(String firstName, String lastName) {
this.firstName = firstName;
this.lastName = lastName;
}
/**
* @gwt.export
*/
public String getFirstName() {
return firstName;
}

public String getLastName() {
return getLastName;
}

to achieve the same effect.

At this time, we are not considering function overloading.

This concludes the specification process, and part 2. Coming up: Part 3, the guts of the generator implementation.

-Ray

GWT Canvas: Rendering rotated text

Chronoscope operates on a Canvas abstraction in GWT. Think of it as a mirror of the Safari Javascript Canvas API, with extensions for text rendering, reading back coordinate transforms, layers, fast-clear, and frame/display list capture.

A typical bit of Chronoscope Canvas code will look like:

Canvas canvas = getCanvas();
canvas.beginFrame();
Layer layer = canvas.getLayer("foo");
layer.setStrokeColor(Color.black);
layer.moveTo(sx,sy);
layer.lineTo(ex, ey);
layer.stroke();

Layer textLayer = canvas.getLayer("bar");
textLayer.setStrokeColor(Color.red);
textLayer.drawText("Hello World", x, y,
gssProperties);

canvas.endFrame();

The begin/end frame abstraction permits capture of display list for faster rendering (Opera lockCanvas), as well as shipping a scenegraph to Flash, Silverlight, or SVG. Layers permit compositing and fast-clear.

Rendering Text

Users of Javascript Canvas dream of support for text rendering. For some unknown reason, Apple (who has awesome text rendering capability in the Quartz canvas) left it out. This has forced many developers to roll their own text rendering, until the day when everyone supports the WHATWG Canvas.

Some of the typical hacks used to get text rendering include:

• Placing DIV tags over the Canvas with text
• Using pre-rendered or on-the-fly server-rendered images
• Extracting glyph vectors from true type fonts, sending them encoded to the browser, and rendering them with Canvas moveto/lineto calls.

The latter two options are the only ones capable of dealing with rotated text.

Chronoscope uses the first solution, even for rotated text, however the individual letters do not get rotated, leading to ugliness. The existing Chronoscope demo shows this on the vertical axis chart labels. Fast-clear comes into play when I want to redraw. I can blow away dozens of hundreds of DIV tags with something like "layerElem.innerHTML=''".

GWT1.4 ClippedImage and FontBook Rendering

I recently began experimenting with the idea of rendering an entire font at the desired 2D transformation and font-properties, and then using GWT1.4 ClippedImage to render letters.

A few hours later, I have an implementation that exceeded my wildest dreams in quality, here's how I did it:

1. Create a GWT RPC Service called getFontMetrics(transform, fontProperties, color)
2. Servlet calculates FontMetrics (ascent, descent, leading, advance, etc) for first 256 characters of the Font (most European languages)
3. Returned metrics includes a URL to a generated font book
   
4. FontRenderer servlet renders 16x16 array of characters with specified transform, color, font, etc
5. new drawTransformedText() method of Canvas first renders according to the old "ugly" method mentioned above and kicks off RPC and Image load
6. On load completed, "ugly" transformed text replaced with text rendered with clipped characters from the font book image

Devil's in the drawTransformedText() details

So how does this method work? The server returns a 256-element array of advance values (essentially character width), along with maxAscent, maxDescent, and leading obtained from Java2D FontMetrics. The returned RenderedFontMetrics RPC object has a method called 'getBounds(char c, Bounds b)' that for any character, returns a clipping bounding box into the second argument (avoiding excessive object creation in the main loop), which is the position of the rendered character in the font book image.

The main loop essentially obtains the bounding box for each character to be drawn, creates a ClippedImage that will display it, and positions the IMG tag along the font-baseline according to the current transform (coordinate system). It uses the 'advance' values for each character to know how far along the baseline to place each successive clipped image.

Screen Shot

Performance

Font Books can be cached, and rendering a font book on the server takes about 10 milliseconds. The resulting image is about 20kb. After that, text rendering performance is pretty much the same as the old "ugly" DIV method.

Caveats

BIDI not supported. Non-ISO-8859 character sets a pain. The issue is, I can't very well send down font-book with 65k characters. However, statistics are on our side. For example, in a language like Chinese, you'll find that most people only use a few thousand characters in day to day use.

We can compute a histogram on a corpus of Chinese text, and render a fontbook containing the most used characters in descending order. We can render several such tiles, in a monotonically descending fashion, on-demand, and cache them. In most applications, you'll probably infrequently request the second-tier characters, and very very infrequently request characters 3 standard deviations away.

Once Chronoscope is out in the wild, I hope some non-ISO-8859-1 native speakers will help contribute improved fontbook rendering for CJK characters and other languages.

p.s. I'll post a demo up on timepedia.org later, after I can verify all of the code is working properly, since I also moved the code base from 1.3 to 1.4.

-Ray

Erratum for GWT Demystified

I noticed a small mistake in the first GWT Demystified article. I was performing some code metrics today and noticed my size estimates for the source base were off by a factor of 2x. When I posted the initial article, the 30 KLOC figure seemed funny, but I let it slide because it was the right order-of-magnitude, and I thought maybe I had generated more boilerplate with my IDE than I realized. (Copy Javadoc checkbox works wonders for KLOC)

What happened is I have a Maven2 build for Chronoscope and I had executed a command like "find chronoscope -name '*.java' -exec cat '{}' \; | wc -l" without running a clean first, and therefore, all the source got counted twice because it gets copied to the 'target' build directory under my project directory as well. (You do this, because GWT artifacts must include the source code if other projects are inheriting the Chronoscope module)

The actual figures are around 15KLOC, 1.5M source, 800k byte code.

Mea Culpa,
-Ray

GWT Demystified: Generators, Part 1

Why do at runtime what you can do at compile time?

If GWT had a mantra, this would be it.

Introduction

As I mentioned in the previous article, the GWT compiler deals with a closed world -- no dynamic class loading, but it does permit deferment of binding decisions until compile time via a rule based mechanism that is part of the external GWT Module metadata. You can choose to replace a given type with other preauthored types, or, and here's the important part: you can replace types with classes that are generated on the fly.

This is the compile time equivalent of what you would do at runtime with libraries like CGLIB or the JDK's Proxy/Interceptor classes, and similar to Sun's APT (Annotation Processing Tool), except that it's all integrated into GWT and you don't need to worry about running separate APT-passes.

The deferred binding mechanism is used heavily in GWT to replace standard implementations of browser widgets with quirky implementations for browsers with divergent behavior. GWT then compiles multiple permutations of your code base, running the binding rules separately for each browser. Thus, if you have rules to target 4 different browsers, you end up with GWT compiling 4 different code bases and producing 4 compiled outputs.

Why does it do this? Because the result is more optimal code. Otherwise, you would have to include all 4 Implementation subclasses in your main code base, and use runtime logic to dynamically call the appropriate target, e.g. "if(brokenDom) DOMImplIE.foo(), else... DOMImplStandard.foo()..." It shortens load time by only forcing your browser to load what is neccessary, and it shortens run time by removing another level of indirection. Finally, it permits the optimizer to actually inline the appropriate implementation directly into the call site.

Generators

GWT also uses compile time generation of Java code to implement many platform features. The most famous of course is the RPC/Serialization mechanism. Here, GWT takes an interface, such as MyServiceAsync, and generates an implementation of this class on the fly, which contains all of the logic needed to serialize non-primitive types, send them across the wire via XMLHttpRequest, invoke the RemoteService, deserialize the rule, and invoke the async handler. It is probably the most complex generator in GWT, but not the only one.

Internationalization and Localization are also handled by the generator mechanism. Instead of loading ResourceBundles at runtime, you instead pre-process them and turn them into an interface, with one method per property. GWT will then use a generator to fill in the implementation of this interface which returns the values that are in the property file when the corresponding method is invoked. And what if you don't use some of the properties? The GWT compiler will remove any unused (uncalled) properties from the compiled application, and the number of HTTP requests is reduced because the locale data is bundled inline with your code.

Finally, truly the coolest and most innovative application of generators to date is the ImageBundle. In the world of 3D graphics programming, there is a technique called Texture Atlas, wherein you combine many textures into a single large texture, because on many graphics accelerators, changing pipeline state, such as binding a new texture before drawing geometry, is an expensive operation or may stall the pipeline. A program using texture atlases instead, binds one or more mondo-big textures, and simply uses texture coordinate manipulations to render portions of them as needed.

GWT 1.4 uses a similar technique to reduce load times and network traffic. With ImageBundle, like the I18N mechanism, you create an interface with a bunch of methods, one for each Image, as well as metadata annotations telling the GWT compiler which image file on disk you want returned by the method. Then, at compile time, the GWT compiler combines all of the images together into one large image file and generates an implementation class to return, essentially, the bounding box location of where each image is located within the overall atlas. Finally, when drawing the images, you just draw the same image (the large one) over and over, but use clipping rectangles to show only the part which corresponds to the image you need.

This reduces the number of HTTP requests drastically, speeds up startup time, and also centralizes media resources to a factory. The I18N and ImageBundle technique may even be combined to produce localized image bundles.

You're starting to imagine the possibilities?

How about buttons rendered with drop-shadows? Rounded corners done at compile time? Object-relational mapping to Google Gears or serialized RowSets? Type-safe JSON wrappers?

The project that I have created as my first generator is one designed specifically to suit my needs. Chronoscope has a JavaScript API that allows pure JS developers to access functionality provided in GWT classes. The way it does this is by creating what I call "bridge" classes and methods in the top level browser namespace. These bridge classes contain the public names (non-obfuscated) of GWT methods that will be invoked by the Javascript-to-Java JSNI Mechanism.

The problem is, everytime I added a method to one of my GWT classes, I had to go write a bridge method for it, and as Chronoscope grew in size and complexity, I needed a more automatic, and safe, mechanism for exporting an external JS interface.

In part 2 of this article, I will take you step by step, how to write this GWT Exporter, with actual code, culminating with the drop of the code on http://code.google.com/p/gwt-exporter/


-Ray

GWT Demystified

This will be a first in a series of tutorials and essays digging deeper into some of the more esoteric (but extremely important!) functionality of the Google Web Toolkit. (This article is written by Ray Cromwell, CTO) I will be talking about some of the nitty gritty details of the GWT compiler, how to create your own custom generators and the cool stuff you can do with them, and finally, how to plugin new optimization algorithms into the GWT compiler itself.

The Google Web Toolkit can be considered a package of three fundamental technologies: The Java to Javascript Compiler, the runtime library/APIs, and the Hosted Mode browser. All three are very important components of GWT in their own right, but because the APIs and the Hosted browser are the most visual pieces (you can SEE THEM operating) they tend to have more impact on people, while the real heavy duty stuff being done by the GWT compiler goes mostly unnoticed.

This is why I am not surprised that many language zealots engaged in language wars often appear grimaced when I tell them I am rewriting my Javascript code in GWT. "Huh!? Why!?" Then the inevitable signs of misunderstanding crop up: "Didn't applets show running Java in the browser is slow?", "Doesn't GWT produce bloated code?", etc. Now, I am by no means a zealot for the Java language, but there does appear to be frequent misunderstandings about what the GWT Compiler is, and does.

The GWT Compiler is a real compiler. Not a simple translator that walks an abstract syntax tree turning Java expressions and statements into the nearest Javascript equivalents. In fact, the GWT Compiler performs optimizations that are very hard to do statically on raw Javascript, and hard to do even in regular Java.

For example, static dead code elimination is very hard to do safely in Javascript, and limited in Java as well (at the bytecode level). You can never really be sure that a public method in Java won't be called, because of dynamic class loading, and frankly, you can't even be sure that private methods won't be invoked due to reflection and interception techniques, thus the only safe way to do dead code elimination is to defer it to runtime and let Hotspot deal with it.

In contrast, GWT has very good information on which methods are reachable, and is extremely aggressive at removing unused methods, saving both bandwidth and start up time. The closest equivalent in the Java world would be MIDLets, since the same sorts of closed-world assumptions can be made.

GWT does more than just remove dead code. It performs inlining, polymorphic-to-monomorphic call conversion (devirtualization), a form of type-inferencing which GWT calls Type-Tightening (GWT can infer that a field of type Animal is only ever assigned type Cat. In fact, it can infer that such a field is always null!), and lots of other little tricks. It doesn't appear to have common subexpression elimination, copy propagation, or in-block dead code elimination yet, but in my third tutorial, I will demonstrate a simple naive way to achieve this.

What this means is that in GWT, you don't pay much for what you don't use, and you don't have to worry about figuring out what's used. It means GWT's Javascript output is not bloated. It has a constant factor overhead related to the bootstrap process, but as your applications grow bigger, this code becomes smaller.

The GWT compiler already does a good job producing compact javascript code (Chronoscope is 30,000 lines of Java, 3.2 Megabytes of source, 1.9 Megabyte of compiled byte code, and 137k of Javascript after GWT 1.4 compiles it, and 45k after gzip -9), and there is a lot of headroom still left in terms of optimization that it can do.

So why do I use GWT? Besides the fact that we want to run in environments that don't have Javascript, GWT's Hosted Mode is extremely helpful providing full access to the universe of Java debuggers, and automatic code completion and popup Javadoc is very nice. (Even though I use IntelliJ IDEA, which already has good Javascript code completion and popup documentation)

GWT is a good platform for building AJAX/RIA applications and allows one to easily port or repurpose existing Java codebases and tools -- another good option for developers. It has a bright future ahead of it.

-Ray
Coming up: GWT Generators HOW-TO: generate classes on the fly at compile time.

Timepedia @ Google Developer Day

We had a blast at Google Developer Day, especially meeting and hanging out with the Google Web Toolkit team. Google's organization of the event was excellent, and they are to be commended for staging a world wide event (free) for developers, providing ample (and good) food and snacks, as well as busing developers to the Google campus for an after hours party. We've been to freebie events before, and they haven't been this good.

At Dev Day, Timepedia launched a preview of one of our components: Chronoscope, which is our visualization platform for timeseries data written in GWT.

We got alot of positive feedback, and had fun hearing people try and guess what our t-shirts meant. Actually, we were surprised how many people got very close. Timepedia has been our personal hobby, our passion, for over two years, and we are both nervous and excited to finally reveal it to the public.

As we come closer and closer to launch time, we'll post developer notes, stumbling blocks, hurdles leaped, and give more information about some of the other pieces of Timepedia (such as codenames: Tardis, Timelord, Geisser, Everett, and Jarocki :) )