Hardcore GWT Hands on Training?

It's the new year, and I've been mulling an idea recently that I'd like to get some feedback on from the community.

The GWT Conference was a blast, and it appears my Deferred Binding presentation was well received. Since then, a number of people have encouraged me to offer some sort of training or mentoring for GWT, either in classrooms, or in the form of corporate on-site training. I hesitate to get too distracted from working on Chronoscope and Timepedia, but after thinking about it some more, I do think it might be worth pursuing, perhaps one 2-day course per month.

What I don't want to do is some token GWT course that glosses over the core of GWT and shows one how to create some cute widgets and stuff them into a web page, without understanding what's going on under the hood. Rather, I'd like to teach, on Day 1, GWT from the ground up exploring in detail what is happening: Java vs JS Output, Deferred Binding, JSNI, Widget attachment, Widget event processing, RPC/Serialization vs On-the-wire representation, the Bootstrap/Selection process, Hosted Mode, debugging, testing, etc.

On Day 2, I'd then delve into more idiomatic patterns: Creating custom widgets, creating your own modules/libraries, Localization, image bundles, integrating with back-end frameworks, RPC and RPC customization, optimizing size and speed, etc

I'd require people to write code in short labs during the class, give personal help in troubleshooting, setting up Eclipse/IDEA/Netbeans environments, and ask that others who have completed labs successfully help participate in helping those having trouble, because I believe a good way to learn something is by in-class participation and trying to help someone else. Who knows, it might be a fun, campfire type atmosphere!

I'd risk losing a lot of newbies doing this, and also risk not getting any advanced users who are self-taught, but hopefully, there are a lot of people in the middle with some GWT experience, who would like a more in depth tutorial hands on. Perhaps I'd do a more marketing oriented/newbie introductory course later.

Ultimately, I'd hope that people who come away from the classes with not only enough knowledge to write code to cover their own use cases, but also have enough knowledge to fix bugs and contribute to GWT itself. The GWT compiler and core libraries should no longer be 'magic' to those taking the course.

Obviously, the course wouldn't be free, especially in offsite training where I'd have to rent the event space and also given the opportunity cost for me, but I would offer discounts if space is donated, charge less for the first 'beta' class, and given some steep student discounts, especially if students would arrive early and help me with class setup.

I'm working out the course material at the moment, and pricing out some meeting spaces. I haven't decided on a per-student price yet, but feel free to email me suggestions, course ideas, and most importantly, whether or not you'd be interested in attending, or your company would. Private replies can be sent to cromwellian / gmail.com

Oh, this would be for the SF Bay Area, hopefully BART/Caltrain accessible.

-Ray

Designing a really simple GWT Accordion control

Yesterday, we started implementing some neat context-sensitive information features to Timepedia's "Timelord" GWT application, which integrates Chronoscope, Everett, and several other backend services, and I found myself in great need of an Accordion-like control.

Now, there are a great many accordion controls in third party JS libraries, many of them very slick and full featured, but I found that I didn't need so many features, and I'm trying to limit the number of pure-JS library wrappers I use in GWT because you lose alot of the benefits of the GWT Compiler when you wrap external JS libraries.

How hard could it be to cook up one in GWT? It turns out, not very hard it all.

Concept

What I want is a VerticalPanel or HorizontalPanel with the ability to open/close the nested widgets, preferably in a cool animated style. That means, we'll create either an Vertical or Horizontal Panel, store it in a Composite, and use even-numbered widget positions to store a Label, which when clicked, will expand odd-numbered widget positions from 0 to their desired width or height. We'll also simultaneously close any currently expanded widgets, back down to zero width/height. The only trick is to figure out what expanded width/height of a widget is.

Implementation

First, let's create a Composite to house a Vertical or Horizontal Panel.

public class AccordionPanel extends Composite {
private Panel aPanel;
private String animField;
private String animBounds;

final private static int NUM_FRAMES = 8;

private Widget currentlyExpanded = null;
private Label currentlyExpandedLabel = null;

public AccordionPanel(boolean horizontal) {
if (horizontal) {
aPanel = new HorizontalPanel();
animField = "width";
animBounds = "scrollWidth";
} else {
aPanel = new VerticalPanel();
animField = "height";
animBounds = "scrollHeight";
}
initWidget(aPanel);

setStylePrimaryName("accordion");
}

public AccordionPanel() {
this(false);
}

I don't really need a horizontal accordion, but I threw in support for it (untested) just in case. Here, if we are using a Vertical accordion, we are animating the height field, and the desired expansion size is in the "scrollHeight" field.

Next, we need a method for adding new widgets to an accordion.

public void add(String label, final Widget content) {
final Label l = new Label(label);
l.setStylePrimaryName(getStylePrimaryName()+"-title");
final SimplePanel sp=new SimplePanel();
sp.setWidget(content);

l.addClickListener(new ClickListener() {
public void onClick(Widget sender) {
expand(l, sp);
}
});
aPanel.add(l);
sp.setStylePrimaryName(getStylePrimaryName()+"-content");
DOM.setStyleAttribute(sp.getElement(), animField, "0px");
DOM.setStyleAttribute(sp.getElement(), "overflow", "hidden");
aPanel.add(sp);
}

Given a string label, and a widget, we do two things. First, we create a Label widget, and add a ClickListener to it. The ClickListener simple calls the expand() function with the label, and the content. Secondly, we wrap the content being added in a SimplePanel, to ensure we get a DIV around it which can be animated.

Finally, we add both the Label and the content wrapper to the underlying Vertical or Horizontal panel, set the primary style name of the label/content to be that of the AccordionPanel + "-title"/"-content", and then collapse the newly added component (height/width: 0, overflow: hidden)

We're almost done

Now all we have to do is implement the expand() function. We essentially want to run an animation loop, linearly interpolating between 0 and the max dimensions (for expansion) or the opposite for collapse.

Let's look at the implementation:

private void expand(final Label label, final Widget c) {

if(currentlyExpanded != null)
DOM.setStyleAttribute(currentlyExpanded.getElement(),
"overflow", "hidden");

Here we make sure that any previous expanded section is changed from overflow: auto back to overflow: hidden. Continuing further...

final Timer t = new Timer() {
int frame = 0;

public void run() {
if (currentlyExpanded != null) {
Widget w = currentlyExpanded;
Element elem = w.getElement();
int oSh = DOM.getIntAttribute(elem, animBounds);
DOM.setStyleAttribute(elem, animField, ""+(( NUM_FRAMES -
frame ) * oSh / NUM_FRAMES)+"px");

}

We create a Timer, whose run method will animate the collapse/expansion. We simultaneously collapse any currently expanded Widget and expand the target widget. All we have to do, is fetch the max dimensions from scrollHeight/scrollWidth, and then interpolate them down to zero for collapse.

if (currentlyExpanded != c) {
Widget w = c;
Element elem = w.getElement();
int oSh = DOM.getIntAttribute(elem, animBounds);
DOM.setStyleAttribute(elem, animField, ""+
(frame * oSh / NUM_FRAMES)+"px");
}
frame++;

Likewise for expansion, we simply interpolate from 0 to the maximum dimensions. Finally, we increment the frame, and we take care not to try and collapse and expand the same widget at the same time.

if (frame <= NUM_FRAMES) {
schedule(10);
} else {
if(currentlyExpanded != null) {
currentlyExpanded.removeStyleDependentName("selected");
currentlyExpandedLabel.removeStyleDependentName("selected");
}
c.addStyleDependentName("selected");
if(currentlyExpanded != c) {
currentlyExpanded = c;
currentlyExpandedLabel = label;
currentlyExpandedLabel.addStyleDependentName("selected");
Element elem = c.getElement();
DOM.setStyleAttribute(elem, "overflow", "auto");
DOM.setStyleAttribute(elem, animField, "auto");
} else {
currentlyExpanded = null;
}

}

Finally, we keep rescheduling our timer as long as we haven't reached our maximum number of frames. To finish, we add a final bit of polish, by adding a "selected" CSS class to the Label which is currently expanded. We also change the width/height and overflow fields back to auto.

}
};
t.schedule(10);

The last lines of the expand() function simply kick off our timer.

Here is the simple GWT Module to test

public class AccordionDemo implements EntryPoint {

/**
* This is the entry point method.
*/
public void onModuleLoad() {
AccordionPanel ap=new AccordionPanel();
AccordionPanel ap2=new AccordionPanel();

ap.add("Label 1", new HTML("Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Nam luctus urna vitae urna. Sed nisl. Praesent nisi nulla, malesuada quis, bibendum quis, egestas nec, pede. Donec ipsum. Duis nulla nisi, tristique eget, fermentum et, gravida non, lorem. Praesent mollis, arcu sed suscipit venenatis, velit erat sollicitudin quam, eget vestibulum odio enim nec libero. Praesent tellus. Vestibulum non justo. Aliquam semper. Nulla mauris ipsum, semper ut, dapibus quis, ultrices nec, est. Mauris nec nisl ut est posuere dignissim. Sed nec magna non purus eleifend mollis. Pellentesque orci. Integer sapien. Cras aliquam."));
ap.add("Label 2", new HTML("Ut tristique convallis nibh. Vestibulum eget nunc eget tellus varius sollicitudin. Vestibulum vestibulum ligula ac nulla. Nulla risus urna, euismod eget, accumsan vitae, posuere sit amet, eros. Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Aenean rhoncus pellentesque justo. Sed purus diam, bibendum ut, posuere a, feugiat sit amet, justo. Donec urna nulla, blandit in, gravida non, mattis a, turpis. Phasellus feugiat leo et justo. Maecenas quam nisl, consectetuer nec, auctor et, pharetra vel, tortor. Nunc justo nulla, tincidunt a, tempor id, viverra eget, sem. Vestibulum ante ipsum primis in faucibus orci luctus et ultrices posuere cubilia Curae; Proin sagittis nonummy urna. Aenean quis massa ac massa rutrum ornare."));


ap2.setStylePrimaryName("accordion2");
ap2.add("Nested Label 1", new HTML("Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Nam luctus urna vitae urna. Sed nisl. Praesent nisi nulla, malesuada quis, bibendum quis, egestas nec, pede. Donec ipsum. Duis nulla nisi, tristique eget, fermentum et, gravida non, lorem. Praesent mollis, arcu sed suscipit venenatis, velit erat sollicitudin quam, eget vestibulum odio enim nec libero. Praesent tellus. Vestibulum non justo. Aliquam semper. Nulla mauris ipsum, semper ut, dapibus quis, ultrices nec, est. Mauris nec nisl ut est posuere dignissim. Sed nec magna non purus eleifend mollis. Pellentesque orci. Integer sapien. Cras aliquam."));
ap2.add("Nested Label 2", new HTML("Ut tristique convallis nibh. Vestibulum eget nunc eget tellus varius sollicitudin. Vestibulum vestibulum ligula ac nulla. Nulla risus urna, euismod eget, accumsan vitae, posuere sit amet, eros. Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Aenean rhoncus pellentesque justo. Sed purus diam, bibendum ut, posuere a, feugiat sit amet, justo. Donec urna nulla, blandit in, gravida non, mattis a, turpis. Phasellus feugiat leo et justo. Maecenas quam nisl, consectetuer nec, auctor et, pharetra vel, tortor. Nunc justo nulla, tincidunt a, tempor id, viverra eget, sem. Vestibulum ante ipsum primis in faucibus orci luctus et ultrices posuere cubilia Curae; Proin sagittis nonummy urna. Aenean quis massa ac massa rutrum ornare."));
ap.add("Nested", ap2);

ap.add("Label 3", new HTML("Neque porro quisquam est qui dolorem ipsum quia dolor sit amet, consectetur, adipisci velit..."));



RootPanel.get("slot1").add(ap);

}

}

and the CSS rules used.

.accordion-title {
background: #EFEFE3 url(gray-bg.gif) repeat-x scroll 0%;
border: 0pt none;
padding: 3px;
color: white;
font-weight: bold;

}

.accordion-title-selected {
background: coral none;
}

.accordion {
width: 300px;

}

.accordion2-title {
background: #EFEFE3 url(gray-bg.gif) repeat-x scroll 0%;
border: 0pt none;
padding: 3px;
padding-left: 10px;
color: white;
font-weight: bold;

}

.accordion2-title-selected {
background: blueviolet none;
}

.accordion2 {
width: 300px;

}

-Ray
p.s. caveats. First, scrollHeight/scrollWidth don't always work. See quirksmode. Secondly, there is a chance you could click another label while an expansion is in progress, so you'll need to add an "isAnimating" flag so that expand() will do nothing if another expansion is in progress. Third, you may want ease-in/ease-out in the animation, so instead of a linear interpolation, drop in your favorite easein/out function (sine, 2x^3-3x^2, etc)

When algorithms work better than you dream...

So, Timepedia is building a time machine, right? It sounds pretentious, but for us, it's really a geeky moniker of love for our project, after all, is Google's "search engine" really an "engine"? How much horsepower does it have? :)

One part of Timepedia, readers of this blog are already familar with: Chronoscope. With Chronoscope, we are attempting to build an open platform of visualization tools for time oriented data, in much the same way that Google Maps and Google Earth deal with spatial data.

However, what good is a time machine, if you don't know where to go, or don't understand what you're looking at? Timepedia has another platform, aimed at data mining time related information, called Everett (owned and implemented by another Timepedia founder, Mat). Everett is a collection of many algorithms for both data mining, and forecasting, some of them bleeding edge academic research. When we started, we weren't sure which of them would work, or how well they would work, we only knew that they had promising features, so Everett was less of a end user product, and more of a research platform.

One of the tools of Everett is an algorithm that lets us find hidden recurring patterns in data, even in the presence of noise, or scaling. Last week, we tested the algorithm on real life data for the first time, and had one of those "holy cow!" moments, which don't occur too often for me personally, where your own code surprises you.

To give you an example, I fed Everett an 18,000 data point series of federal funds rates over the last few decades, and it identified a pattern that occured 3 times in history. Visualizing this in another tool we call Timelord (A Chronoscope married to Everett and other server-side services), I was puzzled as to the significance of these three sequences. My co-founder Shawn spent about 1 hour Googling, until he found the correlation: These sequences corresponded to international financial/currency crises (such as the Mexican currency crises), in which the Fed was forced to take action. The leadup to the crises appeared identical each time. A fluke? It sure the hell was very interesting.

I was worried it was a fluke, so I tried something more mundane. A time series of unemployment benefit expenditures in Indiana, and once again, Everett identified a series of puzzling repetitive sequences. What were they? The dates looked very familar, 1980-81, 1990-91, 2000-1...were they recessions? To check, I used Timelord to overlay a National Bureau of Economic Research official measure of economic expansions and contractions, and sure enough, these patterns intersected with NBER recessions. One other interesting property stood out, the patterns returned prefixed the recessions, that is, Everett was showing us a pattern that leads to a recession.

How cool is that? Ambition got the best of me, I went for broke: I tried a historical time series of average hurricane strength (saffir-simpson scale), as well as a yearly count. There appears to be good evidence that a 40-60 cyclical hurricane season exists, and I was hoping that Everett could find these patterns, but alas, it did not.

Still, the initial results are promising, and we hope that Everett will give average users an ability to query time in ways that have not been previously available.

So, if you're wondering why I haven't released Chronoscope yet, it's because I've been working on integrating Timelord with Everett. :)

-Ray
p.s. Timelord is another GWT application, making it our 4th major GWT application. Everett is C++ coupled via JNI a Java/GWT RPC interface, since performance is absolutely critical in Everett.

GWT Demystified: Generators Part Deux

In the last part, I described what generators are and what they can be used for. In this post, I'll be explaining how I used them to build the GWT Exporter.

GWT Exporter

The first step is to decide what your generator is going to do. In my case, I want the generator to introspect one of my GWT classes, and generate an exported JS API with bridge methods.

What's a bridge method?

Imagine you have the following GWT Class:

package org.foo.bar;
public class Util {
public static String doSomethingUseful(int x) {
return "Hello "+x;
}
}

And you want to allow JS users to call the doSomethingUseful() function?

Today, you would use JSNI to export 'bridge methods' like so:

package org.foo.bar;
public class Util {
public static String doSomethingUseful(int x) {
return "Hello "+x;
}
public native void export() /*-{
$wnd.doSomethingUseful = function(x) {
@org.foo.bar.Util::doSomethingUseful(I)(x);
}-*/;
}
}

$wnd stands for the top-level window object, and by assigning to its doSomethingUseful property, you ensure GWT won't obfuscate it. The JSNI call to Util.doSomethingUseful will be obfuscated however, thus the bridge method is neccessary to export the obfuscated symbol.

It gets more complicated if you want to bridge a non-static function, but here's an example:

public class Employee {
private String firstName, lastName;
public Employee(String firstName, String lastName) {
this.firstName = firstName;
this.lastName = lastName;
}

public String getFirstName() {
return firstName;
}

public String getLastName() {
return getLastName;
}

public static Employee createEmployee(String firstName,
String lastName) {
return new Employee(firstName, lastName);
}

public native void export() /*-{
$wnd.Employee = function(firstName, lastName) {
// call factory method and store GWT reference
this.instance =
@org.foo.bar.Employee::createEmployee(Ljava/lang/String;
Ljava/lang/String)(firstName, lastName);
}

var _=$wnd.Employee.prototype;
_.getFirstName = function() {
this.instance.@org.foo.bar.Employee::getFirstName()();
}
_.getLastName = function() {
this.instance.@org.foo.bar.Employee::getLastName()();
}
}-*/;
}

Which you may use as

x = new Employee('Ray', 'Cromwell');
alert(x.getFirstName());

As you can see, manual bridging gets tedious!

Generators to the rescue!
The first step in implementing a generator is to decide on a type that will be used to trigger the generator. So let's introduce a new marker interface called 'Exportable'. We also want the generated class to implement the Exporter interface, primarily, the export() method.

Here they are:

package org.timepedia.exporter.client.Exportable;
public interface Exportable {
}

public interface Exporter {
public void export();
}

Simple eh? Next we'll add the following line to our GwtExporter.gwt.xml module file:

<generate-with
class="org.timepedia.exporter.rebind.ExporterGenerator">
<when-type-assignable
class="org.timepedia.exporter.client.Exportable"/>
</generate-with>

What does this mean?

It means that when GWT.create() is invoked with a type that can be assigned to an Exportable, invoke the generator. That is, we want to write

public class Employee implements Exportable { ... }

Exporter x=(Exporter)GWT.create(Employee.class);
x.export();

and have it invoke the generator.

Specifying what gets exported

Next we have to decide on the rules for exporting. Which methods of an Exportable get exported? How do we control the generated JS namespace? etc. For now, let's settle on the following logic -- a method is exportable IF and ONLY IF:

1. The class enclosing the method implements Exportable
2. Metadata has determined it's ok to export (more on this later)

Also, we need error checking. It is an error to export a method if any parameter or return type is not one of:

1. a primitive type (int, float, etc)
2. another Exportable
3. an immutable JRE type (String, Integer, Double, etc)
4. JavaScriptObject

Metadata

GWT has its own form of annotations similar to JavaDoc/XDocLet tags. We will use this to control export policy. We will support two forms of export policy:

1. White List
1. By default, nothing exported.
   
2. Each method to be exported must have a "@gwt.export" metadata annotation
2. Black List
1. Place "@gwt.export" on class itself (in JavaDoc for class)
2. By default, all public methods exported
3. Each method to be removed from export consideration tagged with "@gwt.noexport"

Finally, by default, the GWT Class's package will be used as the JS's namespace, (e.g. new $wnd.org.foo.bar.Employee). If you wish to use another package for the JS export, place "@gwt.exportPackage [package1.package2...]" on the class JavaDoc.

As an example:

package org.foo.bar;
/**
* @gwt.export
* @gwt.exportPackage examples
*/
public class Employee implements Exportable {
private String firstName, lastName;
public Employee(String firstName, String lastName) {
this.firstName = firstName;
this.lastName = lastName;
}

public String getFirstName() {
return firstName;
}
/**
* @gwt.noexport
*/
public String getLastName() {
return getLastName;
}

and uses black-list policy to export getFirstName() but supress the export of getLastName(); Using white-list policy, you would write:

package org.foo.bar;
/**
* @gwt.exportPackage examples
*/
public class Employee implements Exportable {
private String firstName, lastName;
/**
* @gwt.export
*/
public Employee(String firstName, String lastName) {
this.firstName = firstName;
this.lastName = lastName;
}
/**
* @gwt.export
*/
public String getFirstName() {
return firstName;
}

public String getLastName() {
return getLastName;
}

to achieve the same effect.

At this time, we are not considering function overloading.

This concludes the specification process, and part 2. Coming up: Part 3, the guts of the generator implementation.

-Ray

GWT Canvas: Rendering rotated text

Chronoscope operates on a Canvas abstraction in GWT. Think of it as a mirror of the Safari Javascript Canvas API, with extensions for text rendering, reading back coordinate transforms, layers, fast-clear, and frame/display list capture.

A typical bit of Chronoscope Canvas code will look like:

Canvas canvas = getCanvas();
canvas.beginFrame();
Layer layer = canvas.getLayer("foo");
layer.setStrokeColor(Color.black);
layer.moveTo(sx,sy);
layer.lineTo(ex, ey);
layer.stroke();

Layer textLayer = canvas.getLayer("bar");
textLayer.setStrokeColor(Color.red);
textLayer.drawText("Hello World", x, y,
gssProperties);

canvas.endFrame();

The begin/end frame abstraction permits capture of display list for faster rendering (Opera lockCanvas), as well as shipping a scenegraph to Flash, Silverlight, or SVG. Layers permit compositing and fast-clear.

Rendering Text

Users of Javascript Canvas dream of support for text rendering. For some unknown reason, Apple (who has awesome text rendering capability in the Quartz canvas) left it out. This has forced many developers to roll their own text rendering, until the day when everyone supports the WHATWG Canvas.

Some of the typical hacks used to get text rendering include:

• Placing DIV tags over the Canvas with text
• Using pre-rendered or on-the-fly server-rendered images
• Extracting glyph vectors from true type fonts, sending them encoded to the browser, and rendering them with Canvas moveto/lineto calls.

The latter two options are the only ones capable of dealing with rotated text.

Chronoscope uses the first solution, even for rotated text, however the individual letters do not get rotated, leading to ugliness. The existing Chronoscope demo shows this on the vertical axis chart labels. Fast-clear comes into play when I want to redraw. I can blow away dozens of hundreds of DIV tags with something like "layerElem.innerHTML=''".

GWT1.4 ClippedImage and FontBook Rendering

I recently began experimenting with the idea of rendering an entire font at the desired 2D transformation and font-properties, and then using GWT1.4 ClippedImage to render letters.

A few hours later, I have an implementation that exceeded my wildest dreams in quality, here's how I did it:

1. Create a GWT RPC Service called getFontMetrics(transform, fontProperties, color)
2. Servlet calculates FontMetrics (ascent, descent, leading, advance, etc) for first 256 characters of the Font (most European languages)
3. Returned metrics includes a URL to a generated font book
   
4. FontRenderer servlet renders 16x16 array of characters with specified transform, color, font, etc
5. new drawTransformedText() method of Canvas first renders according to the old "ugly" method mentioned above and kicks off RPC and Image load
6. On load completed, "ugly" transformed text replaced with text rendered with clipped characters from the font book image

Devil's in the drawTransformedText() details

So how does this method work? The server returns a 256-element array of advance values (essentially character width), along with maxAscent, maxDescent, and leading obtained from Java2D FontMetrics. The returned RenderedFontMetrics RPC object has a method called 'getBounds(char c, Bounds b)' that for any character, returns a clipping bounding box into the second argument (avoiding excessive object creation in the main loop), which is the position of the rendered character in the font book image.

The main loop essentially obtains the bounding box for each character to be drawn, creates a ClippedImage that will display it, and positions the IMG tag along the font-baseline according to the current transform (coordinate system). It uses the 'advance' values for each character to know how far along the baseline to place each successive clipped image.

Screen Shot

Performance

Font Books can be cached, and rendering a font book on the server takes about 10 milliseconds. The resulting image is about 20kb. After that, text rendering performance is pretty much the same as the old "ugly" DIV method.

Caveats

BIDI not supported. Non-ISO-8859 character sets a pain. The issue is, I can't very well send down font-book with 65k characters. However, statistics are on our side. For example, in a language like Chinese, you'll find that most people only use a few thousand characters in day to day use.

We can compute a histogram on a corpus of Chinese text, and render a fontbook containing the most used characters in descending order. We can render several such tiles, in a monotonically descending fashion, on-demand, and cache them. In most applications, you'll probably infrequently request the second-tier characters, and very very infrequently request characters 3 standard deviations away.

Once Chronoscope is out in the wild, I hope some non-ISO-8859-1 native speakers will help contribute improved fontbook rendering for CJK characters and other languages.

p.s. I'll post a demo up on timepedia.org later, after I can verify all of the code is working properly, since I also moved the code base from 1.3 to 1.4.

-Ray

Erratum for GWT Demystified

I noticed a small mistake in the first GWT Demystified article. I was performing some code metrics today and noticed my size estimates for the source base were off by a factor of 2x. When I posted the initial article, the 30 KLOC figure seemed funny, but I let it slide because it was the right order-of-magnitude, and I thought maybe I had generated more boilerplate with my IDE than I realized. (Copy Javadoc checkbox works wonders for KLOC)

What happened is I have a Maven2 build for Chronoscope and I had executed a command like "find chronoscope -name '*.java' -exec cat '{}' \; | wc -l" without running a clean first, and therefore, all the source got counted twice because it gets copied to the 'target' build directory under my project directory as well. (You do this, because GWT artifacts must include the source code if other projects are inheriting the Chronoscope module)

The actual figures are around 15KLOC, 1.5M source, 800k byte code.

Mea Culpa,
-Ray

GWT Demystified: Generators, Part 1

Why do at runtime what you can do at compile time?

If GWT had a mantra, this would be it.

Introduction

As I mentioned in the previous article, the GWT compiler deals with a closed world -- no dynamic class loading, but it does permit deferment of binding decisions until compile time via a rule based mechanism that is part of the external GWT Module metadata. You can choose to replace a given type with other preauthored types, or, and here's the important part: you can replace types with classes that are generated on the fly.

This is the compile time equivalent of what you would do at runtime with libraries like CGLIB or the JDK's Proxy/Interceptor classes, and similar to Sun's APT (Annotation Processing Tool), except that it's all integrated into GWT and you don't need to worry about running separate APT-passes.

The deferred binding mechanism is used heavily in GWT to replace standard implementations of browser widgets with quirky implementations for browsers with divergent behavior. GWT then compiles multiple permutations of your code base, running the binding rules separately for each browser. Thus, if you have rules to target 4 different browsers, you end up with GWT compiling 4 different code bases and producing 4 compiled outputs.

Why does it do this? Because the result is more optimal code. Otherwise, you would have to include all 4 Implementation subclasses in your main code base, and use runtime logic to dynamically call the appropriate target, e.g. "if(brokenDom) DOMImplIE.foo(), else... DOMImplStandard.foo()..." It shortens load time by only forcing your browser to load what is neccessary, and it shortens run time by removing another level of indirection. Finally, it permits the optimizer to actually inline the appropriate implementation directly into the call site.

Generators

GWT also uses compile time generation of Java code to implement many platform features. The most famous of course is the RPC/Serialization mechanism. Here, GWT takes an interface, such as MyServiceAsync, and generates an implementation of this class on the fly, which contains all of the logic needed to serialize non-primitive types, send them across the wire via XMLHttpRequest, invoke the RemoteService, deserialize the rule, and invoke the async handler. It is probably the most complex generator in GWT, but not the only one.

Internationalization and Localization are also handled by the generator mechanism. Instead of loading ResourceBundles at runtime, you instead pre-process them and turn them into an interface, with one method per property. GWT will then use a generator to fill in the implementation of this interface which returns the values that are in the property file when the corresponding method is invoked. And what if you don't use some of the properties? The GWT compiler will remove any unused (uncalled) properties from the compiled application, and the number of HTTP requests is reduced because the locale data is bundled inline with your code.

Finally, truly the coolest and most innovative application of generators to date is the ImageBundle. In the world of 3D graphics programming, there is a technique called Texture Atlas, wherein you combine many textures into a single large texture, because on many graphics accelerators, changing pipeline state, such as binding a new texture before drawing geometry, is an expensive operation or may stall the pipeline. A program using texture atlases instead, binds one or more mondo-big textures, and simply uses texture coordinate manipulations to render portions of them as needed.

GWT 1.4 uses a similar technique to reduce load times and network traffic. With ImageBundle, like the I18N mechanism, you create an interface with a bunch of methods, one for each Image, as well as metadata annotations telling the GWT compiler which image file on disk you want returned by the method. Then, at compile time, the GWT compiler combines all of the images together into one large image file and generates an implementation class to return, essentially, the bounding box location of where each image is located within the overall atlas. Finally, when drawing the images, you just draw the same image (the large one) over and over, but use clipping rectangles to show only the part which corresponds to the image you need.

This reduces the number of HTTP requests drastically, speeds up startup time, and also centralizes media resources to a factory. The I18N and ImageBundle technique may even be combined to produce localized image bundles.

You're starting to imagine the possibilities?

How about buttons rendered with drop-shadows? Rounded corners done at compile time? Object-relational mapping to Google Gears or serialized RowSets? Type-safe JSON wrappers?

The project that I have created as my first generator is one designed specifically to suit my needs. Chronoscope has a JavaScript API that allows pure JS developers to access functionality provided in GWT classes. The way it does this is by creating what I call "bridge" classes and methods in the top level browser namespace. These bridge classes contain the public names (non-obfuscated) of GWT methods that will be invoked by the Javascript-to-Java JSNI Mechanism.

The problem is, everytime I added a method to one of my GWT classes, I had to go write a bridge method for it, and as Chronoscope grew in size and complexity, I needed a more automatic, and safe, mechanism for exporting an external JS interface.

In part 2 of this article, I will take you step by step, how to write this GWT Exporter, with actual code, culminating with the drop of the code on http://code.google.com/p/gwt-exporter/


-Ray

GWT Demystified

This will be a first in a series of tutorials and essays digging deeper into some of the more esoteric (but extremely important!) functionality of the Google Web Toolkit. (This article is written by Ray Cromwell, CTO) I will be talking about some of the nitty gritty details of the GWT compiler, how to create your own custom generators and the cool stuff you can do with them, and finally, how to plugin new optimization algorithms into the GWT compiler itself.

The Google Web Toolkit can be considered a package of three fundamental technologies: The Java to Javascript Compiler, the runtime library/APIs, and the Hosted Mode browser. All three are very important components of GWT in their own right, but because the APIs and the Hosted browser are the most visual pieces (you can SEE THEM operating) they tend to have more impact on people, while the real heavy duty stuff being done by the GWT compiler goes mostly unnoticed.

This is why I am not surprised that many language zealots engaged in language wars often appear grimaced when I tell them I am rewriting my Javascript code in GWT. "Huh!? Why!?" Then the inevitable signs of misunderstanding crop up: "Didn't applets show running Java in the browser is slow?", "Doesn't GWT produce bloated code?", etc. Now, I am by no means a zealot for the Java language, but there does appear to be frequent misunderstandings about what the GWT Compiler is, and does.

The GWT Compiler is a real compiler. Not a simple translator that walks an abstract syntax tree turning Java expressions and statements into the nearest Javascript equivalents. In fact, the GWT Compiler performs optimizations that are very hard to do statically on raw Javascript, and hard to do even in regular Java.

For example, static dead code elimination is very hard to do safely in Javascript, and limited in Java as well (at the bytecode level). You can never really be sure that a public method in Java won't be called, because of dynamic class loading, and frankly, you can't even be sure that private methods won't be invoked due to reflection and interception techniques, thus the only safe way to do dead code elimination is to defer it to runtime and let Hotspot deal with it.

In contrast, GWT has very good information on which methods are reachable, and is extremely aggressive at removing unused methods, saving both bandwidth and start up time. The closest equivalent in the Java world would be MIDLets, since the same sorts of closed-world assumptions can be made.

GWT does more than just remove dead code. It performs inlining, polymorphic-to-monomorphic call conversion (devirtualization), a form of type-inferencing which GWT calls Type-Tightening (GWT can infer that a field of type Animal is only ever assigned type Cat. In fact, it can infer that such a field is always null!), and lots of other little tricks. It doesn't appear to have common subexpression elimination, copy propagation, or in-block dead code elimination yet, but in my third tutorial, I will demonstrate a simple naive way to achieve this.

What this means is that in GWT, you don't pay much for what you don't use, and you don't have to worry about figuring out what's used. It means GWT's Javascript output is not bloated. It has a constant factor overhead related to the bootstrap process, but as your applications grow bigger, this code becomes smaller.

The GWT compiler already does a good job producing compact javascript code (Chronoscope is 30,000 lines of Java, 3.2 Megabytes of source, 1.9 Megabyte of compiled byte code, and 137k of Javascript after GWT 1.4 compiles it, and 45k after gzip -9), and there is a lot of headroom still left in terms of optimization that it can do.

So why do I use GWT? Besides the fact that we want to run in environments that don't have Javascript, GWT's Hosted Mode is extremely helpful providing full access to the universe of Java debuggers, and automatic code completion and popup Javadoc is very nice. (Even though I use IntelliJ IDEA, which already has good Javascript code completion and popup documentation)

GWT is a good platform for building AJAX/RIA applications and allows one to easily port or repurpose existing Java codebases and tools -- another good option for developers. It has a bright future ahead of it.

-Ray
Coming up: GWT Generators HOW-TO: generate classes on the fly at compile time.

Timepedia @ Google Developer Day

We had a blast at Google Developer Day, especially meeting and hanging out with the Google Web Toolkit team. Google's organization of the event was excellent, and they are to be commended for staging a world wide event (free) for developers, providing ample (and good) food and snacks, as well as busing developers to the Google campus for an after hours party. We've been to freebie events before, and they haven't been this good.

At Dev Day, Timepedia launched a preview of one of our components: Chronoscope, which is our visualization platform for timeseries data written in GWT.

We got alot of positive feedback, and had fun hearing people try and guess what our t-shirts meant. Actually, we were surprised how many people got very close. Timepedia has been our personal hobby, our passion, for over two years, and we are both nervous and excited to finally reveal it to the public.

As we come closer and closer to launch time, we'll post developer notes, stumbling blocks, hurdles leaped, and give more information about some of the other pieces of Timepedia (such as codenames: Tardis, Timelord, Geisser, Everett, and Jarocki :) )