The OpenWeb language for the next 100 years is...

Recently, there's been a lot of discussion about frameworks which translate or compile alternate programming languages to Javascript, John Resig's article being one of the more recent. At the heart of John's article is a lament that by abstracting people away from Javascript, they may not choose to learn the language.

There is some merit to this, after all, many developers use programming languages which compile down to assembly, but never take the time to learn the CPU architecture. I'm guilty of such, I grew up in the 80s hacking 6502 (VIC20/C64) and later 68k/80x86 code, but after my computer got powerful enough to compile C, I pretty much didn't keep up with ASM never learned MIPS, SPARC, or SSE (although I did take the time to learn DX9 shader assembly).

John also mentions that sometimes abstractions leak, and when they do, it may be hard to debug problems if you aren't familiar with the native platform. This too is a fair point, of course, the same is true with assembly, in that sometimes you do need to step into assembler code when a heisenbug appears.

One lesson to take away from this is that programmers should strive for deep knowledge of the tools they are using, from how compilers work, to library implementation, and underlying execution environment. Is it really good practice for a Java developer to learn nothing about the internals of the JVM, and how Hotspot optimizations work, for example?

But I'd like to ask a more important question: What if future web applications aren't written in Javascript at all, would that really be a bad thing? What makes Javascript ideally suited for client side coding, which really means, DOM/CSS manipulation, event handling, and network I/O? You see, much of the Javascript community accepts that it is perfectly natural for server-side code to be written in any language: Java, Python, Perl, Ruby, PHP, C#, even Javascript. You do not see a huge push for a singular server-side language. And the server-side has since developed into a very strong, diverse ecosystem, sprouting up robust VMs everywhere. And from there, robust library frameworks, like RoR, Django, etc.

So why is it, that only a single language: Javascript, and it's various VM implementations, is treated as if it is the only True and Correct way to work on the client side? If Javascript VMs one day became more general purpose, accepting perhaps bytecode or some form of intermediate representation as input, and we ended up with Ruby, Scheme, Python, et al, running natively in the browser, would this be the end of the world?

To listen to some Javascript evangelists, you would think so. The OpenWeb seems to mean that you have freedom and openness on the server, just don't dare write any non-Javascript on the client side, that's off limits!

Unfortunately, we don't have a general purpose VM in the browser, like JVM, or CLR, or Parrot. Instead, we have Javascript, which is why all of these third party language translators are treating Javascript as if it were assembly or bytecode. But oh, the nerve of these translators! What an insult! How dare you compile some inferior language or DSL down to basic Javascript, and not write beautiful and elegant human readable Javascript instead.

On a more serious note, there appears to be a great deal of hypocrisy in some of the criticisms of third party frameworks as well. That is, it is impermissible to abstract away the Javascript language or try to "hide" the Web, that is, unless you are a Javascript framework which has nifty DSLs and Widgets which do exactly that -- like hiding the pain of getting CSS layouts right from newbies.

The question for you is, do you believe Javascript is the next big programming language, the language that we should be programming the web with, 5 years, 10 years, 100 years from now? What I've learned from sitting through 20+ years of internet/usenet language flamewars is that no language is perfect, people have different tastes, different aesthetics, and different languages fill different niches. Emacs vs VI will never be resolved. Static vs Dynamic typing will never be resolved. And no one language can satisfy everyone.

This is not to say that there aren't concerns about gluing code written in multiple languages together and making them interoperate. This is being done on .NET CLR and JVM today, so it is certainly possible in the future to do it in the browser. On the other hand, many people today already work with incompatible siloed Javascript frameworks, so maybe it's not really an issue if Ruby code can't reuse Python code on the client.

In the end, I would say to Javascript ninjas, relax. There will be more than enough people writing Javascript to sustain a vibrant community. You should welcome polyglots on the client as well as the server.

Let a thousand flowers bloom.
-Ray

Worst anti-GWT marketing ever

There seems to be an increase in other frameworks attacking GWT, which is probably a indicator that Google is doing something right, but some of these attacks are downright silly, consider Jaxcent.com:

Unlike GWT, Jaxcent is a server-side framework. Instead of being compiled into JavaScript, the Java code directly runs on the server, and communicates with the client via a small JavaScript file.

...

There may be some concern that a server-side framework may be putting more burden on the server, compared to GWT. However, a GWT-like approach does require the server to maintain, manage and deliver multiple JavaScript files. In real terms, that can be a significant server load. In contrast, Jaxcent has a single small JavaScript file, that will be cached by normal browsers. The actual load on the server is comparable to any server side pre-AJAX framework, such as servlets, JSP, ASP etc.

Leaving aside the fact that it is ridiculous to assert that serving up static GWT Javascript files, which can be deployed to a CDN, typically are less than 150k compressed, and cacheable forever so that they are never fetched more than once, represents a signifcant server load, it is even more ridiculous to minimize the burden of keeping all UI state on the server, and making network around trips every time the user takes an action.

Instead of a Rich Internet Application, you end up with a Dumb Terminal Client, that evokes the days of TN3270 and mainframes.

Chronoscope on iPhone

If it hasn't been clear from my videos, I don't film them with a script or much forethought as to what I'm going to say or demo, which is one reason why I forgot to show off the iPhone version of Chronoscope. This time, it runs as Javascript in the Safari browser, but leverages iPhone gesture and touch events. But will it always run as Javascript, or is there another possibility? Time will reveal all secrets.

Chronoscope and GViz on Android

It's been a while since I've updated the blog, with good reason as we've been very busy recently. However, I wanted to share with you the fruit of some of that labor.

Long time readers will recall a prototype demo I posted a long time ago of Chronoscope running on Android. The original version was mainly a proof of concept that it was possible to reuse 98% of your code base and deploy nearly identical functionality both in browser, in the server, and on the phone.

What hasn't been shown however, is the production version, rewritten, with improved performance. Now dubbed Timescope, the new version integrates tightly with Android, as well as our ChartServer, and Google's Visualization API, to allow you to generate charts that render dynamically in web browsers, but launch native high performance applications on the phone.

Here's a video demonstrating a GViz chart, rendered as a static clickable image from our chart server and viewed in the Android Web browser. Click on the image launches a native Android application which allows full interactivity.

All of the full functionality of Chronoscope is retained, scalable to immense streaming datasets, annotatable via a variety of markers, a browser-like history mechanism, bookmarkable links, and a very flexible graph style sheet system.



I'll be posting a fairly substantial update soon on all of the cool stuff we've been working on recently, which is nearing public release.

-Ray

Interesting observations about WebKit/Chrome and Chronoscope Benchmark

Playing around with the benchmark some more, I noticed some strange things. The more I clicked the "Bench" button, the faster Chrome got. After about 4 runs of the benchmark, it increased as much as 2x in performance. Is Chrome using HotSpot-like techniques to count method invocations for inlining?

Secondly, the animation on WebKit/Chrome gets choppy. This isn't because the Javascript isn't drawing the frames. The benchmark draws 100 frames, each in a setTimeout call to avoid slow script warnings. Browsers appear to defer reflecting any mutations to the UI until the Javascript thread yields the CPU back. That is, you don't see the results of a canvas fill() or stroke() until the CPU is yielded. (Opera seems to work differently, but offers a GameCanvas which lets you lock/unlock progressive updates)

Now, I would normally expect that the draw results would be copied/blitted/reflected into the Browser's document as soon as the Javascript thread yields. But what if the update is scheduled in the same timer/event queue as other pending setTimeout calls? And what if the next frame's scheduled timeout is before the native canvas update?

Then what would happen is that the Javascript code would get the CPU again, erase the canvas, and draw the next frame. This would account for the choppiness and disappearance of intermediate frames.

But it could also count for the performance differential between TraceMonkey/FF3.1 and WebKit/Chrome, in the sense that TraceMonkey might "pay the cost" of reflecting every frame into the browser's window which gets counted by my timing code, whereas WebKit/Chrome might get a free ride by skipping the update and somehow avoid the cost by deferring it.

I'm not sure, and it would be interesting if people who are familiar with the internals of Gecko and WebKit could comment on how canvas updates are handled.

In the meantime, I'll look at ways of ensuring that every frame is drawn, perhaps by increasing the intra-frame timeout interval.

Chronoscope Browser Benchmark Live

Hey guys, in order to assist in the browser wars, and to let everyone test browser performance, I've built a publicly hosted version of the benchmark I used to test Google Chrome vs Firefox 3.1 and Safari 4.

Now you can run the same benchmarks too, just hop on over to http://api.timepedia.org/benchmark and let the battle begin!

As an errata for my previously published benchmarks, I did find that somehow my WebKit Nightly batch file wasn't invoking Safari correctly on my Win32 box (it was complaining about a file not found which I didn't see in the console), so my previous results were NOT with SquirrelFish. Upon rerunning with SquirrelFish, I find it actually does hold its own vs Chrome.

If you're one of those poor folks who are running IE, don't try to run the benchmark. :) It might run, but chances are, you'll get a slow script warning. Also, it uses Flash for rendering instead of Canvas, so it's not an Apples-to-Apples comparison.

Google Chrome smokes the competition on Chronoscope

The current browser wars make me feel like I'm watching the Fast and the Furious, and Google's new Chrome browser is like Nitrous Oxide for the web. As soon as I heard that ex-HotSpot guys were working on it, I knew it could be good, as they did a marvelous job with Self (a fully dynamic language), as well as Java, but how good?

To test, I decided to use my own codebase, Chronoscope, for several reasons. First of all, it is not a microbenchmark, it exercises a big swath of the browser's code path. Secondly, it is computationally intensive, so if anything were going to show speed on a "real world" app, this would be it. Finally, I need visualizations to run faster, as I want to use Open Web technologies like Canvas or SVG rather than relying on Flash.

Chronoscope has a workload which contains elements of the following:

• Large amount of affine transforms calculated within Javascript code
  
• Many binary searches and recursive calls
  
• Frequent random and linear access to large objects (arrays with 30,000+ elements)
  
• Numerous DOM calls to absolute position DIVs (emulate text rendering on Canvas)
  
• Huge number of CanvasRenderingContext2D calls for rendering the chart
  
• Ubiquitous use of abstraction, delegation, inheritance
  

Chronoscope is written in GWT, and to some extent, the GWT compiler may negate some of Chrome's V8 technology in the sense that GWT "de-classes" many OO polymorphic dispatches into a more functional style of programming, removing as much dynamic dispatch as possible, and eliminating prototype lookups and function call overhead through inlining. I don't know if GWT hurts "hidden classes" or not, but it might be possible that if GWT didn't provide such optimizations, the performance differential might be larger.

Despite this, the results are still good. The test consisted of calling the chart's redraw() function 100 times per trial, with 10 trials. The slowest and fastest trial are thrown out, and the mean and standard deviation are calculated on the remaining data.

I tested on a Mac Pro 2.66Ghz with 6Gb of memory, OSX 1.5. The tests were conducted within a Parallels VM running XP2 Service Pack 2, given 2 CPUs and 2Gb of memory. For each browser, I rebooted the VM from a clean start, and ran only the test browser.

The tests were conducted with the lastest version of Safari 3.1 for Windows (run via WebKit nightly batch file), the Firefox 3.1 nightly with TraceMonkey enabled, and the Chrome beta download.

Browser	Mean	Standard Deviation	Memory
Firefox 3.1 (TraceMonkey)	3647ms	81ms	49M
Safari 3.1 (SquirrelFish)	3005ms	385ms	106M
Chrome	1690ms	190ms	44M

Chrome looks to be twice as fast as the competition on real world apps. Moreover, it's performance per Mb of memory is good as well. Even more interesting was the variation in Firefox 3.1, which appears to be related to garbage collection or memory allocation. In some runs, it was very fast, but in others, it was 50% slower.

As soon as Chrome is released on OSX, it will replace Safari for me, for two reasons. The first, the process isolation and speed features. Secondly, Chrome includes a very nice Firebug-like debugger/inspector that I like better than the Safari equivalent.

So in summary, Chrome rocks! All they were missing from the launch was Vin Diesel.

-Ray

Powers of Often: Powers of Ten in Time

Hi all. A little bit of self-pimping today. We're looking at doing a cool presentation at next years 2009 SXSW Conference using our technology. For those who like Chronoscope and want to see even cooler stuff, please follow this link http://panelpicker.sxsw.com/ideas/view/1649 and vote for our panel.

Here's the description:

In 1977, Charles & Ray Eames made a fascinating short film, Powers of Ten, showing the relative scales in the universe: from picnic, to city, to solar system, to galaxy, and so on, back to cells, molecules, and atomic nuclei. In the same spirit, Powers of Often will explore relative scales in time using real data and hard estimates: patterns of daily life, demographics, census data, generations, long term trends, forecasts, historical cycles, high-frequency finance, and solar cycles.

Thanks,
-Ray

Cross Domain FormPanel submissions in GWT

While working on Timelord, our next generation charting and analytics tool, I found the need to do cross-domain REST requests because Timelord may be hosted anywhere, but the server-side services are running on a fixed domain (hosted by Google App Engine). Recently, abuse cool hacks using the window.name field have become en vogue, much like the earlier activity around fragment identifiers, and one particularly cool usage is the recent Dojo proposal to use window.name for cross-domain cross-frame communication.

After a little experimenting, and some snooping on the Dojo patch, I implemented a quick patch to GWT using Deferred Binding, here's an example:

public class WindowNameFormPanelImpl extends FormPanelImpl {

  /**
   * Gets the response html from the loaded iframe's name property
   *
   * @param iframe the iframe from which the response html is to be extracted
   * @return the response html
   */
  public native String getContents(Element iframe) /*-{
    try {
      // Make sure the iframe's window & document are loaded.
      if (!iframe.contentWindow || !iframe.contentWindow.document)
        return null;

      // Get the response from window.name
      return iframe.contentWindow.name;
    } catch (e) {
      return null;
    }
  }-*/;

  /**
   * Hooks the iframe's onLoad event and the form's onSubmit event.
   *
   * @param iframe   the iframe whose onLoad event is to be hooked
   * @param form     the form whose onSubmit event is to be hooked
   * @param listener the listener to receive notification
   */
  public native void hookEvents(Element iframe, Element form,
      FormPanelImplHost listener) /*-{
    if (iframe) {
      iframe.onload = function() {
        // If there is no __formAction yet, this is a spurious onload
        // generated when the iframe is first added to the DOM.
        if (!iframe.__formAction)
          return;

        if(!iframe.__restoreSameDomain) {
          iframe.__restoreSameDomain = true;
          // restore same domain property of iframe to read window.name property
          iframe.contentWindow.location = 
             @com.google.gwt.core.client.GWT::getModuleBaseURL()() + 
              "clear.cache.gif";
          return;
        }
        listener.@com.google.gwt.user.client.ui.impl.FormPanelImplHost::onFrameLoad()();
      };
    }

    form.onsubmit = function() {
      // Hang on to the form's action url, needed in the
      // onload/onreadystatechange handler.
      if (iframe) {
        iframe.__formAction = form.action;
        iframe.__restoreSameDomain = false;
      }
      return listener.@com.google.gwt.user.client.ui.impl.FormPanelImplHost::onFormSubmit()();
    };
  }-*/;
}

And in your GWT module

 <replace-with class="org.timepedia.timelord.client.impl.WindowNameFormPanelImpl">
                  <when-type-is class="com.google.gwt.user.client.ui.impl.FormPanelImpl"/>
  </replace-with>

With this code, you simply submit a FormPanel to a service with an additional parameter, like "windowname=true" which indicates you want window.name transport. The resulting output of a POST should contain something like "<script>window.name = '....result string...'</script>. When the FormPanel.submit() method is invoked, this result will be passed to the FormHandler.onSubmitComplete(event), and you can retrieve the result via event.getResult().

-Ray
p.s. there are separate implementations for IE

Chronoscope GViz API and Gadget released

Hey guys, a beta of Chronoscope 0.86 (in the trunk) has been packaged up as a GViz API and Google Gadget (with Spreadsheets integration). Check it out at Timepedia.org to see the new features.

-Ray

Google I/O GWT Extreme! Presentation video now online

or, how I managed to render a million datapoints in Javascript in real time without crashing my browser. :)

For those who missed it, you can now catch my GWT Extreme! presentation on YouTube courtesy of Google.



You can also view the slides directly here:

| View | Upload your own

I would highly recommend watching the following presentations first before watching mine:

Surprisingly Rockin Javascript and DOM Programming
Faster Than Possible Code: Deferred Binding
Resource Bundles and Linkers

If you're interested in just seeing the demo of Chronoscope zooming and rendering over 1 million datapoints, skip to about 27 minutes into my video.

-Ray

Preliminary GWT Export 2.0 release

For 1.5 users, I made two quick changes to the GWT Exporter to implement the first two features I mentioned last time: Zero-overhead Deferred Binding and 1.5 annotations.

Here's the change log:
* @gwt.export,@gwt.noexport,@gwt.exportClosure,@gwt.exportPackage now full fledged annotations instead of old 1.4 comment style
* Replacements are @Export, @NoExport, @ExportClosure, and @ExportPackage
* New deferred binding yields zero-overhead when exporting turned off
** By default, exports turned off, use <set-property name="export" value="yes"/> in your module to compile with them on

Build from the repository, or download the jar file at http://code.google.com/p/gwt-exporter

-Ray

GWT Exporter:The Next Generation

Now that GWT 1.5 is out, it's time to look at revising my GWT Exporter library to take advantage of new features, as well as improve it speed/performance. Here are some of the features I'm looking at including in GWT Exporter 2.0

1) Support for 1.5 annotations (place @Export in front of a method to export it)
2) Export as Deferred Binding controlled by property (e.g. add ?export to URL or use <set-property) to avoid the overhead when you don't need it. For example, today Chronoscope always exports methods, even if you are including it inside of 100% GWT app. There should be zero overhead/cost when not being used.
3) Eliminate wrapper classes and bridge methods. Today, GWT Exporter generates Javascript classes which wrap GWT classes and expose methods. In 2.0, it should achieve exporting by adding non-obfuscated method aliases to GWT objects themselves.
4) Export as pure-JS libraries (no bootstrap script) with JSDoc annotations. Ideally, I copy could Javadoc from Java to JSDoc, but it's problematic since Generators don't get comments exposed to them (except for metadata)

If you have any feature suggestions for the Exporter, leave them here in the comments.

-Ray

How I lost 3 hours of my life to dynamic typing

Today, while working on integrating Chronoscope into Google's GViz API, I ran across a bizarre problem, the depths of the problem would flush 3 hours of work time down the toilet.

In order to integrate Chronoscope with GViz, I decided to provide some lightweight GWT 1.5 overlays on top of the API. To use the API, it is recommended to use the Google AJAX Loader. Quickly reading the docs, it seemed very straightforward. You start out by

<script src="http://www.google.com/jsapi?callback=handler">

After the google.load API is available, you write

function handler() {
  google.load("visualization", "1", function() { alert("visualization api loaded!"); })
}

or do you?

Here comes the pain

So, if you're already familiar with the AJAX Loader API, you'll recognize I made a typo. Between the time I read the documentation, and the time I wrote the code above, the types of the parameters needed for this call morphed in my head. The third parameter can include a callback for asynchronous loading, however, the callback isn't passed as a function, but as a object, like this

google.load("visualization", "1", { callback: function() { alert("visualization api loaded!") } });

and I would have discovered this problem 3 hours earlier if it hadn't been for one fact: the wrong code locks up Firebug and generates an infinite "Loading..." in the browser tab.

If you run the wrong code inside a regular HTML page, it won't have this effect. But I was running this through an Apache Shindig IFRAME, which had loaded script containing this code via _IG_GetCachedURL, and somehow the combination of the IFRAME, the proxy loading of the script, and the cross-domain included JSAPI combined to kill Firefox. I couldn't use Firebug to set breakpoints, to inspect the network to see what stage of loading caused the failure, to see which script might be the culprit, or even to inspect the HTML DOM. I tried FF3 and installing Firebug 1.2beta, no dice. I tried Safari debugging, no dice.

Eventually, I came back to read the docs two or more times again, and discovered how I had screwed up the type of the third argument. In many other AJAX libraries, like Dojo or jQuery, they are more forgiving, and probably would have checked if the third argument was typeof(o) == "function" or "object" and acted accordingly.

I know, you're saying "Serves you right! You should RTFM more carefully!" Unfortunately, these kinds of errors are bound to be made by mere mortals, even if they do RTFM, especially with the size and complexity of todays AJAX apis.

Why do I endure using Java by using GWT my Javascript Ninja Brethren? When a simple mistake, or typo can waste half your day, you'll understand. And now that I've written a strongly typed GWT GViz/AJAX Loader wrapper, it's unlikely any GWT user will ever have to suffer the same fate.

-Ray

Design Patterns vs GWT Compiler: Round 1, Fight!

I've spent a lot of time maximizing performance of GWT code, and in doing so, I've tended to shy away from using too much OOP and Design Patterns, but the question is, is this fear justified? You've heard the GWT team continually tell you that you can have your abstractions without much performance loss, but is it true? It turns out, in many cases, the answer is a resounding yes.

Take for example, the Iterator pattern. I've crafted GwtQuery to iterate over native arrays, because I feared the overhead of iterators. But if you're careful in how you use the type system, the overhead turns out to be approximately, zero.

For example:

public class MyArray implements Iterable {
  private String[] items = {"foo", "bar", "baz"};

  public Iterator iterator() {
    return new StringArrayIterator(items);
  }

  private class StringArrayIterator implements Iterator {

    private String[] items;

    private int index;

    public StringArrayIterator(String[] items) {
      this.items = items;
      this.index = 0;
    }

    public boolean hasNext() {
      return index < items.length;
    }

    public String next() {
      return items[index++];
    }

    public void remove() {
      throw new UnsupportedOperationException();
    }
  }
}

used as

 MyArray m = new MyArray();
  for(String s : m) 
      Window.alert(s);

The generated Javascript is

var m, s, s$iterator;
  m = $MyArray(new MyArray());
  for (s$iterator = $MyArray$StringArrayIterator(new MyArray$StringArrayIterator(), m.items);
        s$iterator.index < s$iterator.items.length;) {
    s = s$iterator.items[s$iterator.index++];
    $wnd.alert(s);
  }

As long as GWT can type-tighten the Iterable interface to the concrete implementing class, this output will usually hold true. The following will break the optimization and impose polymorphic dispatches:

 public void onModuleLoad() {
    MyArray m = new MyArray();
    MyArray2 m2 = new MyArray2();
    iterate(m);
    iterate(m2);
  }
  
  public void iterate(Iterable m) {
    for(String s : m)
      Window.alert(s);
  }

Here we introduce a new type, MyArray2, and we use an iterate() method which invokes the Iterable interface. Since GWT now has two classes in existence which implement Iterable<String>, it can't decide whether it has a MyArray or MyArray2, and therefore can't elide the iterator. You might ask, what happens if we remove MyArray2, but keep the iterate() method? We return to optimal code:

function $iterate(m){
  var s, s$iterator;
  for (s$iterator = $MyArray$StringArrayIterator(new MyArray$StringArrayIterator(), m.items);
     s$iterator.index < s$iterator.items.length;) {
    s = s$iterator.items[s$iterator.index++];
    $wnd.alert(s);
  }
}

Neat huh? If you're curious, try the same thing with Delegation, Decorator, Facade, or Flyweight, I think you'll be pleasantly surprised.

Fatality!, GWT Compiler. :)
-Ray

GwtQuery 0.2 released

GwtQuery 0.2 alpha is now available in the repository, both as a jar file, and as a Maven project. Nothing major has changed since the last release except to relayout the directory structure for maven.

Download it at gwtquery.com and have fun!

See the samples/ directory for some example code of how to setup a GWT module to use gwtquery.jar.

-Ray

GwtQuery 0.1 alpha released

I just finished my presentation at Google I/O. It was a real blast, and I met a lot of really cool, smart people.

To celebrate the occassion, I am releasing an alpha of my jQuery-API-clone-in-GWT today. Visit gwtquery.com to download the source. It's a partial implementation at the moment, no deployable, or build script. Expect that to change in the near future. I welcome patches/contributions to flesh out the API and add the missing functionality.

-Ray

Decentralizing the Web

I have tried to keep this blog free of politics and opinion and centered on implementation and algorithms, but lately, certain things have bugged me, so for the first time, I'm climbing aboard the soapbox, and these opinions are not those of my company.

Isn't the web fundamentally decentralized?

The old days

I've been on the internet/usenet/netnews since the mid 80s. In the early days, my communications mostly occurred in news readers, email clients, Unix talk clients, later MUDs and IRC. In those days, the net was ironically, more federated and decentralized. Servers and Clients were mostly heterogenous flavors of Unix, which practically demanded that most communication server/client code were distributed as source. Servers were typically run from college campuses on modest machines, so the architectures tended to be federated. The lack of a single, uber GUI client, tended to drive people to invent specialized protocols for each application, and the poor portability of clients coupled with eager college students tended to produce multiple server/client implementations. There was no single news server, no single email host, no single chat server. Granted, big sites did exist (UUNET), but there was no virtual monopoly of one server being the sole provider of channel services.

Maybe it's nostalgia, but in the pre-web days, the spirit of the early IETF, it seems to me that engineers more commonly architected for the federated case, architectures were more open for inspection, and people cooperated openly.

The web

The browser ushered in a whole new way of designing applications, with some interesting repercussions. In the pre-web days, if I wanted to create a new messaging platform, I'd have to create a protocol spec, write the server and client, and get tons of people to port it and run servers. Something like RSS wasn't as important either because the protocols already defined machine readable ways to syndicate content.

The browser changed all that. Since the client portion was now handled by a third party, and ported to every piece of hardware known to man, and since it provided a very flexible way of displaying custom user interfaces, focus shifted to building big centralized servers which blossomed when coupled with declining hardware costs, and increased network bandwidth.

With the web, new "channels" sprang up everywhere, AOL/Yahoo/MSN/QQ/ICQ/etc and even today, there's Twitter, there's your Facebook wall and inbox, and tons more. What's disturbing is that these communications channels are often proprietary, and have a single point of failure. If Facebook or Twitter goes down, you're screwed. If my SMTP/IMAP server goes down, I'm screwed, but 50 million other people aren't.

Oh, this is about Twitter crashing again

Yes and no. I was irked into posting this while reading a bunch of blogs where authors simultaneously defend Twitter's downtime, insist that it is virtually a national infrastructure in importance, but also asserted that it should not be decentralized.

HUH?

First of all, I think it is disgraceful, that in 2008, instant messaging is still balkanized. The IETF standardized XMPP a long time ago and implementations have been proving their worth for a long time. Yet, have AOL, MSFT, Y!, et al adopted it? No, because they don't want to relinquish control. Remember all that talk of a truce in the late 90s? It never panned out.

Blogging and RSS are decentralized, there is no single, universal, host of blogs. Imagine that you could not read, nor write blogs without having a blogger account! So why should microblogging be centralized? The arguments as to why you cannot federate Twitter are pretty weak IMHO. Communication channels are like roads, they are infrastructure, and fundamentally affect the information economy, and it doesn't make sense to me that such fundamental services be centralized behind a single point of failure nor closed in implementation. Besides, federating Twitter would probably produce new business opportunities for aggregators.

Not just about Twitter

If social networks are as important as everyone makes them out to be, why should I have to log into MySpace or Facebook to access them? Why must MySpace or Facebook applications only run on those sites? Shouldn't I be able to access my social network anywhere, on any site, in any application, web or otherwise? On the desktop, I can access my Address Book or corporate LDAP server in many applications, but I can't do that on my own site, without becoming a Facebook application.

Isn't it time to federate social networks?

The OpenID, OAuth, OpenSocial, and Data Portability initiatives are going a long way to address the architecture needed to do this. FOAF/XFN provide possible ways for discovery of social network, but there are pieces missing to the puzzle that need to be addressed before the user experience can approach that of centralized social networks, or give developers easy access to building applications that can access federated data. Google has been doing a good job thus far, but perhaps the IETF and W3C should create working groups to study the issue as well.

Peer to Peer Social Networking

One interesting possibility is using something like Gears to store and replicate your social network. That is, take my social network offline. When I add you as a friend, I could store that information locally on my machine, as well as optionally broadcasting it to several public services which also record the information. Third party websites could embed social applications which use Javascript snippets to ask my permission to run queries against my Gears database, or use OAuth to ask for the data from public services I replicated to. This could be more than a simple contacts list, since authenticated peer-to-peer exchanges (you share your offline social network with me), or social graph crawl/share services, would enable more client side aggregation.

Social networks typically provide enhanced services like profile pages, server-side 'shared' data storage for applications, and activity streams. Designing a federated system for these is an interesting exercise. Federated publish/subscribe message queues are not without precedent, nor is distributed storage. I'm not saying it's a solved problem, but shouldn't we at least try?

In any case, I'd still like to partially solve the problem by taking my social network offline, and enabling social applications which need to probe my graph to do so.

I'm tired of having to log into FB every day, or import my friends into each new site I sign up with, just as surely as I am tired of creating new username/password accounts on every site for login.

Let's return to the good old days and get away from single points of failure, cathedrals, and make the social web, truly a fundament piece of internet architecture, like TCP/IP, DNS, HTTP, and SMTP, and less about Bubble 2.0 startups.

-Ray
(It's late, I'll probably regret this rant when I read it tommorow). I now return you to your regularly scheduled engineering related posts.

Google I/O and GWT Extreme!

I've been invited to do a session at the upcoming Google I/O conference. The initial idea was to do a presentation on Syndroid, but I felt that was too specific and not generally useful to the audience. What would be useful, would be to cover general purpose techniques in GWT 1.5 to bridge Javascript execution environments as well as high performance tricks. Thus was born: GWT Extreme!.

For those attending Google I/O and want to see this session, I recommend attending Bruce Johnson's presentation on Deferred Binding and Bob Vawter's presentation on Linkers first. In this session, I will go over the following techniques:

• GQuery and compiling CSS selectors at compile time


• The Android Native Interface, packaging GWT applications as Android apps and automatically making calls to native Android Java from within Javascript.


• The ActionScript Native Interface, compiling GWT code to SWF, generating RPC stubs, and making calls between browser GWT and Flash GWT code


• Writing Gears workers in GWT


• If there's time, extreme graphics performance with GWT

As the session is only one hour long, I will not be going in-depth with source examples like I did in my Deferred Binding Presentation, but rather, I will describe the overall architectures and steps needed to achieve them in general terms. I may follow up with more in-depth blog posts later containing source code.

Finally, I am planning to release GQuery (or GWTQuery) at Google I/O, there will be a usable beta-quality build on the project code hosting site.

-Ray

A screenshot of Syndroid

Edit: Ha! I just noticed I spelled Dalvik wrong! There goes my prize!

Work on GQuery progresses, Comment on Slickspeed

GQuery is progressing nicely. I implemented all of CSS3 selectors by following ExtJS's DomQuery implementation, only I added the ability to parse at runtime as well as via a generator at compile time. The compile time generator turns a CSS selector into 100% inlined code. That is, a selector like "#foo", will turn into "return document.getElementById("foo");", no parsing step involved. I've still got a bunch of optimizations to make, add support for XPath and native getElementsByClassName, but even now working with the library in GWT is very cool. I just started looking at DomAssistant as well, to incorporate (i.e. steal) the best algorithms from each.

I've been using SlickSpeed to evaluate compliance of my implementation, as well as check performance, but I noticed an issue with SlickSpeed's benchmarking which is endemic to a lot of microbenchmarks: failure to account for timer resolution.

When you run SlickSpeed on some browsers/operating systems, alot of the results will be 0ms, 4ms, 8ms, or some other multiple, because the Javascript Date.getTime() function has a fixed update resolution on some system, for example, on some systems it seems to be around 16ms or roughly 1 screen update (60Hz = 1/60th second). SlickSpeed runs each selector 4 times, and then divides by 4, which is why you see results like 4ms, 8ms, and the like.

The problem is, there is always the issue of timer jitter and aliasing. For example, if you happen to sample the timer twice before the next update happens, you end up with a 0ms result, even if it took 15ms, whereas if you get unlucky and start the benchmark half way into an update interval, you may record a result rounded up to the next. I need to hack SlickSpeed anyway to allow it to bench GWT, and in the process I'm going to run each selector such that the time exceeds 500ms, plus toss out the min/max values to evaluate the effectiveness of caching on some of those APIs. This will disadvantage my compile time selectors, which avoid a parsing step, but be more fair to comparing the actual implementation of the selector routines. I might make a separate benchmark to time 'setup' speed that these libraries use for parsing the selector, since that is obviously a factor in many end user apps that don't neccessarily run the same query more than once.

-Ray

GWT: The Road to 1.5, Language features and GQuery

Hi folks,
I had to delay the second part of my article on Linkers because the APIs have been changing radically lately and have only just settled down in the trunk. However, to pass the time, I decided to demonstrate some of the huge benefits of 1.5 by implementing a type-safe JQuery clone in GWT.

That's right, for all those folks who don't want to deal with GWT Widgets, but just want to query, wrap, and manipulate plain old DOM elements in GWT, this article is for you. I'm talking to you, you self-hating Java programmer with an inner Javascript Ninja begging to be let out.

Yes this is working code

First, let's just dispense with the details and show you a working example, yes, this is working code:

public void onModuleLoad() {
    $("div").css("color", "red").click(new Function() {
      public void f(Element e) {
        Window.alert("Hello");
        $(e).as(Effects).fadeOut();
      }
    });
  }

What does this do? It makes every DIV element red and clickable, and if you click one, it pops up an alert message, and fades the DIV out.

The API is mostly faithful to jQuery's public API, with the exception of the lack of succinct closures (Hello, Neal Gafter!), but that's ok, we can introduce enough prebuilt composable functors to eliminate alot of the need for inline anonymous functions for common operations.

Small, Fast, Efficient Code, and Other Benefits

Since GWT prunes any code not being used, this example produced a obfuscated + gzipped script size of only 5459. If I change the click handler to simply $(e).setInnerHTML("Boom"), the compiled + gzipped script size is 1400 bytes.

There's a few small differences to take note which are either a benefit, or a hinderance depending on who you ask:
1. Type-Safety, IDE code completion, debuggability, etc
2. Lack of prototype modifications forced slighty change for GQuery plugin usage (see Effects above)
3. Selectors can be compiled at compile time, more on that later.

How 1.5 Enables GQuery

Now, let's talk about the features in GWT 1.5 that make GQuery nice and efficient to implement.

1) Joel Webber of the GWT Team just submitted a patch to GWT 1.5 that implements every DOM2 Core+HTML class as a subclass of the new JavaScriptObject. This not only made the coding of GQuery very easy, but it produces nicely optimized JavaScript.

2) Static Imports. The $ function would not nearly be as nice without static imports.

3) Generics and Covariant return. The Plugin mechanism utilizes this to fake prototype based overrides of the GQuery object.

How to write GQuery plugins? Damn simple

Java doesn't feature metaclasses or prototypes, so there is no way to add methods to an existing instance or class without subclassing, but in order to preserve jQuery style syntax, we can use covariant return, type capture, and other features of JDK 1.5, to fake it.

There are two ways this occurs, first, the $() method can take a final parameter which is a Plugin class literal, or, you can call the as() method on any GQuery and "cast" it to a plugin interface. Here's the implementation of the as() method:

/**
   * Convert to Plugin interface provided by Class literal.
   */
  public <T extends GQuery> T as(Class<T> plugin) {
    return (T) plugins.get(plugin).init(this);
  }

The type of the plugin, which is a subclass of GQuery is captured by the class literal parameter, the Class literal is mapped to a Plugin class factory which creates instances of the plugin, and finally, the current GQuery is passed in, so that the Plugin can access the set of matched elements.

Here is an example plugin implementation:

public class Effects extends GQuery {
  
  static {
    GQuery.registerPlugin(Effects.class, new EffectsPlugin());
  }
  public static final Class<Effects> Effects = Effects.class;
  
  public Effects(Element element) {
    super(element);
  }

  public Effects(Element[] elements) {
    super(elements);
  }

  public Effects(NodeList list) {
    super(list);
  }
  
  public Effects fadeOut() {
    Animation a = new Animation() {

      public void onCancel() {
      }

      public void onComplete() {
        for(Element e : elements) {
          e.getStyle().setProperty("opacity", "1.0");
        }
      }

      public void onStart() {
      }

      public void onUpdate(double progress) {
        for(Element e : elements) {
          e.getStyle().setProperty("opacity", String.valueOf(1.0-progress));
        }
      }
    };  
    a.run(2000);
    return this;
  }
  
  public Effects fadeIn() {
    Animation a = new Animation() {

      public void onCancel() {
      }

      public void onComplete() {
      }

      public void onStart() {
      }

      public void onUpdate(double progress) {
        for(Element e : elements) {
          e.getStyle().setProperty("opacity", String.valueOf(progress));
        }
      }
    };  
    a.run(2000);
    return this;
  }
  
  public static class EffectsPlugin implements Plugin<Effects> {
    public Effects init(GQuery gq) {
      return new Effects(gq.get());
    }
  }
}

That's it. Plugins are nothing more than subclasses of GQuery which have an extra registry step and factory.

I'll put up the source code on http://code.google.com/p/gwtquery after GWT 1.5 milestone 2 is released, which is needed to run this.

You forgot to talk about compile-time selectors

So, yeah, as Generators are a frequent topic of this blog, you really think I wouldn't find a way to sneak them in? GQuery supports two mechanisms for selector evaluation. It can compile and evaluate selectors at runtime (currently just using XPath for prototyping, which isn't available everywhere), or, you can create an interface containing selectors you know about at runtime, and have them precompiled and ready to execute.

As an example:

public interface MySelectors extends GQuery.Selectors {
  @Selector("div")
  public GQuery allDivs();

  @Selector("div.foo")
  public GQuery allFooDivsBelow(Element context);

}

Then to use:

  MySelectors s = (MySelectors)GWT.create(MySelectors.class);
  s.allFooDivsBelow(context).css("backgroundColor", "yellow");

A generator performs a browser-specific compile-time transformation of the selector in the annotation into the most optimal form that the browser supports (XPath, pure-JS, DOM traversal, document.getElementsByClass, etc) Not only does this produce smaller per-browser code, it produces faster code as well.

One last thing, the $$ function

jQuery contains a lot of methods which take essentially property/value pairs in a Javascript object literal. Emulating this with Java syntax would be too tedious (Scala frontend to GWT, please!), however, to ease this use case, I added a global $$ function for manipulation JavaScriptObject/JSON objects easily.

For example, if you want to create a JS object literal, you write:

Properties literal = $$("{ foo: 'bar', baz: 'bam'}");

You can then use this Properties literal in GQuery functions ala jQuery, or, you can access them with JS-like syntax

$("div").attr($$("{ foo: 1, bar: 2 }"); // set every DIV to have a foo attribute = 1, and bar attribute = 2

// And;
$$("{foo: 1, bar : 2}").get("foo") == 1;
$$("{foo: 1, bar: { baz: 3} }").get("bar.baz") == 3;

-Ray

GWT: The Road to 1.5, Linkers

The Google Web Toolkit compiler is a marvelous tool for turning Java code into highly optimized Javascript, but prior to GWT 1.5, there was no mechanism for developers to customize the packaging of the compiler output, or the bootstrap of the application.

C developers are used to the ability to override the entry point glue code ('start' method in crt1.o for example), or to package statically, dynamically, etc, so why not provide this capability for GWT developers as well?

This will allow GWT compiled code to packaged and bootstrapped into environments like Flash/AIR Apollo, Gears, Gadget Containers, and the example I'm going to show today: Android Offline/Online Hybrid applications.

What's a Syndroid

As part of my ongoing prototyping in Syndroid, I stumbled onto the idea of a hybrid Android application. One where some of the UI can be rendered via GWT in the embedded WebKit, and the other part, running as native Java/Dalvik code accessing Android APIs.

For example, you may wish to develop a Gadget with GWT, that runs in several containers, but when it runs on Android, it has additional access to information like Contacts, Location, etc. How can GWT code call native Android functions?

Generators Again? The Android Native Interface

Before showing you the magic sauce that permits this, let's look at an example API of how we might like this to work. This will all seem familiar to those who use GWT RPC. First, let's create an interface that will trigger a GWT Generator.

package org.timepedia.syndroid.client.android;
public interface AndroidNativeInterface {}

and a sample interface exposing a method to retrieve our GPS coordinates

package org.timepedia.syndroid.client.android;
@ImplementedBy(LocationServiceImpl.class)
public interface LocationService extends AndroidNativeInterface {
  String getLocation();
}

next, Android implementation code (*warning, code may not work, I just typed it into the blog without testing)

package org.timepedia.syndroid.android;
public class LocationServiceImpl implements LocationService {
   public Location getLocation() {
       LocationManager lm = (LocationManager)
           Context.getSystemService(Context.LOCATION_SERVICE);
       return locationManager.getCurrentLocation("gps").toString();
   }
}

finally, some GWT code that uses it

LocationService lService = (LocationService)GWT.create(LocationService.class);
Window.alert("My location is: "+lService.getLocation());

Great, but how does it work, and how do Linkers fit in?

There are a few ways that RPC calls between GWT and Android could work. For example, a traditional XHR request could be made to a HTTP service running on Android. But this would require the API to be asynchronous and have a high overhead.

It turns out, that the native WebKit that comes with Android has the capability to extend the native Javascript APIs available via Java. For example,

 WebView wv=new WebView(this);
 wv.getSettings().setJavaScriptEnabled(true);
 wv.addJavascriptInterface(new LocationServiceImpl(), "locationService");

This binds our Android LocationServiceImpl instance to a globally scoped JavaScript object called 'locationService', allowing Javascript code to invoke the locationService.getLocation() method.

We now have two problems to solve, first, generate GWT code in a Generator that invokes this $wnd.locationService.getLocation() global object, and secondly, to generate Android bootstrap code (shown above), an AndroidManifest, and package up everything into a APK file that can be installed on a phone. The latter problem is what Linkers help solve.

The Generator's Job

I won't go into detail about how to implement the Generator, as it's been covered before in this blog, the Generator essentially creates a LocationServiceImpl class, with JSNI method getLocation(), which calls $wnd.locationService.getLocation()

The Linker's Job

The Linker has the bulk of the grunt work here. Here's what it must accomplish:
1. Generate Bootstrap Android application
2. For each generated AndroidNativeInterface, add Javascript bindings
3. Place all generated resources in proper Android asset layout
4. Create Android Manifest file
5. Add extra code to load the initial WebView start page with GWT app
6. Invoke Android tools to compile Dalvik code, and package APK files.


The Linker and LinkerContext interface in GWT 1.5 allow one to specify in a module file, which sets of Linkers are run. GWT 1.5 provides some standard ones like the IFrameLinker and XSLinker, which generate the ordinary selection scripts we see in 1.4. The LinkerContext interface provides a mechanism to discover the outputs from the compiler pass, and to emit new ones.

I'll have to end this installment here, because it is getting long and I need to get back to work, in the next installment, I'll show some actual Linker code for an AndroidGadgetLinker.

GWT:The Road To 1.5, Part 1

The next version of Google Web Toolkit is almost upon us, GWT 1.5, but don't take the minor version bump as an indication of how much it's improved, GWT 1.5 has so many awesome improvements, it would be more proper to call it GWT 2.0. That's one reason why it's been almost a year since the last release.

To celebrate the release, I will be writing a series of brief articles on each of the many improvements of 1.5, hopefully with sample code demonstrations.

Zero Overhead Javascript Interop
One of the cool things about GWT has always been the JSNI concept, or 'Javascript Native Interface', that neatly parallels JNI in ordinary Java, except the 'native' implementation is Javascript.

One of the most common uses of JSNI is to produce wrappers for third party Javascript libraries like Scriptaculous, Dojo, ExtJS, etc. Unfortunately, these wrappers are somewhat expensive prior to 1.5

Common Wrap Patterns
There are two common ways people wrap third party Javascript libraries: Encapsulation of JavaScriptObject ('JSO'), or extension of JavaScriptObject. Prior to 1.5, subclassing of JSO worked, but was not officially supported.

Regardless of which technique is chosen, the wrappers work by providing access to Javascript object properties via JSNI getters and setters, and methods via JSNI methods that delegate to Javascript methods.

Unfortunately, delegation getter methods or other instance methods, were not optimized away or inlined by the GWT compiler. Let's look at an example.

public class TestWrapperEncapsulation {
    private final JavaScriptObject jso;
    public TestWrapperEncapsulation(JavaScriptObject jso) {
        this.jso = jso;
    }
    public native String getFoo() /*-{
      return this.@TestWrapperEncapsulation::jso.foo;
    }-*/;
    public static native JavaScriptObject makeObject() /*-{
      return {foo: 'Hello World'}
    }-*/;
    public static TestWrapperEncapsulation create() {
        return new TestWrapperEncapsulation(makeObject());
    }
}

Given the above wrapper class, if you were to write:

Window.alert(TestWrapperEncapsulation.create().getFoo());

You would not end up with the desired optimal JavaScriptCode, e.g.

$wnd.alert(TestWrapperEncapsulation.create().jso.foo)

rather, this

$alert(this$static, $getFoo(create()));

function $alert(this$static, foo){
 $wnd.alert(foo);
}

function $TestWrapperEncapsulation(this$static, jso){
  this$static.jso = jso;
  return this$static;
}

function $getFoo(this$static){
 return this$static.jso.foo;
}

function create(){
  return $TestWrapperEncapsulation(new TestWrapperEncapsulation(), makeObject());
}

function makeObject(){
 return {'foo':'Hello World'};
}

Now let's take a look at GWT1.5's output:

$wnd.alert($TestWrapperEncapsulation(new TestWrapperEncapsulation(), {foo:'Hello World'}).jso.foo);

function $TestWrapperEncapsulation(this$static, jso){
 this$static.jso = jso;
 return this$static;
}

That's a massive reduction, but still not good enough. There is in fact, no reason to have a wrapper encapsulate the JSO. Instead, with GWT 1.5, we can subclass the JSO and pretend that our Java methods exist on the underlying JSO.

alert(TestWrapper.makeWrapper().getFoo());

public class TestWrapper extends JavaScriptObject {
    protected TestWrapper() { }
    public final native String getFoo() /*-{
       return this.foo;
    }-*/;

    public static native TestWrapper makeWrapper() /*-{
       return { foo: "Hello World" } 
    }-*/;
}

and what does the 1.5 compiler produce?

$wnd.alert({foo:'Hello World'}.foo);

Perfect! Absolutely zero overhead.

Chronoscope 0.8 Released / Upcoming Milestones

Major new features:
1) Internet Explorer/Flash Rendering support
2) Experimental Java2D renderer (servlet/applet)
3) Appendable Datasets / Dataset change listeners

Miscellaneous:
4) many bug fixes
5) code aligned with GWT checkstyle

Internet explorer is supported by using a special Flash9 helper which emulates the WHATWG Canvas rather than using VML/ex-canvas as is traditionally done because performance and compatibility is not as good with VML. You can force flash rendering for all browsers by adding ?_force_flash to the URL of your app.

The Servlet/Applet support isn't transparent in the sense that you have to write different code to instantiate a servlet or applet graph. This will be rectified in the near future by harmonizing the chart factory to produce various types (Flash/Canvas, Servlet, Applet) depending on configuration parameters or browser detection, that is, if you lack both Flash9 and are running on IE, but have Java, then it can fall back to Java, and if you don't want Java, it can fall back to asking a servlet to assist.

Appendable datasets allow you to do streaming charts (finance, network sensors, etc) by appending new points to the end of a dataset. This are efficiently integrated into the internal multiresolution datastructure (log(N) ops), and the chart can now detect when the dataset has been mutated and automatically redraw itself.

Also, we are now committing to weekly releases, here are the planned upcoming milestones.

Next Week, 0.85 release
1) RangeMutableXYDataset - all features of Appendable datasets, plus
supports modifying Y values of already added points
2) Auto-precision/scaling of Axis labels

0.90+ releases (not prioritized yet)
1) IE GSS support (use CSS properties to style chart)
2) Shared Domain charts (two plots, sharing a single X axis,
vertically stacked, like Financial Price/Volume charts)
3) Generalized partition strategy for multiresolution algorithm
4) Restore bar chart to full functionality + control over span intervals (bar width)

Please file any bugs or RFEs in the issue tracker, which is available on the project page where you download the code at http://code.google.com/p/gwt-chronoscope

-Ray

Project Syndroid:Synthesis of GWT and Android for Platform Independent Gadgets

Over the course of the last few months, I've steadily documented my work in attempting to produce portable 'cloud-safe' code that runs anywhere, both in the browser, and in native environments, and now I'd like to take it a step further.

Project Syndroid

My vision is to produce a high level Java Gadget API, that allows the authoring of both OpenSocial and Gears enabled gadgets, that can run in a variety of containers and platforms, the biggest difference being the ability to deploy to Java based phones like Android. The presentation below goes into a little more detail of what I'm proposing and why.

| View | Upload your own

I Need the A-Team (cue music)

Project Syndroid is my submission to the Android Developer Challenge. I can't do it alone, I don't have that much time to dedicate to doing all the platforms. However, I do have a handle on doing the GWT version. What I would like to do is find 2 other expert developers, one in Android, and one in doing Gadgets/Widgets for Sidebar/Dashboard/Google Desktop/Konfabulator, and together, as the syndroid-team, we will split any prize money if we win anything in the developer challenge. I'll take on the task of hashing out the GWT plumbing, while the other two developers work on Android and Widget container code, plus packaging/build/deploy tools to build all versions.

I have set up a group, syndroid@googlegroups.com, for those that would like to discuss the project/idea further, or become one of the core syndroid-team members responsible for major development work.

-Ray

Chronoscope Demo in Flash + WHATWG Canvas on IE

Since it's inception, I've been talking about the design goal of Chronoscope as a scalable visualization platform that runs in any environment. However, until recently, one big hole in that vision remained: Internet Explorer. IE does not support the <CANVAS> element, it does however support a retained-mode/scenegraph-style markup language called VML. Many attempts have been made to emulate the canvas using VML, but they all leave something to be desired, especially when it comes to performance.

This leads naturally to thoughts of using Flash. It just so happens I am mostly finished with a Canvas implementation for IE using Flash, and here is a demo of Chronoscope using Flash. Oh, it runs on Internet Explorer now! (Edit: Flash Version 9 plugin required)

Nobody has ever done this before

"That's why it's going to work"

To be fair, many many people have tried this before. Paul Colton for example with AFLAX, and numerous others have floated the idea or prototyped it. When I started, I did not want to waste time duplicating effort, so I searched for any complete WHATWG Canvas emulations I could find for flash, but found none. And to be sure, there were problems with many of the prototypes that turned up, such as trying to map JS calls to CanvasRenderingContext2D directly to Flash calls via Flash's ExternalInterface, which needless to say would be incredibly slow.

I did find one interesting library that would turn out to help me a lot: AS Canvas by MixMedia, an ActionScript implementation of most of the WHATWG Canvas API, not for the browser, but for Flash developers. I am not a Flash developer nor expert, and I was vaguely familiar with the MovieClip API, however puzzles remained as to how to turn what is a scenegraph style API into an immediate mode one. ASCANVAS achieves this by flushing each stroke()/fill() call into a BitmapData object and then clearing the previous drawing commands. That was the inspiration I needed.

Buckle your seatbelt Dorothy, 'cause CANVAS, is going bye-bye.

Well, not exactly. Rather, Flash will become an option for rendering in Chronoscope.

Chronoscope's CANVAS API was designed to help accelerate performance in drawing when individual drawing commands have a high overhead (such as making RPC calls to a Flash VM or doing lots of DOM operations). The way it achieves this is by super-setting the WHATWG Canvas API with several additional features:

• All drawing happens between beginFrame() and endFrame() calls
• Multiple layers can be created within a Canvas and composited
• OpenGL-style display lists which record a sequence of commands and play them back
• Text rendering and specialized text-layers
• Rotated text
• Fast Clear

The beginFrame()/endFrame() abstraction allows the canvas to defer execution and buffer up multiple commands into a single batch. For Flash, this allows an entire frame of drawing commands to be sent to Actionscript for rasterization in a single call. For DOM-oriented interfaces (SVG/VML/Silverlight/etc), it would allow using innerHTML techniques over DOM operations to render the frame.

The layers API yields the possibility of accelerated hit-detection, since layer creation is cheap, while allowing efficient compositing operations that are non-destructive, which allows fast scrolling and the ability to only update layers which change. It also makes the Java2D version work better.

The Display List abstraction allows one to record a bunch of API calls, and play them back over and over, yielding a more compact buffer of commands, as well as the potential to accelerate parsing of the commands, and cache the results of some drawing operations. For example, a display list with 100 drawing operations, could be executed 100 times for a total of 10,000 drawing operations, while the command buffer in the Canvas only stores and transmits a total of 200 commands.

Text rendering is my biggest complaint about WHATWG Canvas, so it was a no-brainer to include it. Finally, Fast Clear is useful for erasing an entire canvas and/or layer if there is a cheaper way of doing it then clearRect(0,0,width,height) for example.

Unfortunately, no one can be told what VML is, you have to experience it for yourself

And once you have, you'll wish you hadn't. Flash/Actionscript3 with MXMLC however turned out to be a pleasure (mostly).

Here's how the Flash Canvas works. Canvas calls are converted into tokenized commands and pushed into a Javascript array, for example lineTo(10,20) becomes array.push('l', 10, 20) beginFrame() clears this array, and endFrame() uses Array.join() to send the entire stream to the Flash Plugin, which as exported an interface. The Flash code parses this Array, and translates Canvas API calls into semantically equivalent Flash operations.

Needless to say, getting all of the WHATWG Canvas semantics correct is tricky. For example, 'globalCompositeOperation' is tricky to implement because Flash lacks Porter-Duff compositing modes. Path drawing and filling, especially with curves, has subtle differences. And drawing Images from the browser requires a lot of bookkeeping.

My current implementation is about 95% complete. I have a few Porter-Duff modes to implement and CanvasPattern. After completion, I'm looking forward to exporting a pure-JS (non-GWT) version of this that can replace excanvas/iecanvas for high performance (and correct) WHATWG rendering in IE6+

Overall, I'm happy performance seems adequate for my uses. Again, here's a demo of Chronoscope using Flash (Flash Version 9 required)

Timepedia now has a graphics/charting platform thats runs in the browser as Javascript (WHATWG Canvas and Flash), as a Applet or Desktop Java (Java2D version), on the Server (Java2D), and in mobile environments (Android, later J2ME). As a future exploration, I'm looking at modifying the GWT compiler to produce ActionScript and translate the entire Chronoscope codebase into an SWF.

-Ray

Chronoscope in Flash soon

Just a quick Chronoscope status update. I've been working the last few days on a Canvas implementation based on using an embedded Flash component. The primary goal of this is to support Internet Explorer which lacks browser canvas support, but it can be used on any browser. Initial performance is on par or superior to native Canvas (and much much better than SVG/VML) I'll be releasing it in the next 1 or 2 weeks as I clean up some glitches in the rendering.

IE Canvas rendering finally solved (and performant!)
-Ray

Google Gears Image Manipulation API not ambitious enough

Google Gears is a very subversive and disruptive technology IMHO, and I mean that in a good way. Google has the muscle to extend native browser functionality in a cross browser way by sneaking extensions into Gears. Of course, Adobe and Microsoft can do this as well (Flash and Silverlight), but the difference is, Google is offering AJAX-level building block extensions to browser functionality, not an alternative environment-within-the-browser-environment that Flash, Silverlight, and Java applets yielded.

Gears is steadily building momentum, and I'm sure many of Google's properties will soon support offline or enhanced modes using it, which will tend to make it a 'must-have' plugin, hopefully achieving 80-90% penetration in the future. So, before the vast majority of people start using the plugin, let's try to be as ambitious with the extension functionality as we can, so when the rush-to-install happens, people will be getting a version with very rich functionality as the base. We want to avoid the need to check Gears version all over the place and force people to upgrade plugins continually ("what, oh, your Gears version 1.21 doesn't have the image.shear() function, you need Gears 1.25 for that...")

Case in point, the proposed Gears Image Manipulation API. Granted, it's just starting, but I'd like to offer some upfront suggestions before this thing gets finalized.

Don't duplicate Canvas, extend it

The proposed API adds resize() and crop() operations to an image object. And also the ability to turn images back into blobs. The resize() and crop() operations can be done today with JS Canvas, but only WHATWG Canvas allows you to turn an image back into data. I don't think this API goes anywhere near far enough to justify its existence.

Also, the biggest pain with using Canvas today is that every browser but IE supports it, so why not implement a cross-browser offscreen Gears WHATWG Canvas API to start with.

But don't stop there. WHATWG Canvas lacks text rendering, and image rendering that obeys affine transforms, two of the big complaints against the existing Canvas. The Web 2.0 world will love anyone who can get such an extended cross-browser canvas widely deployed.

So start with an off-screen WHATWG Canvas API, add text rendering (and atleast 90 degree rotated text would be nice), plus drawImageWithTransform() that obeys transforms.

Resize, Flip, and Crop aren't enough

Anyone looking to build a client-side photo-manipulation library will want more than just image scaling, cropping, composing, and flipping. They need the ability to run convolution kernels, lookup tables, and rescale/colorspace transforms as well. The most common operations people want to run on photographs, like contrast/brightness enhancement, sharpen/unsharpen, conversion to black and white or sepia, etc use these.

Don't forget RAW and EXIF/image metadata

In addition, the ability to open RAW files, manipulate exposure compensation, and extract image metadata would be a huge boon. An offline Gears photo album would be much cooler if EXIF info could be extracted as images are imported.

My own wish list for Gears Image API

• Implements WHATWG CanvasRenderingContext2D as Base
• Adds text rendering, with minimally 90 degree rotation support 
• Image composition that obeys affine transforms (not just rotate)
• Convolution and Lookup operators (NxN square kernels, N atleast up to 5)
• Support opening RAW images
• Support read (write would be good too!) access to image metadata

Objections?

The first objections that will be raised is bloat in the plugin. Leaving aside the fact that Flash delivered outstanding capability in a slim plugin, the proposed libgd implementation already has many of the core functions needed to satisfy the proposed richer API, and I'm sure it would not be hard for Google engineers to implement the rest. Convolves and Lookups aren't rocket science.

A likely second objection and real hiccup will be achieving cross-platform, antialiased, internationalized text rendering with rotations. I don't have an answer for this one, only that users want it. Almost everyone I've talked to who does client-side rendering wants this.

A third issue is simply complexity and time to market. Resize, Flip, Rotate, and Crop are trivial to implement as simple libgd glue code, whereas full support of WHATWG semantics, would require significantly more time. Although some of this might be mitigated by stealing WebKit or Gecko's implementation and hacking it into Gears.

Fourth, dealing with large photographs, especially RAWs will bring memory issues to bear (unless a smart tile cache oriented system is used), and could be a point of denial-of-service against the Gears plugin if care isn't taken.

Regardless of these objections, I'd still urge Google to go for it. Make client-side image processing and visualization a major part of the next Gears API. Please!

-Ray

Hardcore GWT Hands on Training?

It's the new year, and I've been mulling an idea recently that I'd like to get some feedback on from the community.

The GWT Conference was a blast, and it appears my Deferred Binding presentation was well received. Since then, a number of people have encouraged me to offer some sort of training or mentoring for GWT, either in classrooms, or in the form of corporate on-site training. I hesitate to get too distracted from working on Chronoscope and Timepedia, but after thinking about it some more, I do think it might be worth pursuing, perhaps one 2-day course per month.

What I don't want to do is some token GWT course that glosses over the core of GWT and shows one how to create some cute widgets and stuff them into a web page, without understanding what's going on under the hood. Rather, I'd like to teach, on Day 1, GWT from the ground up exploring in detail what is happening: Java vs JS Output, Deferred Binding, JSNI, Widget attachment, Widget event processing, RPC/Serialization vs On-the-wire representation, the Bootstrap/Selection process, Hosted Mode, debugging, testing, etc.

On Day 2, I'd then delve into more idiomatic patterns: Creating custom widgets, creating your own modules/libraries, Localization, image bundles, integrating with back-end frameworks, RPC and RPC customization, optimizing size and speed, etc

I'd require people to write code in short labs during the class, give personal help in troubleshooting, setting up Eclipse/IDEA/Netbeans environments, and ask that others who have completed labs successfully help participate in helping those having trouble, because I believe a good way to learn something is by in-class participation and trying to help someone else. Who knows, it might be a fun, campfire type atmosphere!

I'd risk losing a lot of newbies doing this, and also risk not getting any advanced users who are self-taught, but hopefully, there are a lot of people in the middle with some GWT experience, who would like a more in depth tutorial hands on. Perhaps I'd do a more marketing oriented/newbie introductory course later.

Ultimately, I'd hope that people who come away from the classes with not only enough knowledge to write code to cover their own use cases, but also have enough knowledge to fix bugs and contribute to GWT itself. The GWT compiler and core libraries should no longer be 'magic' to those taking the course.

Obviously, the course wouldn't be free, especially in offsite training where I'd have to rent the event space and also given the opportunity cost for me, but I would offer discounts if space is donated, charge less for the first 'beta' class, and given some steep student discounts, especially if students would arrive early and help me with class setup.

I'm working out the course material at the moment, and pricing out some meeting spaces. I haven't decided on a per-student price yet, but feel free to email me suggestions, course ideas, and most importantly, whether or not you'd be interested in attending, or your company would. Private replies can be sent to cromwellian / gmail.com

Oh, this would be for the SF Bay Area, hopefully BART/Caltrain accessible.

-Ray