Chronoscope Demo in Flash + WHATWG Canvas on IE

Since it's inception, I've been talking about the design goal of Chronoscope as a scalable visualization platform that runs in any environment. However, until recently, one big hole in that vision remained: Internet Explorer. IE does not support the <CANVAS> element, it does however support a retained-mode/scenegraph-style markup language called VML. Many attempts have been made to emulate the canvas using VML, but they all leave something to be desired, especially when it comes to performance.

This leads naturally to thoughts of using Flash. It just so happens I am mostly finished with a Canvas implementation for IE using Flash, and here is a demo of Chronoscope using Flash. Oh, it runs on Internet Explorer now! (Edit: Flash Version 9 plugin required)

Nobody has ever done this before

"That's why it's going to work"

To be fair, many many people have tried this before. Paul Colton for example with AFLAX, and numerous others have floated the idea or prototyped it. When I started, I did not want to waste time duplicating effort, so I searched for any complete WHATWG Canvas emulations I could find for flash, but found none. And to be sure, there were problems with many of the prototypes that turned up, such as trying to map JS calls to CanvasRenderingContext2D directly to Flash calls via Flash's ExternalInterface, which needless to say would be incredibly slow.

I did find one interesting library that would turn out to help me a lot: AS Canvas by MixMedia, an ActionScript implementation of most of the WHATWG Canvas API, not for the browser, but for Flash developers. I am not a Flash developer nor expert, and I was vaguely familiar with the MovieClip API, however puzzles remained as to how to turn what is a scenegraph style API into an immediate mode one. ASCANVAS achieves this by flushing each stroke()/fill() call into a BitmapData object and then clearing the previous drawing commands. That was the inspiration I needed.

Buckle your seatbelt Dorothy, 'cause CANVAS, is going bye-bye.

Well, not exactly. Rather, Flash will become an option for rendering in Chronoscope.

Chronoscope's CANVAS API was designed to help accelerate performance in drawing when individual drawing commands have a high overhead (such as making RPC calls to a Flash VM or doing lots of DOM operations). The way it achieves this is by super-setting the WHATWG Canvas API with several additional features:

• All drawing happens between beginFrame() and endFrame() calls
• Multiple layers can be created within a Canvas and composited
• OpenGL-style display lists which record a sequence of commands and play them back
• Text rendering and specialized text-layers
• Rotated text
• Fast Clear

The beginFrame()/endFrame() abstraction allows the canvas to defer execution and buffer up multiple commands into a single batch. For Flash, this allows an entire frame of drawing commands to be sent to Actionscript for rasterization in a single call. For DOM-oriented interfaces (SVG/VML/Silverlight/etc), it would allow using innerHTML techniques over DOM operations to render the frame.

The layers API yields the possibility of accelerated hit-detection, since layer creation is cheap, while allowing efficient compositing operations that are non-destructive, which allows fast scrolling and the ability to only update layers which change. It also makes the Java2D version work better.

The Display List abstraction allows one to record a bunch of API calls, and play them back over and over, yielding a more compact buffer of commands, as well as the potential to accelerate parsing of the commands, and cache the results of some drawing operations. For example, a display list with 100 drawing operations, could be executed 100 times for a total of 10,000 drawing operations, while the command buffer in the Canvas only stores and transmits a total of 200 commands.

Text rendering is my biggest complaint about WHATWG Canvas, so it was a no-brainer to include it. Finally, Fast Clear is useful for erasing an entire canvas and/or layer if there is a cheaper way of doing it then clearRect(0,0,width,height) for example.

Unfortunately, no one can be told what VML is, you have to experience it for yourself

And once you have, you'll wish you hadn't. Flash/Actionscript3 with MXMLC however turned out to be a pleasure (mostly).

Here's how the Flash Canvas works. Canvas calls are converted into tokenized commands and pushed into a Javascript array, for example lineTo(10,20) becomes array.push('l', 10, 20) beginFrame() clears this array, and endFrame() uses Array.join() to send the entire stream to the Flash Plugin, which as exported an interface. The Flash code parses this Array, and translates Canvas API calls into semantically equivalent Flash operations.

Needless to say, getting all of the WHATWG Canvas semantics correct is tricky. For example, 'globalCompositeOperation' is tricky to implement because Flash lacks Porter-Duff compositing modes. Path drawing and filling, especially with curves, has subtle differences. And drawing Images from the browser requires a lot of bookkeeping.

My current implementation is about 95% complete. I have a few Porter-Duff modes to implement and CanvasPattern. After completion, I'm looking forward to exporting a pure-JS (non-GWT) version of this that can replace excanvas/iecanvas for high performance (and correct) WHATWG rendering in IE6+

Overall, I'm happy performance seems adequate for my uses. Again, here's a demo of Chronoscope using Flash (Flash Version 9 required)

Timepedia now has a graphics/charting platform thats runs in the browser as Javascript (WHATWG Canvas and Flash), as a Applet or Desktop Java (Java2D version), on the Server (Java2D), and in mobile environments (Android, later J2ME). As a future exploration, I'm looking at modifying the GWT compiler to produce ActionScript and translate the entire Chronoscope codebase into an SWF.

-Ray