Tuesday, January 29, 2008

Chronoscope Demo in Flash + WHATWG Canvas on IE

Since it's inception, I've been talking about the design goal of Chronoscope as a scalable visualization platform that runs in any environment. However, until recently, one big hole in that vision remained: Internet Explorer. IE does not support the <CANVAS> element, it does however support a retained-mode/scenegraph-style markup language called VML. Many attempts have been made to emulate the canvas using VML, but they all leave something to be desired, especially when it comes to performance.

This leads naturally to thoughts of using Flash. It just so happens I am mostly finished with a Canvas implementation for IE using Flash, and here is a demo of Chronoscope using Flash. Oh, it runs on Internet Explorer now! (Edit: Flash Version 9 plugin required)


Nobody has ever done this before

"That's why it's going to work"

To be fair, many many people have tried this before. Paul Colton for example with AFLAX, and numerous others have floated the idea or prototyped it. When I started, I did not want to waste time duplicating effort, so I searched for any complete WHATWG Canvas emulations I could find for flash, but found none. And to be sure, there were problems with many of the prototypes that turned up, such as trying to map JS calls to CanvasRenderingContext2D directly to Flash calls via Flash's ExternalInterface, which needless to say would be incredibly slow.

I did find one interesting library that would turn out to help me a lot: AS Canvas by MixMedia, an ActionScript implementation of most of the WHATWG Canvas API, not for the browser, but for Flash developers. I am not a Flash developer nor expert, and I was vaguely familiar with the MovieClip API, however puzzles remained as to how to turn what is a scenegraph style API into an immediate mode one. ASCANVAS achieves this by flushing each stroke()/fill() call into a BitmapData object and then clearing the previous drawing commands. That was the inspiration I needed.

Buckle your seatbelt Dorothy, 'cause CANVAS, is going bye-bye.


Well, not exactly. Rather, Flash will become an option for rendering in Chronoscope.

Chronoscope's CANVAS API was designed to help accelerate performance in drawing when individual drawing commands have a high overhead (such as making RPC calls to a Flash VM or doing lots of DOM operations). The way it achieves this is by super-setting the WHATWG Canvas API with several additional features:

  • All drawing happens between beginFrame() and endFrame() calls
  • Multiple layers can be created within a Canvas and composited
  • OpenGL-style display lists which record a sequence of commands and play them back
  • Text rendering and specialized text-layers
  • Rotated text
  • Fast Clear
The beginFrame()/endFrame() abstraction allows the canvas to defer execution and buffer up multiple commands into a single batch. For Flash, this allows an entire frame of drawing commands to be sent to Actionscript for rasterization in a single call. For DOM-oriented interfaces (SVG/VML/Silverlight/etc), it would allow using innerHTML techniques over DOM operations to render the frame.

The layers API yields the possibility of accelerated hit-detection, since layer creation is cheap, while allowing efficient compositing operations that are non-destructive, which allows fast scrolling and the ability to only update layers which change. It also makes the Java2D version work better.

The Display List abstraction allows one to record a bunch of API calls, and play them back over and over, yielding a more compact buffer of commands, as well as the potential to accelerate parsing of the commands, and cache the results of some drawing operations. For example, a display list with 100 drawing operations, could be executed 100 times for a total of 10,000 drawing operations, while the command buffer in the Canvas only stores and transmits a total of 200 commands.

Text rendering is my biggest complaint about WHATWG Canvas, so it was a no-brainer to include it. Finally, Fast Clear is useful for erasing an entire canvas and/or layer if there is a cheaper way of doing it then clearRect(0,0,width,height) for example.




Unfortunately, no one can be told what VML is, you have to experience it for yourself


And once you have, you'll wish you hadn't. Flash/Actionscript3 with MXMLC however turned out to be a pleasure (mostly).

Here's how the Flash Canvas works. Canvas calls are converted into tokenized commands and pushed into a Javascript array, for example lineTo(10,20) becomes array.push('l', 10, 20) beginFrame() clears this array, and endFrame() uses Array.join() to send the entire stream to the Flash Plugin, which as exported an interface. The Flash code parses this Array, and translates Canvas API calls into semantically equivalent Flash operations.

Needless to say, getting all of the WHATWG Canvas semantics correct is tricky. For example, 'globalCompositeOperation' is tricky to implement because Flash lacks Porter-Duff compositing modes. Path drawing and filling, especially with curves, has subtle differences. And drawing Images from the browser requires a lot of bookkeeping.

My current implementation is about 95% complete. I have a few Porter-Duff modes to implement and CanvasPattern. After completion, I'm looking forward to exporting a pure-JS (non-GWT) version of this that can replace excanvas/iecanvas for high performance (and correct) WHATWG rendering in IE6+

Overall, I'm happy performance seems adequate for my uses. Again, here's a demo of Chronoscope using Flash (Flash Version 9 required)

Timepedia now has a graphics/charting platform thats runs in the browser as Javascript (WHATWG Canvas and Flash), as a Applet or Desktop Java (Java2D version), on the Server (Java2D), and in mobile environments (Android, later J2ME). As a future exploration, I'm looking at modifying the GWT compiler to produce ActionScript and translate the entire Chronoscope codebase into an SWF.

-Ray

3 comments:

Unknown said...

Great post thank you much. I do have a question. Can the flash rendered graphs be styled with GSS. If so how can I do that using GWT. I added style to the panel control but it does not seem to take.

mrclay said...

"I'm looking forward to exporting a pure-JS (non-GWT) version". I'm still looking forward to this, too ;)

Anonymous said...

(法新社倫敦四日電) 英國情色大亨芮孟的a片下載公司昨天AV片說,芮孟日成人影片前去成人網站世,sex享壽八十二歲;色情這位身av價上億的房地產開發情色電影商,曾經在倫敦推成人網站出第一場脫衣舞表av演。

色情影片
芮孟的財產成人影片估計成人達六億五千萬英鎊(台幣將a片近四百億),由於他名下事業大多分布在倫敦夜生活區蘇活區色情成人因此擁有「蘇活情色視訊之王」日本av的稱號。
部落格

他的成人電影公司「保羅芮孟集團」旗成人網站下發行多a片種情色雜av誌,包括「Razavzav女優leavdvd」、「男性世界」以及「Mayfai情色電影r」。色情a片
a片下載
色情
芮孟情色本名傑福瑞.安東尼.奎恩,父av女優親為搬運承a片包商。芮孟十五歲離開學校,矢言要在表演事部落格業留名,起先表演讀心術,後來成為巡迴歌舞雜耍表演av女優的製作情色人。


許多評論a片成人電影認為,他把情色表演帶進主流社會,一九五九部落格年主持破天荒的脫衣舞表演,後來成人影片更靠著在蘇活區與成人光碟倫敦西區開發房地產賺得大筆財富。


有人形容芮孟是英國的海夫納,地位等同美國的「花花公子」創辦人海夫納。