The current browser wars make me feel like I'm watching the Fast and the Furious, and Google's new Chrome browser is like Nitrous Oxide for the web. As soon as I heard that ex-HotSpot guys were working on it, I knew it could be good, as they did a marvelous job with Self (a fully dynamic language), as well as Java, but how good?
To test, I decided to use my own codebase, Chronoscope, for several reasons. First of all, it is not a microbenchmark, it exercises a big swath of the browser's code path. Secondly, it is computationally intensive, so if anything were going to show speed on a "real world" app, this would be it. Finally, I need visualizations to run faster, as I want to use Open Web technologies like Canvas or SVG rather than relying on Flash.
Chronoscope has a workload which contains elements of the following:
- Large amount of affine transforms calculated within Javascript code
- Many binary searches and recursive calls
- Frequent random and linear access to large objects (arrays with 30,000+ elements)
- Numerous DOM calls to absolute position DIVs (emulate text rendering on Canvas)
- Huge number of CanvasRenderingContext2D calls for rendering the chart
- Ubiquitous use of abstraction, delegation, inheritance
Chronoscope is written in GWT, and to some extent, the GWT compiler may negate some of Chrome's V8 technology in the sense that GWT "de-classes" many OO polymorphic dispatches into a more functional style of programming, removing as much dynamic dispatch as possible, and eliminating prototype lookups and function call overhead through inlining. I don't know if GWT hurts "hidden classes" or not, but it might be possible that if GWT didn't provide such optimizations, the performance differential might be larger.
Despite this, the results are still good. The test consisted of calling the chart's redraw() function 100 times per trial, with 10 trials. The slowest and fastest trial are thrown out, and the mean and standard deviation are calculated on the remaining data.
I tested on a Mac Pro 2.66Ghz with 6Gb of memory, OSX 1.5. The tests were conducted within a Parallels VM running XP2 Service Pack 2, given 2 CPUs and 2Gb of memory. For each browser, I rebooted the VM from a clean start, and ran only the test browser.
The tests were conducted with the lastest version of Safari 3.1 for Windows (run via WebKit nightly batch file), the Firefox 3.1 nightly with TraceMonkey enabled, and the Chrome beta download.
| Browser | Mean | Standard Deviation | Memory |
|---|---|---|---|
| Firefox 3.1 (TraceMonkey) | 3647ms | 81ms | 49M |
| Safari 3.1 (SquirrelFish) | 3005ms | 385ms | 106M |
| Chrome | 1690ms | 190ms | 44M |
Chrome looks to be twice as fast as the competition on real world apps. Moreover, it's performance per Mb of memory is good as well. Even more interesting was the variation in Firefox 3.1, which appears to be related to garbage collection or memory allocation. In some runs, it was very fast, but in others, it was 50% slower.
As soon as Chrome is released on OSX, it will replace Safari for me, for two reasons. The first, the process isolation and speed features. Secondly, Chrome includes a very nice Firebug-like debugger/inspector that I like better than the Safari equivalent.
So in summary, Chrome rocks! All they were missing from the launch was Vin Diesel.
-Ray
10 comments:
hi Ray,
SquirrelFish is enabled with Safari 4.0 not 3.1.*. Have you run tests using WebKit built? If so which one? Otherwise table is misleading in browser name/version description,
regards,
Peter
I have seen similar comparison on the web since yesterday, but people who where using FF3.1 all said it had TraceMonkey JIT turned off (that's the default), as it would not work. Yet the results are the same (FF is 2x slower). Are you sure your FF3.1 build was IT enabled ?
I don't see how the chrome inspector can be better than the standard webkit inspector. Given that it *is* the webkit inspector. Just an old revision that comes from slightly after the Safari 3.1 version (Chrome is based on safari 3.1's webkit)
piotr,
I ran the tests with WebKit nightly, however when I run the batch file to start WebKit, it launches Safari in Windows, which reports "3.1.2" when you choose About from the menu. It does appear to be using the nightly WebKit core.
Also, I specifically verified I had the TraceMonkey JIT option enabled in about:config for the Firefox 3.1 nightly. (I did not enable the JIT for Chrome)
@oliver,
You're write, I made a mistake. I confused Firebug lite with the Chrome inspector. It's the same (sigh). I much prefer Firebug's inspector in Firefox because it is not read only. You can play 'what if' scenarios and edit the styles in real time.
Have you enabled the deveoper tools in Safari? (Edit|Preferences|Advanced -> Enable Developer Menu)
That will give you the Web Inspector, and if you use a nightly or the Safari 4 preview from developer.apple.com you get a more up to date version of the inspector.
Chrome is basically Safari 3.1 (in terms of the webkit revision it uses, but they pulled in a slightly more up to date inspector, something from between Safari 3.1 and WWDC) -- The inspector in the more recent builds is much improved.
Where's Opera ?
@saulob,
You can try Opera yourself on http://api.timepedia.org/benchmark
I didn't post the results because I need to add a specific change for Opera's canvas to stop it from flickering (Opera has a specific Canvas extension called GameCanvas to prevent this flickering)
For a bit wider (but shallower) test of just SVG & DOM manipulation for Opera, FF, IE/ASV, i-phone, Safari, Chrome, see
http://srufaculty.sru.edu/david.dailey/Benchmarks.htm
Just ran http://api.timepedia.org/benchmark in chrome 2, opera 9.5 and FF 3.5 on my PC. Opera results may be somewhat distorted - it's my default browser, and it had 14 tabs open while performing the test. Here are the results:
Chrome 2:
Summary Mean: 2830.25 stddev: 141.97424942573213
Estimated FPS 35.33256779436446 frames per second
Speed happens at the cost of quality: the rendering is wrong (both Opera and FF render the chart the same way, which is diferent from what chrome shows).
FF 3.5:
Summary Mean: 6505.75 stddev: 29.153687588365216
Estimated FPS 15.371017945663452 frames per second
Opera 9.5:
Summary Mean: 5993.5 stddev: 73.52720584926371
Estimated FPS 16.68474180362059 frames per second
My personal conclusion: if you want just speed, go with Chrome. If you also need standards compliance, but speed also counts, go with Opera. Using Opera also has some other benefits: it has the most user-friendly UI out there. Drawbacks for Opera: some apps from google don't work properly in it (like for instance calendar).
Post a Comment