Pages

January 15, 2013

Firefox 18 Gets Reviewed and Benchmarked Against Chrome 24, Internet Explorer 10



Keeping up with the schedule Mozilla has shipped another release of Firefox for Desktop. The latest release from Firefox has some really impressive features up its sleeves. It is noteworthy that majority of the changes pertain to under the hood improvements resulting in direct performance gains. As always, we will be comparing the top browsers based on benchmarks. This time around, we have also included the latest Chrome release, Chrome 24 which released closely following Firefox 18’s release.
How does Firefox 18 fare against Chrome 24 & Internet Explorer 10? Well, let’s have a look.


firefox cover
(image courtesy)

Faster JavaScript performance via IonMonkey compiler

IonMonkey Compiler succeeds Firefox’s own TraceMonkey, and JägerMonkey Javascript Compilers bringing in impressive performance gains of up to 26% which is incredible. The new IonMonkey compiler will be especially useful in heavy games and websites making extensive use of JavaScript. Speaking crudely, as Firefox is built on XUL which is again based on JavaScript, the new IonMonkey Compiler should definitely lay foundation to some solid speed improvements everywhere. On normal usage, the performance is agreeably more snappy than before, a welcome benefit especially if you browse JavaScript heavy sites. As you will see in the benchmarks below, IonMonkey does indeed benefit Firefox in performance considerably.

Support for Retina Display on OS X 10.7 and up

Firefox has been surprisingly slower on implementing Mac OSX feature support, be it trackpad gestures on Lion or Full screen mode. Nevertheless after Chrome, Firefox has finally added proper Retina-display support for Mac OS X Lion and above. This would come as a major relief as anyone who would have used a retina MBP would have encountered the sheer pain of using apps that do not support retina scaling and do their own sub-pixel rendering.

Preliminary support for WebRTC

WebRTC stands for Web Real Time Communication and is a open source initiative led by Google to enable open standards for communications like Voice Calling, Video Chat and P2P file sharing without any plugins. In simpler terms, you could think of WebRTC as open alternative to Microsoft Silverlight and Adobe Flash.
Google Chrome (obviously) was the first web browser to support WebRTC, followed by Opera. Finally, Firefox has done the catching-up act by implementing preliminary support for WebRTC. Remember though that the WebRTC standard is still not yet complete.

Performance Improvements

Apart from the newly introduced IonMonkey JIT, Firefox 18 features various other improvements as well.
  • Better Image Quality due to new HTML Scaling algorithm: A very old bug that was not marked as fixed until this release, resulting in better quality of images within websites. This fix had landed in Aurora as early as in October but obviously due to the release cycle, took quite while to land in the stable channel. You can see the difference it makes in this image. The one on the right is Firefox 18
  • Improvements around Tab Switching: Firefox 18 also improves upon tab switching performance, which should be subtly noticeable to casual users.
  • Lower Startup times : Firefox should now start a lot more quickly. Firefox earlier would unnecessarily check for a certificate which would end up in delay in startup due to time spent in validating. This has been done away with. The result is quite impressive and would be quite noticeable for all users.
  • Performance improvements for Proxy users :Finally Firefox 18 includes significant performance improvements for users using proxies. The reason turns out (funnily) that none of them at Mozilla used proxies, but acting upon a complaints about proxy performance lead to plenty of fixes in the network stack.

Firefox 18 vs Chrome 24 vs Internet Explorer 10

Benchmarks

This time around we included the top three browsers only, excluding Opera due its slouch for a long while. We have expanded our benchmarks this time around , with two new entrants Psychedlic Browsing and Peacekeeper. We have compared Firefox 18 against the most recent stable releases of Chrome and Internet Explorer respectively. ( Chrome 24  and Internet Explorer 10.0.1)
All the tests were performed on a HP-G42 478TX laptop with Core i3 2.40 GHz (first gen) processor, 6 GB DDR3 RAM, Western Digital 5400 RPM hard disk and ATI Mobility Radeon 6370M GPU running Windows 8 Pro with all the latest updates installed
Please note that all the add-ons were disabled in Chrome and Firefox. Also hardware acceleration was enabled across all the browsers

Sunspider

Sunspider, the much acclaimed JavaScript benchmark showed no visible gains….for Firefox. Continuing it’s tradition of kicking some serious arses in this benchmark ( which started with IE 9) Explorer 10 was around 32% faster than Firefox. Finally, it is about high time to end all those meme’s targeting Internet Explorer. Tsk Tsk.
image

Peacekeeper

Peacekeeper by Futuremark indicates overall performance of the browser by performing all kind of test a humble geek could think of for a browser. Tests vary from rendering HTML5 videos in various codecs, rendering webGL, DOM trees, Javascript assessment, 2D game performance, string manipulation tests and what not. Peacekeeper in a nutshell is a very comprehensive browser benchmark. Speaking of the scores sadly, Chrome demolished Explorer 10 and Firefox by margin of more than 85%, which might just point Mozilla dev’s that they need to look at other aspects of the browser as well apart from churning out the numbers. Explorer surprisingly matched up to Firefox quite well in this benchmark.
Something that should be duly noted is the lack of H264 codec playback in Firefox might have cost some loss in the score
image

Google Octane

Octane, the successor of the famous v8 benchmark performs various types of JavaScript tests, which emphasizes on performance of a browser in modern web applications. Firefox 18’s IonMonkey surely seems to help here, coming fairly close to Chrome this time around. This is a very naive assumption though, because the underlying tests could be fairly different between Google v8 and Octane. Explorer however was a sad story, coming a (very) distant third.
image

Psychedelic Browsing

You can take this benchmark with a pinch of salt just because it is developed by Microsoft. (seriously though, you be sure of anything without having a look at the source code, speaking of which reminds me that none of the IE Test Drive benchmarks are open). Just as expected, Internet Explorer bests the competition easily. However, Firefox is not far behind this time around with a pretty decent score. Chrome, for a change bites the dust.
image

Kraken

Perhaps the only comprehensive benchmark in our tests which has witnessed a lot of changes in browser performance over the course of time. There used to be a time when Chrome and Firefox would go neck and neck over in Kraken and at times Chrome would trump Firefox quite convincingly. Finally, we witnessed the contrary this time around (thanks to IonMonkey JIT for this). Firefox 18 beat Chrome 24 by a pretty wide margin. Also, we must stress the sheer quality of work being done on both the camps, with the scores improving (or falling, technically) with every release. The touted JavaScript improvements in both Chrome 24 and Firefox 18 show up clearly. Internet Explorer 10 though, decides to have a mind of it’s own.
image
Real World Performance : Memory usage
While gauging CPU usage for browsers is somewhat a tough job, checking memory consumption while being the easier chore often indicates the resource usage of a browser on a abstract level. This time, we have also included memory consumption when using a single tab, which agreeably quite a lot of normal PC users do.

Memory Consumption with Single Tab Open

What would be the site most of the PC’s would have open? Facebook. Pretty straight forward. We started each of the browsers afresh, opened Facebook and waited for two minutes after all the content had been loaded to get a sharp picture of memory consumption when idle.
image
Yup. We were surprised too. Internet Explorer 10 was the browser consuming least amount of memory. We re-did the tests, cross verified results with different tools to obtain the same result. Internet Explorer 10 is well ridiculously memory efficient with single (or few tabs open, say 2 3). Firefox was not lagging far behind as well, luckily. Though Chrome was the resource-hog clearly taking a ludicrous 315 MB to display a single Facebook tab.

Memory Consumption with 10 Tabs open

We opened 10 sites of varied complexity and waited for 4 minutes after all tabs were done loading for the memory consumption to stabilize. Firefox, quite simply displayed it’s natural strength which has been the result of very diligent efforts going into reducing memory consumption from past year, consuming only 370 MB! Internet Explorer and Chrome were far, far behind with the former bringing Chrome to shame with a impressive memory consumption of 621 MB. Chrome had successfully managed  that it’s idea of sumptuous lunch was gobbling your RAM at will.
image

Now for those Chrome fans who would argue that Chrome opens each tab in a separate process the memory consumption would high well, Internet Explorer 10 does the same but still keeps the memory consumption in acceptable limits. Ultimately, it is beyond doubt that Chrome needs to trim down it’s memory consumption.

Firefox 18 vs Chrome 24 vs Internet Explorer 10 : Where does each browser stand?

Internet Explorer [10]: While some would say that Internet Explorer 9 was Redmond’s redemption, Internet Explorer 10 goes way further and undoubtedly is a very solid performer that comprehensively beats Chrome, be it in either memory management or in a host of other benchmarks. The cool thing resulting from this is, all the not-so-tech savvy PC users who mostly do not even bother to download an alternative browser when they buy a new PC will not be browsing the Internet with some-substandard-pain-the-rear browser shipped with windows thanks to Windows 8’s Internet Explorer 10.
Internet Explorer 8, if you had forgotten which shipped with Vista/7 PC’s was so terrible that it almost took a small nap before it finally loaded. Oh, don’t rejoice that it is unmaintained yet, Microsoft is just done releasing a fix-it for Internet Explorer 7/8’s vulnerability

Chrome : Chrome 24 has definitely improved its performance especially in Javascript as touted, but no longer we can proclaim it as the “king” of the browsers. Both Internet Explorer and Firefox beat Chrome comprehensively in different tests. Also, Chrome certainly needs to work on the memory consumption. That said Chrome is a solid browser with special brownie points going to it’s UI responsiveness and hassle-free Chrome Sync.


Firefox : The devs at Mozilla are working really hard to improve Mozilla performance, and it truly shows. Firefox 18, as it’s obvious from these benchmarks, is giving both IE 10 and Chrome a good run for the money. BUT, the devs don’t seem to care about Firefox’s UI responsiveness. Yes, there is Project Snappy and we can observe some noticeable improvements in this release, but that still does not help Firefox; faring badly against Chrome and Internet Explorer. Page loading times are getting quicker with every release, neat but how about improving UI snappiness? Because responsiveness, and not benchmarks, drive people’s browser choice.



For more such updates, You can follow us on Twitter - @TechSplurge.