Along with the long list of new features and improvements we can find in Firefox 3 betas so far, there is a series of performance improvements coming to Firefox 3 Beta 4.
I’ve finally managed to run a set of tests to see how Firefox 3 performance compares to Firefox 2’s.
It must be noted, that as any benchmark tool it is not perfect or free of criticism: an engine will most probably perform better in one test than another.
It is also said that an engine or application could be tweaked to perform better in a particular well known test. I think if the test is well designed and effectively represents what affects most users, then tweaking those particular factors only makes sense. What would be the point of fixing some obscure interfaces if it’s not what real world applications will benefit from?
Firefox 2 has fame for consuming lots of memory and most annoyingly eating more of it when it is supposed to be sleeping.
So I designed a simple test with a set of 9 web pages on Windows XP SP2, with an empty profile and a blank page set as home page. Measures were taken with Windows Task Manager. The test covers a few simple web pages with simple browsing. Then more complex pages are added keeping the first pages opened. A couple of tabs are then closed to see how much memory is freed and finally the browser is left running for an hour to see if memory consumption remains flat.
Here are the steps in case you want to reproduce:
1. Visited The New York Times, clicked on Technology section, open a couple of articles
2. Visited Digg, clicked on Upcoming and clicked on pages 2 and 3
3. CNET News, clicked on each of the top three stories. Get back to the homepage afte reach
4. To cover international support, I visited the Russian and
5. Chinese Editions of Wikipedia. I then clicked on the first link in the left sidebar on each and then went back to the main page.
A. I took a first measure here.
6. Visited this YouTube video, view the video and one more from the right related videos sidebar.
7. On Gmail, I logged in, composed an email with 2 large images (about 2MB each), sent it to myself, and then opened the received email.
8. On Google Docs, I created a new text document, wrote a five word sentence, made it bold, centered it, added a small 20K GIF image and saved the document.
9. I visisted Netvibes, visit all the default tabs, change the page title and added a short item to the todo list
B. Second measure here.
10. Then closed Netvibes and You Tube and waited 20 seconds for Firefox to do its cleanup.
C. Third measure here.
12. The I left it opened for about one hour.
D. Last measure here.
|Measure||Firefox 22.214.171.124||Firefox 3 Beta 3|
|A. Regular web pages||73 MB||80 MB|
|B. Web 2.0 web sites||149 MB||135 MB|
|C. Close a couple of pages||142 MB||123 MB|
|D. After 1 hour with no activity||181 MB||123 MB|
As you can see, Firefox 3 shows better memory management as it freed more memory when a couple of tabs were closed and it kept memory usage flat when not in use.
This test is far from comprehensive: depending on your browsing habits you may have much more or less tabs opened at the same time. Also, memory piles up as you close and open new tabs and windows, with more and different plugins. So this is just a reference of what some very casual browsing takes on each version and most importantly shows the many memory management improvements Firefox 3 will introduce.
I restarted my laptop before each test to get a clear cache and good cold start times. Then closed and restarted Firefox to see how much it benefits from the OS caching.
In summary I got the same results: 10 seconds for a cold start, 4 seconds for a restart. There may be some milliseconds there but that’s beyond what a regular user can perceive.
Starting today, Firefox nightly Windows builds have the benefit of profile guided optimization (PGO). With PGO, after Firefox is compiled (written code is made ready for the computer as binary code), the just generated binary is ran while another application monitors how it behaves and creates a profile of what parts of Firefox are more used and in what order. Then the code is compiled again but this time guided by the produced profile generating a new optimized build.
Ted Mielczarek, who led the effort to PGO-enable the Mozilla build process, says a 10% performance gain can already be seen in some of Mozilla’s performance metrics. Linux builds may get PGO after Beta 4 and hopefully Mac OS X builds wil also benefit from it before the final release.
Another improvement came a few weeks ago as told by Vladimir Vukicevic thanks to the discovery of an undocumented Mac OS X interface that will now help Firefox do some work instead of just waiting for its turn to draw, which translates to more efficiency.
Firefox 3 Beta 4 should be out within the next couple of weeks.