There is no WebKit on Mobile

Last week I spent a lot of time on WebKit in order to produce a comprehensive comparison of all WebKits. My purpose was to prove there is no “WebKit on Mobile,” and to gain some more insight in the complicated relations between the various WebKits.

Senior web thinkers from Alex Russell to Tim Bray have envisioned a bright new future for the mobile web based on the argument that “WebKit on Mobile” is taking over, and that, with browser incompatibilities out of the way, developers can concentrate on building compelling web applications.

Much as I hate to disagree with them, I feel honour-bound to point out that there is no “WebKit on Mobile.” There’s iPhone WebKit, Android WebKit, S60 WebKit (at least two versions each), Bolt, Iris, Ozone, and Palm Pre, and I don’t doubt that I’ve overlooked a few minor WebKits along the way.

Media queries, especially, which are the most important single technique for creating consistent interfaces across inconsistent devices, have wildly differing support levels.

Therefore, testing all your mobile web applications and sites in several WebKits will remain mandatory for the time being.

The tests

Most of the tests in the new table are taken from my existing compatibility tables. It contains golden oldies, such as the dynamic problems in the + selector and :first-child, cutting-edge JavaScript methods such as querySelectorAll(), and a few new HTML5 functionalities such as geolocation and localStorage.

My main criterion for test inclusion was that a certain method, property, or declaration must be supported by at least two WebKits, but not by all. I made an exception for geolocation, since it’s so totally crucial for the mobile platform. (Geolocation is currently supported only by iPhone 3.1)

I tested three Safari versions, three Chrome versions, one Konqueror version (with a newer one hopefully coming up), the 10 mobile WebKits I mentioned above, and at the request of Vodafone I added the JIL Emulator (it's an Android G2 except that it supports media queries).

Emulators are not allowed (except for the JIL one, which doesn’t claim to emulate any existing browser). Few mobile browser vendors will take the trouble to port their browser to Windows or Mac. Instead they’ll opt for an already-existing WebKit on those platforms; probably Safari. However, as the tests show, Safari is wildly different from all mobile WebKits. Therefore emulators are not to be trusted.

Note that I only test for compatibility. Other factors, notably user interface and performance, are left out of the equation — and that may have to be corrected in the future, seeing that S60v3 WebKit, which is by far the worst-scoring one, at least has a workable user interface, something that much-higher scoring WebKits such as Iris don’t.

I devised a scoring system. I’m not 100% certain that this is a good idea, but I desperately needed an automated method to judge WebKits. Details are on the page.

With the scoring system in place I created a script that does a bit of basic data mining in trying to establish the number of differences between WebKits as well as an exact list of those differences. The results were surprising:

All in all I hope that this research shows that there are many flavours of WebKit, and that “WebKit taking over the mobile space” does not equate “there are no more browser differences.” (Besides, let’s not forget Opera, shall we?)

Comments

1 Posted by boriscallens on 7 October 2009 | Permalink

Although I deeply appreciate all the testing you are doing regarding web on mobile devices, I feel less and less enthusiastic for my oncoming (mobile) project..

As a side note, I root for Opera :)

2 Posted by Matt Brubeck on 7 October 2009 | Permalink

Can you explain what you mean by "Android G1" and "Android G2"? In the U.S. the HTC Dream was sold as the "T-Mobile G1", but I don't think any Android phones were sold in the U.S. under the name "G2".

More importantly, what version of Android was each phone running? There were significant differences in rendering between Android 1.0 and Android 1.5 on my HTC Dream (T-Mobile G1).

3 Posted by Sarven Capadisli on 7 October 2009 | Permalink

I'm curious to know, if any, which ones support input type="file" (even though it partially depends on the device).

4 Posted by Rick Boatright on 7 October 2009 | Permalink

why is palm pre "to be tested" Can we facilitate that?

Rick

5 Posted by ppk on 7 October 2009 | Permalink

G2 runs Android 1.5 . The naming confusion in Android land is indeed ... confusing.

Palm Pre is "to be tested" because I don't have a Palm Pre to test. However, it seems this will be solved pretty soon by the newly minted developer relations managers.

6 Posted by Curt on 7 October 2009 | Permalink

Yeah, I'm a little worried by the G1/G2 moniker here. My G1 runs Android 1.6 at the moment, but it started on 1.1. Perhaps it would be better to list the OS versions like you did with the iphone E.G. "Android 1.1" "Android 1.5"

7 Posted by Steve Souders on 7 October 2009 | Permalink

This is a great anlaysis PPK. Thanks for all you do!

8 Posted by unscriptable on 7 October 2009 | Permalink

Hey PPK. Thanks for all of your hard work!

It's clear that there's a lot of work to be done by the webkit implementors going forward, but I am encouraged nonetheless.

A few of the inconsistencies you've uncovered have already been abstracted away by the frameworks, and I already know to avoid them myself (attributes, cssText, rows). Most of the others are advanced (but unfortunately very useful) CSS selectors and HTML methods. Since most of us are still forced to code to IE 6&7 compatibility (in addition to webkit, opera, and moz), these won't bother us *yet*. (Die IE, die!!!!)

A few of these, though, are very, very disappointing: overflow, position:fixed, and opacity are sure to be a thorn in my side in the near future. I'll be coming back to check out this list many times, I think.

I absolutely love that you compiled this list. If anything, I think you've helped me compile my list of "supported mobile browsers" when my clients ask for mobile browser compatibility. :-)

Regards,

unscriptable

9 Posted by Andrea Giammarchi on 7 October 2009 | Permalink

thanks for the hard work ppk, the panorama seems to be truly sad.
The only thing I am not sure about, is the missed support for opacity in Konqueror.
What I mean is that you used -webkit- for box-sizing, so I think you should consider -khtml- as well for opacity, am I wrong? Regards

10 Posted by Erik Arvidsson on 7 October 2009 | Permalink

The important thing to remember is that no two WebKit build numbers are equal. The problem is that the WebKit distributors are all on different time schedules and therefore are not using the same build numbers.

11 Posted by Astro on 8 October 2009 | Permalink

Not particularly interested in the mobile sector, I saw you tested for Konqueror WebKit. May I find WebKitGtk (used by Epiphany, Midori and a bunch of other stuff) in any column?

12 Posted by Dean Edwards on 8 October 2009 | Permalink

Excellent work collating these results. I'll have to depress myself later by looking in detail.

Webkit is a widely deployed rendering engine. You missed YAWB (Yet Another Webkit Browser) in Arora. Another browser where version numbers mean little. It has its own unique idiosyncrasies.

Web development is getting harder and harder. All of these forks of rendering engines are not helping.

13 Posted by John Daggett on 8 October 2009 | Permalink

You need to update the Acid3 test score for Chrome. Chrome does not yet support @font-face and the Acid3 test uses @font-face. The score shows up as 100/100 but look carefully, the X in the upper right corner shouldn't be there, it's failing the @font-face based test.

14 Posted by Dave Johnson on 8 October 2009 | Permalink

Let us not forget the BlackBerry browser either :)

Though they did just buy their own version of WebKit as well!

15 Posted by ppk on 8 October 2009 | Permalink

What is Epiphany?
What is Arora?
Blackberry does not use a WebKit browser yet. When it comes out I'll certainly test it.

As to Acid, I took the number and that's it. If you want to do a retest on all browsers, go ahead.

16 Posted by Jack on 8 October 2009 | Permalink

Great test, I like it. Could you confirm which S60v3 you used (as there are S60v3, S60v3sp1 and s60v3sp2). If using s60v3 (the first version) then I am not supriced that it failed all, probably rest will fail too but it would be great to get it confirmed. If you do not know then tell the device name + sw version (press *#0000#).
Thanks again.

17 Posted by Maciej Stachowiak on 8 October 2009 | Permalink

Konqueror doesn't use WebKit, it uses KHTML. Putting it on the chart is a bit misleading.

18 Posted by Dave on 8 October 2009 | Permalink

I must be doing something wrong. I just ran the acid3 test on my palm pre and got 1/100.

I'm using WebOS 1.2.1

19 Posted by Jason on 9 October 2009 | Permalink

It would be nice to include a row detailing support for @contenteditable, which doesn't work, for example, on the iPhone.

20 Posted by Kimble on 9 October 2009 | Permalink

S60v3 Webkit supports input type file. It also supports plugins which is something the iphone doesn't.

Which all goes to prove running an acid test doesn't prove anything. Although S60v3 Webkit according to the author may fail the test the worst It will give you a better web experience on major sites than the iphone will.

Go ahead and compare, take a phone like the Nokia E71 (18 months old) and the iphone (heck you can even use the latest one) and the top 20 websites and see which renders them more like the desktop. The E71 will win hands down.

This proves two things:
1) the acid3 test is just a techy test and has no impact on UX
2) http://iphonessuck.com

21 Posted by HereAndNow on 9 October 2009 | Permalink

I don't doubt that some effort was put into the testing, but it's difficult to gage the significance of the results, without having some idea about:

1. browser market share - some of the tested browsers seem unlikely to have many users.

2. version split - e.g. most people use Safari 4 now, so Safari 3 is less relevant (see Ars Technica's recent article).

3. WebKit version - variations should be expected. Features are being added every release.

4. importance of tested features - localStorage? geolocation? Perhaps browsers should get bonus points, for support.

5. expected usage of features in "normal" website designs - is the feature obscure or used regularly?

I understand that gathering this kind of information would require additional effort, but it would make the analysis a bit more complete and comprehensible.

The results might also be more digestible, if they were separated into desktop & mobile browsers.

22 Posted by Ryan on 10 October 2009 | Permalink

Thanks for all the hard work. Its disturbing to see such huge inconsistencies between browsers that are using webkit.

One incorrect statement you made about geolocation and the iPhone, is it has been supported since OS 3.0 not 3.1.

23 Posted by MySchizoBuddy on 12 October 2009 | Permalink

@ppk
Arora is new webkit based browser
( http://code.google.com/p/arora/ ). Installs on Windows

Epiphany just switched to webkitGtk. Its the default browser of gnome ( http://projects.gnome.org/epiphany/ ). linux only

24 Posted by hawkman on 12 October 2009 | Permalink

This is an important point to make, but it does feel slightly over-dramatic. Will your design look essentially the same in each platform? Yes. Will you have to do some tweaks to make sure it's compatible? Yes. Will you have to spend anything like as much time ensuring compatibility between WebKit versions as you do between WebKit/Firefox/Opera (or, heaven forbid, IE)? Good grief no.

Because browsers improve, it'll always be necessary to design for a recent *range* of versions -- nice though it would be to just pick the latest. I'm not sure that this situation is any different in that respect, and in the mobile space it's a breath of fresh air compared to desktop browsers. The table, though, will be a very useful reference for those designing cutting-edge sites.

25 Posted by Thomas Worrall on 12 October 2009 | Permalink

I believe the position: fixed; result for iPhone ("incorrect") is at least intentional. The iPhone renders the page to a virtual window that is 980 pixels wide, and scales the rendered image down to the display size. When the user scrolls/zooms, they are panning a clipping view around a larger rendered page, rather than actually scrolling as in a web browser. It's from that line of thinking that makes position:fixed; stuff appear wrong, as it's fixed to the virtual window rather than to the phone's screen.

26 Posted by Rufo Sanchez on 12 October 2009 | Permalink

The comment regarding app caching is a bit confusing. If I read it correctly, the browsers marked "cache" don't actually support HTML5 app caching, but their regular cache is so aggressive as to make offline apps semi-feasible?

27 Posted by Mike Brittain on 12 October 2009 | Permalink

I look forward to results you can provide on BlackBerry when (let's hope) they implement WebKit through their acquisition of Torch Mobile. (I saw your note in one of the comments above that you'll be testing this when it's available.)

Thank you for running these tests and sharing these great results.

Yes, there are deficiencies between each of these versions of WebKit, but something that is not given much credit is the number of baseline features that *are* consistent. If you can avoid a number of the features called out in the compatibility table, then you're still left with a fairly consistent rendering platform across many mobile phones.

28 Posted by Patrick Chanezon on 12 October 2009 | Permalink

Great analysis PPK: I look forward to see these numbers get better as mobile payers move towards more recent builds of webkit.

29 Posted by Hamranhansenhansen on 13 October 2009 | Permalink

I think it's important that Web developers don't try to use these comparisons to try and build a site that runs identically in all of these browsers, like we did during the failed HTML4 era, spackling over the differences in good and bad browsers and whining to Microsoft the whole time without any success. That is literally anti-competitive behavior on our part that benefits Microsoft or other bad browser vendors. How is the user supposed to know they have chosen a low-quality browser if we hide that from them by expending many hours and much effort to make the bad browsers render like the good ones? Users ought to be able to compare their favorite website on S60 or even Windows Mobile against an iPhone and see the iPhone doing a better job. Users need to be able to see the quality difference when browsing if they are to light a fire under their vendors to make a better browser, or if they're going to choose vendors who care about a good Web experience.

So I think it is our responsibility to let bad browsers fail -- gracefully, but publicly -- because then users can see for themselves what we see here in this chart.

30 Posted by Michael on 14 October 2009 | Permalink

also don't forget its only a minority of people who have such high-end phones.
So don't forget all those low-end phones with browsers in them (most of which don't have any javascript support and very limited and varying CSS support) and older browsers that are still in common use.
(how often does the average person buy a new mobile phone?
How many of them use something other than the browser that was supplied with the phone?)

You can't expect to get identical rendering on every device or browser but you can try to make sure it is in some way still usable. (at least on phones made in the last few years)

31 Posted by fr on 19 November 2009 | Permalink

To make things even more complicated, there are two different versions of webkit for s60v5. It looks like you tested the older version, newer devices like the N97 claim in the useragent to have AppleWebkit/525. Would be interesting if this can be tested as well at some point.

32 Posted by Wothan on 30 December 2009 | Permalink

Good chart, just like i decided earlier: build web apps for desktop browsers, build a nice version for the iPhone, and a fallback solution for all the other mobile devices, as it makes no sense to serve their lack of technology and lack of functions. in any large city there are tens of thousands or 100'000 of happy iPhone users willing to make use of any good web application. the others will be served watered down versions, without geolocation functionality, without special effects, without the ease o use. this will send more customers to the iPhone and in the end, the others might adapt to the apple-tech-level. say goodbye to the old skunky mobile phones, say hallo to standards.