I'm writing a CSS book

JavaScript archives Compatibility Thidrekssaga QuirksBlog Donations CSS book Politics Mobile About COH

contact

Testing mobile browser compatibility — the beginning

About a month ago the software department of Vodafone Internet Services, based in Düsseldorf, Germany, asked for my help in creating mobile widgets according to the W3C Widgets specification. In particular, they’d noticed there are differences between browsers even on mobile phones (imagine my surprise), and decided they needed advice from a specialist (that would be me).

Better still, it quickly turned out that they were willing to pay me for doing serious mobile browser compatibility tests and publishing them on this site. The payment thingy is quite unusual, I can tell you (though not entirely unique).

This is easily the best job offer I’ve gotten in my entire freelance career, so I hurried to accept it. Meanwhile I’ve done mobile tests for five days; enough to offer some guidance for setting up a doctrine for mobile browser testing.

As far as I’m concerned you can look over my shoulder while I’m working, but please PLEASE remember that everything I say may change radically without notice after I’ve tested the same browsers on other devices.

Right now I’ve only done a few tests of functionality that’s basic to the mobile experience, and even these basic tests will likely have to be expanded. Besides, right now getting a general feeling for mobile testing and its manifold problems is more important than running lots and lots of tests.

If you’re interested in helping me you can follow me on Twitter, which is shaping up to be a fantasically valuable aid.

I report oddities and bugs within minutes of finding them, and I’ve received quite a bit of useful feedback already. In fact, it’s likely that these tests won’t ever succeed without cloudsourcing some of the more complex problems.

(Please remember I do most of my tests during European office hours.)

Sheer numbers

The first and most obvious problem in mobile browser testing is the sheer numbers. The Vodafone master list of devices that must support their widgets contains 43 phone types, and those are only the modern ones that have a reasonable chance of containing a modern browser.

For the moment I’m considering every single browser on every single device unique; so if I test five Nokia phones that all have WebKit, I’m testing five different WebKit versions and the resulting compatibility tables will show five entries.

Sure, I assume that these five WebKits will resemble each other, and I expect to be able to eventually merge these five entries into one. I’m not going to hurry that process, though; one of the points of this research is to find out if there are differences between these five devices. There could be, and if there are Vodafone and I (and, in fact, the world at large) have to know about them.

Fortunately, the Vodafone team has quite a few phones, and although it’s absolutely impossible to test everything on every phone, it does allow me the chance to validate my conclusions time and again.

The browsers

How many browsers are there in the mobile space? Right now I think the correct answer is five, but I may be wrong. They are:

• Opera, in the flavours Mobile and Mini.
• WebKit, in more flavours than you can imagine.
• NetFront, which is supposed to be very popular in Asia. Right now I assume it comes in flavours, but there’s so little information about this browser that I just don’t know.
• IE Mobile. The current default is based on IE4. Version 6 will be released pretty soon, and it is based on the IE6 rendering engine. I assume it comes in flavours.
• the Blackberry browser, although I heard rumours Blackberry is going to switch to WebKit. It don’t think it comes in flavours, as there’s only one Blackberry vendor, but I assume it has versions instead.

There’s also Fennec of the Mozilla Project, but as far as I know it’s in early alpha, and I was even unable to find a formal homepage for it. It’s not yet the default browser on any mobile device; as soon as it is I’ll include it in my tests.

If you know of more mobile browsers, please leave a comment.

Opera and WebKit form the vanguard of the mobile space, IE, NetFront and Blackberry the rearguard. We’ll leave it at that for the moment; I will provide a more detailed breakdown when I have more data.

Test pages

I’m creating new test pages for my mobile tests. The main reason is readability; my current test pages are optimised for screen, and the events tests, especially, require ample amounts of space to see all event reports. Mobile devices just don’t have that space available. Besides, the event test pages refuse to work in Nokia WebKit (two versions tested).

After three or four tests I found that the version of IE Mobile I’m using doesn’t support the event object at all and requires you to use inline events. document.onclick, for instance, is not supported. (Oh, and createTextNode() seems to be missing, too.)

So theoretically, if I want to give IE Mobile a fair chance to compete, I’d have to write tests that use inline event handlers only and don’t depend on the event object.

Right now I’m not giving IE Mobile a fair chance. Obviously, I will validate my conclusions by an IE Mobile test on a completely different phone, but if that IE version has the same problems, I’m going to quietly forget about it. I’m just not going to waste my time on a browser that forces me to break one of the core rules of unobtrusive JavaScript.

By the way, don’t blame the regular IE team for this fantastically incapable browser. IE Mobile is created by a completely different team.

But exactly which IE Mobile was I testing? That turns out to be a tricky question.

Identification

Identifying exactly which browser is running on a certain phone is going to be a major problem. Mobile browsers, being chromeless, are usually impossible to distinguish by sight, so I need a browser detect script to help me out. Unfortunately, writing such a script is not as easy as it seems.

Right now browser vendors are running around like headless chicken strewing identification strings like chaff, making sure they’re totally incompatible with desktop identification strings, other mobile identification strings, and occasionally even other identification strings sent by the same browser.

Opera is rumoured to be pulling its act together, but I haven’t yet personally verified that.

In any case, distrust any market share stats of mobile browsers you see. The detection script is quite likely to be totally wrong.

IE

When I started writing this entry I was convinced that the IE Mobile I was testing was the brand-new IE Mobile 6. Why? Because it identified itself as IE6. Silly me. I’m not going to make that mistake any more.

When I saw this IE Mobile 6 promo video I started to doubt my identification. The IE Mobile I test doesn’t have all these new features, even though it runs on a touchscreen phone. Besides, IE Mobile 6 has not yet been released (I think).

So the mobile browser identifying itself as IE Mobile 6 is in fact the earlier IE Mobile that was branched off from IE4. Still, it supports basic Ajax, something that IE4 didn’t.

Now if the old IE Mobile identifies itself as IE6, how will the new IE Mobile, officially announced as version 6, identify itself? Also as IE6? I wouldn’t put it beyond Microsoft.

To make things worse the old IE Mobile also has IEMobile 7.11 in its identification string. No doubt the new IE Mobile will announce itself as IEMobile 8.01 at the very least.

WebKit

WebKit version numbers are mostly useless. The most common mobile one I encountered is 422, which is distinctly lower than Safari 3.0’s 522. The funny thing is that WebKit on Nokia supports :only-child and Safari 3.0 doesn’t.

Therefore it seems likely that Nokia has branched off its browser from WebKit 422 and then continued to work on it by itself, importing (apparently) certain newer WebKit modules but not updating to an entirely new version. They didn’t bother to update the identification string, either.

Worse, many other mobile device manufacturers are doing the same. WebKit/422 on Nokia is likely to be different from WebKit/422 on Motorola. I’m inches away from declaring the WebKit version number completely worthless.

By the way, the navigator.vendor of all mobile WebKits I’ve seen so far is Apple! This is pure misinformation (except on the iPhone), and I assume it’s a mistake of the WebKit core team who’ve accidentally left this value in place when they received updates from Apple. Please make this property empty; no information is better than false information.

Opera

Although modern Opera’s Mobile or Mini have Mobile or Mini in their identification string, the particular versions I’m using don’t. Fortunately Opera Mobile version numbering follows the Opera desktop one, so the knowledge that I’m testing version 9.5 helps me instead of hindering me.

As to the Mini, it took me and an Opera employee quite some time to identify it as version 2; distinctly antiquated. Besides, when I sent it over to Steve Souders’s UA String Parser, which uses the HTTP User-Agent string instead of JavaScript’s navigator.userAgent, this browser identified itself as NetFront! The string was totally, and I mean totally, different.

You can’t make this up.

New classes of incompatibilities

It seems that Nokia devices refuse to send key codes to the browsers. Although all key events are properly fired, keyCode and friends, which are supposed to contain the key number, stubbornly remain 0. This is not the browsers’ fault; if the device withholds this information there’s nothing they can do.

Until now I’ve seen this Nokia bug only on devices with numerical keyboards. Maybe touch devices or devices with qwerty keyboards will behave properly. One more thing to test.

This is an example of an entirely new class of problems: device compatibility errors. Since I’ve only discovered them just now I’m not yet sure how to treat them, except that we should be very careful to distinguish them from browser compatibility errors.

There’s also a new class of browser compatibility errors: when browsers are unable to make full use of the device’s capabilities. An example is an orientation change from portrait to landscape or vice versa: the device may support this fine, but if the browser doesn’t, nothing happens. On the HTC Diamond Opera supports orientation change; IE Mobile doesn’t.

The beginning

This is only the beginning of our research into the mobile sphere and our discovery of exciting new classes of incompatibilities. Again, if you’re interested in helping me out and/or have experience in mobile browser compatibility problems, follow me on Twitter. I’ll need the help.

Comments

Comments are closed.