New browser on the block: Flow

2020 is only three weeks old, but there has been a lot of browser news that decreases rendering engine diversity. It’s time for some good news on that front: a new rendering engine, Flow. Below I conduct an interview with Piers Wombwell, Flow’s lead developer.

This year alone, on the negative side Mozilla announced it’s laying off 70 people, most of whom appear to come from the browser side of things, while it turns out that Opera’s main cash cow is now providing loans in Kenya, India, and Nigeria, and it is looking to use 'improved credit scoring' (from browsing data?) for its business practices.

On the positive side, the Chromium-based Edge is here, and it looks good. Still, rendering engine diversity took a hit, as we knew it would ever since the announcement.

So let’s up the diversity a notch by welcoming a new rendering engine to the desktop space. British company Ekioh is working on a the Flow browser, which sports a completely new multi-threaded rendering engine that does not have any relation to WebKit, Gecko, or Blink.

The last new rendering engine to come to the desktop was KHTML back in 2000 in the form of the Konqueror browser. Later Apple adapted KHTML into WebKit. And then Google forked WebKit to become Blink. And ... well, almost everyone browses with a KHTML descendant now. Let’s not forget how it all began.

It is far too early to tell if Flow will have a similar impact, but the news was reason enough for me to conduct an interview with lead developer Piers Wombwell.

PW: I’m Piers Wombwell, the co-founder of Ekioh, the company behind the Flow browser. I’m also the architect of the project and one of the software engineers on it.

In 2006 we started developing an SVG engine for user interfaces in the set-top box market. No existing browser was full-featured, or was fast enough on the low-powered set-top box chips available at the time. User interface developers wanted HTML, but couldn’t get the performance they needed, especially in animations. SVG seemed better suited to user interfaces as there was no time spent in complex box model layout.

A user interface running on our SVG engine was much faster than any of the HTML browsers at the time and was very popular in this niche market with millions of STBs running it across most continents.

Over the next six or so years, STB chips started to move to multi-core GPUs, at the same as TV resolutions were moving to 4K. HTML was becoming fast enough on set-top boxes. On the other hand, a 4K TV has four times as many pixels as an HD TV, and a multi-core GPU doesn't make each individual core any faster. Thus, a single threaded browser won’t really see any significant speed improvements. That's why we decided to make Flow multi-threaded.

Dabbling with HTML/CSS layout seemed equally fun technically as building an SVG browser, so that’s been the main focus since. It started off being an XHTML/CSS layout engine on top of SVG, but we got carried away and over time moved to full HTML.

Well, it can render and interact with Gmail quite well. It’s pretty much perfect on a few sites we’ve targeted as focuses during development, but it struggles with many others. We only started implementing HTML forms in the last few months in order to log into Gmail.

It’s not yet available for download as I think we need to address the usability of it first. It currently needs a configuration file tailored to your computer, and has no toolbar. You don't want a toolbar for TV interfaces, so we never implemented one.

For Mac, Linux, and Android. Plus, of course, for the set-top boxes that are our main market, most of which run Linux. As to Windows, none of us run Windows so its development is untested and lags behind a bit, but I’ve just compiled a version and it seems to work.

It’s not. There’s no current plan for that as we don’t have a large corporation backing our development.

We chose Spidermonkey in 2006, and as far as I recall it was because of both licensing and a documented embedding API. It was around the time that TiVo were having arguments over the GPL. The paranoia over that also ruled out use of LGPL licensed libraries for a few years.

The core browser code is abstracted away from any Spidermonkey APIs, largely so we could handle upgrades over the years - we can still handle its legacy garbage collection model quite happily.

The primary goal is stability, followed by getting more websites rendering perfectly in Flow. They generally fail because of either layout bugs or missing JavaScript APIs in Flow, so we have to solve those. Even for the embedded market, getting as many websites working as possible improves our confidence that a new HTML user interface will function correctly, first time.

Our roadmap is very flexible, usually because of commercial needs, but also we prioritise what’s interesting to a developer at that given time.

You said Flow is multi-threaded. Which tasks exactly are divided among the multiple threads?

HTML and CSS parsing is single-threaded, as is JavaScript (if you ignore WebWorkers). It’s the layout, primarily word wrap of text, that is done in parallel. Several caveats apply, but in general, two paragraphs can be laid out in parallel since they don’t impact each other apart from their vertical position.

Is the word wrap of paragraphs the computationally most expensive part of laying out an HTML page?

Yes. Each letter is a separate rectangle, plus you have word wrap rules for groups of letters. It’s also probably the hardest to achieve, so it's a good place to start. Desktop browsers haven’t touched layout, and have instead concentrated on making whole components run in separate threads.

There are multiple passes across the tree, all in parallel. We first calculate, in parallel, essentially the min-content and max-content widths of each paragraph, flexbox or table cell. Once we have those constraints, a relatively quick pass (not in parallel for that one flex box) works out the final widths of each box.

But we can handle multiple flexboxes in parallel, or one flex box and a paragraph outside the flexbox, and so on.

How integral is multi-threading to Flow and its architecture? Could you remove it? Would other browsers be able to copy Flow's multi-threading?

Multithreading can be turned off with a config setting. I suspect it’s always going to be easier to rewrite the layout code with multithreading in mind than rework existing layout algorithms - Mozilla took that approach that with Servo, rather than rework Gecko. The new layout engine could then, in theory, be combined with the rest of an existing browser.

Can you give an example of tricky problems you encountered while creating this browser?

Many sites, Gmail being a good example, were very frustrating as the JavaScript can be so large and obfuscated. It’s almost impossible to tell what they are doing, and much of the debugging was educated guesses as to what it was trying to do. Thankfully, the web platform tests help us make sure we are compatible with other browsers once we figured out the blocking bug or missing feature.

We can’t realistically pass these tests 100% as they test such a huge set of APIs - it would take us years to catch up with other browsers so we can only focus on what is used by priority websites.

The HTML parser. I first wrote an HTML parser back in 2002, and back then there was no detailed specification of how to handle badly-nested elements. We spent so much time writing test cases to figure out what desktop browsers did in each situation, and trying to behave the same. Ten years later, the detail in the WHAT-WG specification was amazing, and it was perfectly possible to write an HTML parser that is completely compatible with all other browsers.

HTML forms. A TV user interface doesn’t use most, if any, of the features of HTML forms so it was a very low priority. We started adding them because they are needed for general web browsing, but they are not complete.

We haven’t yet implemented WebGL or IndexedDB because they are not used on most of the websites we’ve tried. Obviously Google Maps uses WebGL and Google Docs uses IndexedDB but both have fallbacks. Implementing more features to allow a larger number of websites to work is a priority.

The strings vary depending on the device, but the "EkiohFlow" and "Flow" strings should always occur.

We’ve spent ages on that UA string… I could probably write a blog post about it. Essentially, I copied Chrome. Things mostly worked. Then I hit the Instagram site, which decided to use ES6 features based on the UA string. I changed it to FireFox’s, using the version of SpiderMonkey that we were using (53 in the build you have), and the site worked. Then I added more afterwards (the rc:53) to get us to the more modern Google login box.

Ekioh creates browsers for set-top boxes. What is Flow’s main purpose on set-top boxes?

It is used to render the UIs created by the box’s vendors, and not for actually surfing the web. But we don’t always get to see the UIs the vendors create, so being able to render all HTML flawlessly is the goal. That way, UI developers can do as they please.

Sort-of, but not really. I have a 2012 Sony TV with that functionality, but it was useless then and is useless now. IR has a significant lag, and that makes TV remotes far too painful to control a TV browser with. I don’t recall any modern TV/STBs that let you have open internet, but they probably exist. I can’t imagine anyone seriously using them.

Flow also runs on TVs and embedded devices. Could you give a few examples of embedded browsers? And TV browsers?

Back before we started our SVG engine, there were many HTML 4 browser engines for the TV market, such as ANT Fresco and Galio (which I also worked on), Access’s NetFront, Oregan, Espial and Opera. For the non-TV market, we have replaced Internet Explorer Mobile on a line of Windows CE devices. These days, almost all embedded browsers are based on Blink or WebKit.

The main competitors to Flow are Blink and WebKit. Most STB providers often do their own port of one of these browsers. WebKit can be optimised for these low-powered devices, but Flow is usually able to out perform other browsers, and in the areas it’s not as fast, we can usually optimise it.

In a strange way, we also compete with ourselves - we offer our own embedded WebKit-based browser that is more feature-complete than Flow. The same developers work on maintaining and improving that.

Written on 21 January 2020
Categorized in Browsers
Previous entry: CSS4 is here!
Next entry: What is CSS4?

This is the blog of Peter-Paul Koch, web developer, consultant, and trainer. You can also follow him on Twitter or Mastodon.
Atom RSS

If you like this blog, why not donate a little bit of money to help me pay my bills?

Categories: