Evolution of the QML engine, part 1

Published Monday April 15th, 2013 | by

QML as a technology has become increasingly important for the success of Qt. It allows for the creation of smooth, animated user interfaces, in line with expectations on the market today. There are three main features, that make it so suitable for UI creation. The first one is the declarative syntax, that makes it extremely easy to create these user interfaces and have both developers and UI designers work on the same code base. Secondly, the technology makes the integration of native code relatively easy, allowing to do much of the application logic and heavy lifting in C++. And finally, the choice of Javascript as an integral part of the QML language allows for easy prototyping and opens the technology up to a wider audience.

The engine as it is today is a fantastic basis and fills most of our users needs. However, it also has a couple of shortcomings and problems that will need to get solved in the future. This series of blogs is going to outline our findings and present the solutions that we are planning to implement for them. The goal will be to get to a better, more flexible and easier to maintain QML engine in the future.

So let’s start of with some of the problems we’re facing with the current engine.

  • Several object models
    The current QML engine uses the V8 Javascript engine to execute property bindings. Each QML item has several representations internally. One towards V8 (using the public V8 API), one towards the QML engine, and one to expose it as a QObject to native Qt. The problem here is that this requires the QML engine to keep these different representations in sync, leading to a lot of glue code and rather high memory consumption to hold the different representations.

  • Bindings done through JS closures
    Each binding is represented as a JS function closure towards V8. This requires rewriting the expression as a closure, and then letting V8 parse the rewritten expression once again. Evaluating the binding requires calling a JS function in V8 with quite a bit of overhead to instantiate the proper context for executing the binding. This is rather slow and lead to the development of a small expression interpreter (called V4) inside the QML engine that would evaluate simple expressions. This interpreter is many times faster than evaluating through V8.

  • Data type conversions
    One reason why evaluation of expressions in V8 is slow is the overhead of converting data types from Qt to V8 and vice versa. Accessing a Qt property would for example create a QVariant, that would then need to be converted to a V8::Value. If the variant was a string this involved copying the string data. To soften this, various caching mechanisms were introduced, but those came at the cost of increased memory usage and additional complexity in the code.

  • QML scoping rules
    QML is syntactically a tree of items. Inner items implicitly see properties of outer items leading to a well defined scope chain. Unfortunately this scope chain works differently than traditional Javascript and couldn’t be directly implemented with existing Javascript engines. The only way to do it directly would have been through nesting slow and deprecated with() statements. The current solution involves a very intrusive patch to V8 and still requires all QML properties to be looked up by name.

  • Throwing away QML type info
    QML as a language has type information. Javascript however is completely untyped. This lead to the fact that all type information got thrown away at QML compile time. Retaining the type information would allow us to optimise QML expressions a lot better.

  • iOS and WinRT support
    iOS does not allow making memory both executable and writable, something that existing JS engines require. WinRT doesn’t allow making memory executable at all. This makes it impossible to use V8 on these platforms without writing a full interpreter backend to V8.

  • Working with upstream V8
    As said above, we are currently forced to maintain a rather large and intrusive patch set on top of V8. There is no chance of merging these patches upstream, as V8 fully focuses on the browser use cases.

These different arguments have lead to a research project that was started Roberto Raggi and Aaron Kennedy in Nokia around a year ago, where they explored the feasibility of having a JS engine that is tailored to the use cases QML requires. The project with the codename v4vm was then transferred to Digia, where Simon, Erik and myself took it up and continued with it. It lived as a playground project on qt-project.org for the last couple of months.

Today we integrated this research project into a development branch of Qt Declarative, and we will continue to work on it there from now on.

The new engine contains a fully compliant ECMAScript 5.1 implementation, that runs on all platforms Qt supports. It contains a JIT that currently works on Linux, Mac and iOS. Integration into the current QML engine is currently done through a V8 compatible API layer which we plan to remove in the coming weeks. The engine can run all Qt Quick demos (with some minor glitches).

One of the things to note is that the engine is focused at QML use cases. This implies that we expect QML performance to become a lot better then it is today. Pure JS performance will on the other hand not be quite as good as what you get with V8 as a backend.

Still our current benchmark numbers are very promising also for pure JS code. The V8 benchmark runs currently around three times slower than the JS engine used in Qt 4.8 and Qt Quick 1, and we see a lot of potential for further optimisations.

Details about the new engine and our future plans will come in a followup post. You can find the code in the wip/v4 branch in the qtdeclarative repository. Feel free to play around with it. If you want to join development, feel free to contact us on #qt-v4vm on freenode.

Did you like this? Share it:

Posted in Qt, Qt Quick

44 comments to Evolution of the QML engine, part 1

Thanks a lot for your effort! Looking forward when the new QML engine gets integrated into a major release. Is Qt 5.2 targeted for that addition?

Martin Zajac says:

QML is a great but I think that combining it with Java Script was wrong decision. Lua language for example in my opinion much better fit to the philosophy of QML and is more correlated with C++.

Simon says:

Martin, can you elaborate on why Lua would be a better choice?

Daniel says:

I understand that java script was choosen, because it was already there (QScript) and because HTML5 has a lot of traction.

I recently looked at the QtQuick Controls and the (java script) code is essentially trivial.

This means QtQuick could use any reasonable language and could even be language agnostic.

Why I do understand that being language agnostic would be a maintance nightmare (especially for tooling) and java script was choosen, it would be really nice if there would be some python integration midterm.

This does not mean that Python should become a first party citizen like java script, but it should nicely and naturally integrate.

Qt is used a lot in conjunction with Python and Qt Quick should not become a problem in this regard.

Simon says:

I think your observation is correct, most of the binding expressions are so trivial that they hardly qualify as JavaScript. Only when for example the Math object or function closures are used, then ECMAScript becomes visible.

However the main issues I see with supporting multiple languages is that it may actually be confusing to new developers (which language am I currently using?) and it makes it certainly impossible to unify the object models in the implementation.

Alejandro Exojo says:

I also think that multiple languages would be confusing, but I’m worried that having a JS implementation that is only used in Qt, and that may have some design decisions that make it better suited for Qt Quick, is like having your own fork of the language. That’s also terribly confusing, since a developer, instead of feeling confident because it already knows the language, feels the exact opposite because there might be differences or not.

it reminds me of the old scripting features. QSA (Qt Script for Applications) never felt like it was really JavaScript, even though it probably was very close.

In an ideal world, the scripted code in Qt Quick should be very small, so it shouldn’t matter very much, but if it doesn’t matter very much, then probably is not necessary to optimize it that much either, if to do it you require to write and maintain a new VM.

With respect to Lua: I’m a complete ignorant about it, since I’ve never written code in it, but from what I’ve heard from others, seems an interesting balance between being acceptably licensed (MIT), performant, small, easily embeddable, and easy to learn. The issue of wheter the architechture of Lua is appropiate or not for Qt Quick is another matter, of course. At least I think the performance would be sufficient, since is used in lots of video games, and these tend to be resource intensive.

Martin Zajac says:

I understand the argument of popularity and it is hard to disagree with him. I decided to put this thesis after experience with Lua while developing games in Corona SDK. This language It is exactly, as Alejandro wrote later in this topic – a perfectly balanced in terms of simplicity and efficiency. I know also the story of an abandoned QtScript and I know about already existing attempts to replace him with Lua http://www.nongnu.org/libqtlua/

Slartibart says:

IMHO Python would have been best option, but as long as it’s just an _opinion_ (and I can’t/won’t back it up with some very good real _arguments_) and I’m not a core developer of Qt and therefore don’t have accurate knowledge about all the inner workings, problems and challenges facing the Qt Team it doesn’t really matter.

Dean says:

Umm, no, Python is terribly slow compared to JS

Slartibart says:

Riiiiiight….

Joakim D says:

Martin, I just want to thank you. I was set on using javascript and the V8 engine (and now possibly the v4 engine instead if that was possible) for my own scripting language in my Qt app but having looked over Lua and LuaJIT this seems so much better than a complex javascript engine of V8 caliber.

Joakim D says:

Will this mean that the v8 engine will be removed from Qt in the future? Was planning to use the included v8 engine to allow the enduser to create some own scripts to my program but if it’s going to be removed I need to rethink that plan.

Simon says:

Joakim, note that the v8 engine was never public API in the Qt sense. It has always been a module (qtjsbackend) that was an implementation detail of qtdeclarative, without any promise about binary or source compatibility.

Joakim D says:

Yes, that’s true. The other question then would be if this new engine would be published for public use? The ideal would be a fully supported javascript engine like the old deprecated QtScript but with the speed closer to V8 and working on all platforms supported by Qt. But is that even on the far of roadmap?

David says:

@Lars Since you say that V8 will reject non-browser focused patches, I understand the need look for an alternative.

The prototypes you’ve made are valuable research, but I worry about Qt taking man-hour hit to support it’s own javascript engine long term.

Have you considered working a different community that might welcome a non-browser-focused javascript engine? If the only other option is an unfriendly upstream V8, then v4vm makes sense. But are there really no other options?

In particular, I see you are using LLVM internally. Have you considered creating this as a Digia sponsored LLVM-level project? BSD style licensing might be needed, but you’d probably pickup outside help.

Simon says:

Just to clarify this one quickly: This project is not using llvm as back-end. The llvm related code you may find in there is not compiled by default and was an experiment.

David says:

@Simon Thanks for the correction on the usage of LLVM. That’s what you get for browsing github for 30 seconds. :)

I think the point still stands, though. I have a great respect for Lars and the team, but this smells a little of NIH. Just a little. (Not being sarcastic.)

Particularly as a commercial licencee, I would fail to see a cogent argument as to why I should be subsidizing a Qt-only language VM when BSD/MIT licensed options seem to exist.

As research project to explore what Qt’s scripting needs are, I think this was an absolute necessity. Should Qt really use it? I’m as of yet unconvinced.

The case that V8 is a bad choice has been made. The case for v4vm being a good choice has not (yet) been made.

charley says:

@David, you make a reasonable argument, but I *really* like the v4vm direction:

(1) V8/Javascript isn’t intended to be declarative (using “bindings”)
(2) V8/Javascript isn’t type-safe

Those are *huge*. That’s why Google is (very) concerned about Javascript and scalability going-forward. My impression is that the QML “needs” are different-enough (from a paradigm and internal-use-need) to require a different type of engine.

True, it’s new-invention. But, IMHO, QML is itself a “new-take” on the declarative-paradigm, which *every* mobile platform is pursuing right now.

Simon says:

Well said :) Stay tuned for part 2 and the pitch :)

David says:

@Simon @charley

I’m suitably intrigued and await part 2. :-)

They key assumption behind my concern is that I see a difference between something that’s fundamentally Qt/QML and something that’s a back-end implementation artifact, even if it is critically important one.

In other words: “This is Qt” vs. “Qt needs this”

Ideally, some “Qt needs this” bits have enough stand alone value to be best developed in cooperation with a wider development community.

Not having seen part 2, for now v4vm seems like a good candidate. (As does QBS, but I’m picking my battles here. :-) )

Andreas says:

Is v4vm aiming to offer compilation of qml+js to binary (or c++), and thus produce apps that will be allowed in the iOS and WinRT app stores?

Simon says:

Andreas, it doesn’t seem like it is necessary to compile qml+js to binary in order to run on iOS, at least not according to our experiments.

But yeah, perhaps we’ll do something like that in the future. At the moment this is not a priority, if you look at our findings above :)

Bruno says:

This is obviously the only way out.
No other engine would support QML use-case and run on IOS and WinRT.

I really happy with this choice, as it is a guarantee that C++ still a first-class citizen and the way to go. And Qt is not taking the JavaScript non-sense fever.

Thomas Boutroue says:

Great article ! Want to read more about V4VM and LLVM bytecode and JS / C++ integration,
can’t wait for your next article Lars !!

Keep on the good work ;-)

It would be interesing to have a security model. The possibility, prior to loading a QML file, to define what api and modules, can be used.
QML will than a very intersting tool to let user extend application without compromising the security.

Richard Moore says:

Are there any design documents or similar?

Alejandro Exojo says:

I watched the talk by Thomas McGuire at DevDays 2012 about Qt Quick under the hood and he explained many things very well. KDAB also had a series of blog posts. In the talk I think he mentioned that there was no documentation besides the source. :-(

Simon says:

Design documents for a QML JS engine that’s very young and is still undergoing massive changes? Not yet :D

www says:

Why not integrate Python? There is already PySide which is an orphaned project from Nokia.

Simon says:

I’m not sure how that would help to address our findings above?

Tim Peeters says:

Do I understand it correctly that the currently used V8-engine cannot run on iOS? I see that Qt 5.1 is supposed to work on iOS, does this mean that the new engine will be in 5.1?

Bruno says:

No, the 5.1 is a tech preview. It will not run QtQuick 2 , only widgets and QtQuick 1. (at IOS)

Dean says:

It is a pity to see the development efforts of the trolls are wasted on such pointless processes of going back and forward. This time last year the use of V8 and JS in Qt was the best thing since sliced bread, a year later working to replace that. It does look like Qt is in need of software architects with a little better vision to guide the project the right way and not waste time on such short lived features…

Bruno says:

Dean, when V8 decision was made 3 years ago, was a Nokia decision and a IOS and WinRT ports was not planned.
The V8 is a very complex beast, whois limiting the Qt potential on different platforms.

The v4vm will make Qt smaller, more portable and faster !

Dean says:

So, that makes it worse – 3 years lost on a solution that is overly complicated and in the end turned out to be unsuitable. You mean the guy who decided to use V8 was a Nokia guy and is no longer involved with Qt?

Tob says:

Lets hope so :)

Dmitrii says:

How about smooth move to typed Javascript / Typescript / Dart like language with Qt type system?

If I understand correctly, we can use the wip/v4 branch of qtdeclarative to start experimenting with QtQuick2 on iOS ?

J.R. says:

Not just a privilege of QML alone. Can be done, and in the context of Qt, should be able to do using C++ also. There is no technical argument why not to provide suitable api for certain things, only political. The reason why I often underline this point of view is that for a big group of users using QML is not possible, for various reasons.

J.R. says:

I was referring to:

“QML as a technology has become increasingly important for the success of Qt. It allows for the creation of smooth, animated user interfaces, in line with expectations on the market today. There are three main features, that make it so suitable for UI creation.”

Dan Craft says:

“iOS and WinRT support”

This is the driving force. By Mr/Mrs Market!

Dan Craft says:

The trend of using Tablet as primary computing device for work/play will be the norm in pretty near future. Evidenced by the recent IDC report on PC sales. Qt needs to be relevant and going along with this as well. No way out but going along with it.

The other reasons are secondary. and following qt blogs.

Dan Craft says:

By the way, I admire the Qt guys for such fantastic efforts for the years in the past and to come.

At the same time, from an engineering point of view on the data conversions, (QObject, V8Object, QMLObject), if there is only one QMLObject that is relevant, that would be ideal. Less code, less maintenance, less resource and thus less bugs!

Give Qt Quick2 on iOS to me. I do not care whether there is V8 or V4 inside. But the efficient and beautiful Quick2.

Thank you guys again.

Aaron Seigo says:

Will the “single object model” come with any new ways to create bindings from non-QObject C++ code into QML? That is the one thing we’ve found in moving from QML1 to QML2 that has been bumpy, and making absolutely everything a QObject (or at the very least, not being able to define metaobjects for non-QObject derived classes) has been a drag. (Not painful, just not pretty in places.)

Commenting closed.