Qt Weekly #18: Static linking with Qt

Published Wednesday August 27th, 2014 | by

Qt has supported static builds for a long time, but iOS made this way of shipping Qt more widespread. To make the discussion about static linking more practical, we will not discuss things abstractly, but we will look at how to compile the Weather Info example application from the Qt Positioning module statically for iOS, and address questions as they come up.

 

Weather Info example app

Weather Info example app

First of all, to build a static application, we need a statically compiled Qt. This is not difficult — adding the “-static” option to configure mostly does it. However, ensuring that the dependent libraries are also available and linked statically can take some effort. In the case of iOS, Qt is statically built by default, so we can simply use the default Qt packages.

To build the Weather Info application statically, let’s just try building as usual, running qmake and make (or even better, let Qt Creator call them for us).

cd weatherinfo
qmake -r weatherinfo.pro CONFIG+=debug
make

…and… it runs!

So, it seems that we don’t have to do anything special once we have a static Qt.
Great, so is this blog post already finished?

Well, not quite. If we take a look at the linking step, we see a huge command line (EDIR=/Users/fawzi/Qt5.3.1/Examples/Qt-5.3/positioning/weatherinfo):

/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++ -arch armv7 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS7.1.sdk -L$EDIR/Debug-iphoneos -L/Users/fawzi/Qt5.3.1/5.3/ios/plugins/platforms -L/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS7.1.sdk/System/Library/Frameworks -L/Users/fawzi/Qt5.3.1/5.3/ios/lib -L/Users/fawzi/Qt5.3.1/5.3/ios/qml/QtQuick.2 -L/Users/fawzi/Qt5.3.1/5.3/ios/plugins/accessible -L/Users/fawzi/Qt5.3.1/5.3/ios/plugins/qmltooling -L/Users/fawzi/Qt5.3.1/5.3/ios/plugins/position -L/Users/fawzi/Qt5.3.1/5.3/ios/plugins/bearer -L/Users/fawzi/Qt5.3.1/5.3/ios/plugins/imageformats -F$EDIR/Debug-iphoneos -filelist $EDIR/weatherinfo.build/Debug-iphoneos/weatherinfo.build/Objects-normal/armv7/weatherinfo.LinkFileList -dead_strip -headerpad_max_install_names -stdlib=libc++ -u _qt_registerPlatformPlugin -L/Users/fawzi/Qt5.3.1/5.3/ios/plugins/platforms -framework UIKit -L/Users/fawzi/Qt5.3.1/5.3/ios/lib -framework OpenGLES -L/Users/fawzi/Qt5.3.1/5.3/ios/qml/QtQuick.2 -L/Users/fawzi/Qt5.3.1/5.3/ios/plugins/accessible -L/Users/fawzi/Qt5.3.1/5.3/ios/plugins/qmltooling -L/Users/fawzi/Qt5.3.1/5.3/ios/plugins/position -L/Users/fawzi/Qt5.3.1/5.3/ios/plugins/bearer -L/Users/fawzi/Qt5.3.1/5.3/ios/plugins/imageformats -lz -lm -miphoneos-version-min=5.0 -lqios_debug -framework Foundation -framework QuartzCore -framework CoreFoundation -framework CoreText -framework CoreGraphics -lQt5PlatformSupport_debug -framework Security -framework SystemConfiguration -lqtquick2plugin_debug -lqtaccessiblequick_debug -lqmldbg_qtquick2_debug -lQt5Quick_debug -lqmldbg_tcp_debug -lQt5Qml_debug -lqtposition_cl_debug -framework CoreLocation -lqtposition_positionpoll_debug -lQt5Positioning_debug -lqgenericbearer_debug -lQt5Network_debug -lqdds_debug -lqicns_debug -lqico_debug -lqjp2_debug -lqmng_debug -lqtga_debug -lqtiff_debug -lqwbmp_debug -lqwebp_debug -lQt5Gui_debug -lqtharfbuzzng_debug -lQt5Core_debug -Xlinker -dependency_info -Xlinker $EDIR/weatherinfo.build/Debug-iphoneos/weatherinfo.build/Objects-normal/armv7/weatherinfo_dependency_info.dat -o $EDIR/Debug-iphoneos/weatherinfo.app/weatherinfo

It looks mostly OK, but contains a lot of libraries. System and Qt libraries are expected, but we also have:

  • the platform plugin:-lqios_debug that loads plugins/platforms/libqios_debug.a
  • the QtQuick2 plugin:-lqtquick2plugin_debug that loads qml/QtQuick.2/libqtquick2plugin_debug.a
  • the accessibility plugin for qtquick:-lqtaccessiblequick_debug that loads accessible/libqtaccessiblequick_debug.a
  • the plugin to improve debugging of qtquick2:-lqmldbg_qtquick2_debug that loads plugins/qmltooling/libqmldbg_qtquick2_debug.a
  • the plugin to talk to the QML debugger:-lqmldbg_tcp_debug that loads plugins/qmltooling/libqmldbg_tcp_debug.a
  • the plugin giving the position using the core location framework: -lqtposition_cl_debug that loads plugins/position/libqtposition_cl_debug.a
  • the generic polling plugin for position:-lqtposition_positionpoll_debug that loads plugins/position/libqtposition_positionpoll_debug.a
  • the plugin giving the generic bearer implementation: -lqgenericbearer_debug that loads plugins/bearer/libqgenericbearer_debug.a
  • support for the DDS image format:-lqdds_debug that loads plugins/imageformats/libqdds_debug.a
  • support for the ICNS image format:-lqicns_debug that loads plugins/imageformats/libicns_debug.a
  • support for the ICO image format:-lqico_debug that loads plugins/imageformats/libqico_debug.a
  • support for the JPEG 2 image format:-lqjp2_debug that loads plugins/imageformats/libqjp2_debug.a
  • support for the MNG image format:-lqmng_debug that loads plugins/imageformats/libqmng_debug.a
  • support for the TGA image format:-lqtga_debug that loads plugins/imageformats/libqtga_debug.a
  • support for the TIFF image format:-lqtiff_debug that loads plugins/imageformats/libqtiff_debug.a
  • support for the Wireless Application Protocol Bitmap image format:-lqwbmp_debug that loads plugins/imageformats/libqwbmp_debug.a
  • support for the WebP image format:-lqwebp_debug that loads plugins/imageformats/libqwebp_debug.a

The platform plugin and a couple of others should come as no surprise, but Wireless Application Protocol Bitmap? A format by WAP for black and white images? When WAP has basically disappeared? Does our application really need it?

While, the non-debug version libqwbmp.a is only around 15 KB in size, things like libqjp2.a can be around 600 KB, so this is definitely worth some investigation.

One could hope that unneeded code gets stripped away. After all, that is one of the advantages of static linking which leads to a smaller executable than shipping all dynamically loaded libraries. If a function in a library is not used, that function is not part of the static executable.

Unfortunately, plugins make the situation more complex.

Qt Plugins

Qt internally uses dynamic linking in several places, mainly with plugins. Plugins are used for platform dependent support, image formats, audio and video codecs, sensors, and so on.

If you take a look in the plugin directory (qmake -query QT_INSTALL_PLUGINS) you will see the various plugins that Qt might use.

For example, linking QtPositioning, and QtMultimedia is not enough to have a position fix and being able to display a movie, respectively. The required plugins also need to be loaded.

Calling the Q_IMPORT_PLUGIN marcro from user code will load that plugin (which will thus need to be linked).

When using dynamic linking, the plugins are loaded automatically at runtime when needed.
With static linking, one has to decide which plugins are required already at build time.

Starting with Qt 5.3, thanks to Ossi, a meaningful set of plugins (depending on the Qt modules you added) is automatically loaded.
That is the reason why without any extra effort things just worked in the above example, but that is also the reason we have so many plugins linked to our executable.

qmake does not just add the -l<plugin> to the link flags (which has no effect if not used), but really uses the plugin.
Indeed, if we look at the linked files from $EDIR/weatherinfo.build/Debug-iphoneos/weatherinfo.build/Objects-normal/armv7/weatherinfo.LinkFileList:

$EDIR/weatherinfo.build/Debug-iphoneos/weatherinfo.build/Objects-normal/armv7/main.o
$EDIR/weatherinfo.build/Debug-iphoneos/weatherinfo.build/Objects-normal/armv7/appmodel.o
$EDIR/weatherinfo.build/Debug-iphoneos/weatherinfo.build/Objects-normal/armv7/weatherinfo_qml_plugin_import.o
$EDIR/weatherinfo.build/Debug-iphoneos/weatherinfo.build/Objects-normal/armv7/weatherinfo_plugin_import.o
$EDIR/weatherinfo.build/Debug-iphoneos/weatherinfo.build/Objects-normal/armv7/qrc_weatherinfo.o
$EDIR/weatherinfo.build/Debug-iphoneos/weatherinfo.build/Objects-normal/armv7/moc_appmodel.o

we see weatherinfo_plugin_import.o which comes from the compilation of the autogenerated
$EDIR/weatherinfo_plugin_import.cpp which contains the Q_IMPORT_PLUGIN that takes care of loading the Qt plugins.

This is nice, but we want a lean application, without unnecessary cruft, so how can we get rid of the plugins we do not need?

As described in How to Create Qt Plugins, it is possible to switch off the automatic loading of plugins with CONFIG -= import_plugins, but this disables the generation of the weatherinfo_plugin_import.cpp file, and then we would have to do everything manually: add Q_IMPORT_PLUGIN somewhere in our code, define the QT_STATICPLUGIN preprocessor variable and link the plugin (as one has to do without qmake).

We just want to specify which plugin to load, but still leave the ugly details up to qmake.
How to Create Qt Plugins also states that one can set the list of plugins to load for each class with QTPLUGIN.<className>. The position plugins we will actually need, but the image conversions? We should be able to get rid of all of them.

So let’s add

QTPLUGIN.imageformats=-

to our project file, weatherapp.pro, and indeed no more image formats plugins are linked, the release executable becomes around 1MB smaller (13’492’384 -> 12’539’088 bytes), avoids unneeded plugins initializations, and… still works.

If some imageformat plugins are needed, it is possible to add them to QTPLUGIN.imageformats, or directly to QTPLUGIN.

It is worth noting that for this to work, the plugin creator has to define both PLUGIN_TYPE and PLUGIN_CLASS_NAME to make the automatic loading work. There were some issues with that, but hopefully all have been fixed.

QML Plugins

Our example basically uses just QtQuick 2, but in general one has to scan recursively the imports of all QML files used to find all plugins to load, and to load them. Morten wrote a tool that scans the QML files (qmlimportscanner) and qmake uses it to generate $EDIR/weatherinfo_qml_plugin_import.cpp which gets compiled to the weatherinfo_qml_plugin_import.o file we have previously seen in the linked files.

For this to work, the QML files have to be found by the tool (which might for example require installing Qt in some cases) and the qmldir files have to specify the classname of the plugin.

Conclusions

Static linking avoids the dependency on external libraries and leads to smaller packages for a single self-contained application (all the unused code of a library is stripped).
But for the linker, “used code” means code that might be reached, and to make plugins work, one has to pull in the plugin using Q_IMPORT_PLUGIN and link it.

qmake simplifies this operation by generating special files and linking the modules, but its default choices about which modules your application wants to use are not always correct. Furthermore, reachable code does not necessarily mean code that will be executed, or that does something useful for your application. For example, the initialization of a plugin might be a bit expensive, so doing it unconditionally at startup is not always a good idea.

Thus removing plugins that are not needed is a good way to make your application leaner.
There are also other places where Qt might waste CPU time (to initialize things) or space (both in application size and memory), as for example the MIME DB or Unicode information, but that is another story…

It is important to note that static linking has consequences with respect to the licenses of Qt you might want to use. While you are probably safe if your application is distributed according to the terms of a recognized free and open-source license, you should definitely check the licensing issue for proprietary or closed-source applications. Qt Enterprise license is the recommended one for closed-source applications.

I hope this has made the issue and advantages of static linking with Qt clearer. Happy coding and let me know if you have comments or tips.

Did you like this? Share it:

Posted in iOS

18 comments to Qt Weekly #18: Static linking with Qt

Violet Giarffe says:

Doesn’t the LGPL license forbid statically linking LGPL-licensed binaries with proprietary binaries?

famohame says:

Yes, that is what I wanted to highlight with the legal consequences, if your application is GPL, no problem, but if it is proprietary… (as always I am not a lawyer…)

gigaplex says:

Static linking LGPL libraries with GPL binaries is fine, that’s not a problem. I don’t think it’s possible to distribute an application on the iOS app store and remain GPL compliant. As this blog post focuses on iOS applications, this isn’t something you can just shrug off so easily.

Laszlo Papp says:

Agree. I have always wished to see Trolltech, Nokia, and Digia to put more exceptions into their license files. Static linking is something like that, which could be put into the exception section along with the main LGPL license. It may be partially done already, but this could be more widespread.

Fawzi Mohamed says:

You are right, I choose a poor example. Always with the disclaimer IANAL, this is not legal advice.
My understanding for iOS applications is the following: an open source licensed application can be linked statically linked to LGPL V 2.1 without problems, bug a GPL one has issues with the Apple Store restrictions. MIT,… should be fine.
The new LGPL V3.0 is incompatible with the Apple Store restrictions. There is work to see what can be done about that, so that publishing open source applications is still possible (again as far as I know, I am not speaking as Digia). Sorry for all the disclaimers, maybe someone with more knowledge on the legal side can chime in…

It doesn’t forbid it but it’s widely considered a legal gray area. Generally, people agree that providing your object files to your customers to re-link with a different version of Qt is sufficient to use static linking with a closed source app and LGPL Qt.

Disclaimer: IANAL; this is not legal advice.

Jason Crawford says:

Would it be possible to include pre-compiled libraries suitable for static linking in the official Qt distributions, for those who aren’t building Qt from source?

Jason Crawford says:

Would it be possible to distribute optional pre-compiled Qt libraries suitable for static linking, for those who aren’t building Qt from source?

Fawzi Mohamed says:

While supporting more configurations is always better for the user, it has a cost in testing, creating, maintaining.
To have a high quality release of a product that lives and evolves requires making choices, and the dynamic version is clearly preferred as it is more flexible and used.
Compiling from source is not so difficult, and you might want to do it with your specific toolchain, setup, …

dslsynth says:

Which is why a personal commercial Qt license for something like a hundred dollars a year would be such a nice idea.

Suggested license contents: same source code as open source edition, no support, just a different license permitting (1) closed source, (2) statically linked applications and (3) selling applications in smaller scale personal/startup company contexts.

The current Qt Enterprise licenses are only for rich companies which is not exactly the typical type of user making mobile applications.

Nor are the hobbyist developers that do not want to distribute their source code for their Qt application or provide statically linked binaries for iOS.

Fawzi Mohamed says:

I woun’t say that the current offering is perfect (and I don’t have much to do with that, but currently you can buy license for 3 months (during which you ship the application) and then you continue if you see that it is worth the investment. There is a promotion that puts it in couple of 100s range.
It won’t satisfy everybody, but you never can if you also want to make a living out of it…

dslsynth says:

Thanks a lot for your reply, Fawzi!

The trouble is that one may not be able to make a sustained living of ones project. Or one may simply want to be safe for the unwanted effects of GPL in ones hobbyist projects.

See, there is an unaddressed hobbyist and very small company market that Digia really should go and make good offerings for. Not every possible author of closed source Qt applications is a rich company.

For some such possible low budget customers the price of a commercial Qt license is more than their monthly food budget. Sometimes things are that small and scalable solutions are very much needed. Of cause it requires Digia to want to support such types of customers. Really hope so!

The mobile application market is not exactly a gold mine. Nor are customers with very small budgets. That is the core challenge.

Fawzi Mohamed says:

Yes well the commercial license does offer some advantages over the opensource one (and some extra stuff),… luckily otherwise we would be in big trouble :)
offering it to more people is better, but that is always so, and one has to be careful not to cannibalize its own income.

So what is available at http://qt.digia.com/buy/ is the current status, so you see a way to improve it, aside slashing the prices for everybody? Would you do something like a “lower income” “indie” offer? Might be doable, but for sure not easy to define…

dslsynth says:

Thank you so much for listening, Fawzi! :-)

Imagine a developer with a $300 to $500 monthly budget for food and everything else in a Scandinavian country. For such a developer the current commercial license price of $399 a month is simply way too expensive. Even $20 a month can be too expensive.

A low budget developer may not need the support, the cloud features nor the enterprise add-ons. What is important is the ability send out binaries without having to worry about legal issues with GPL while still having money for food and be able to sell such binaries eventually. If the app generates a real income the user can (and may have to) upgrade to a more expensive license.

Defining the terms will surely be interesting! Think in terms of modular offers (license, support, cloud, add-ons) and a scalable set of customer types: indie/hobby, small startup, normal enterprise license. Its important that customers can start out small (aka risk sharing).

Fawzi Mohamed says:

Qt is so good in big part thanks to the community, and constructive dialog is always welcome, so thank you for sharing your ideas…

dslsynth says:

Just wanted to add that there can be quite some potential in the donate and/or crowd sourcing perspective for Qt licenses. Even though each indie/personal/lowcost licenses will not provide much income to Qt there can be very many of them. Especially if they are cheap!

The idea is that you guys give people something on the license front – as explained in my other posts here – and they give you some money to improve Qt with.

Market these licenses as “help us develop Qt” and you could very well see many people join in with a little monthly donation. Something that could provide an even more solid foundation for the Qt project!

Look at an organization such as Copenhagen Suborbitals which essentially is a crowd sourced manned space program. While you guys sell a product and give some of it away for free its still a project making something for the pleasure of many people. So maybe the “contribute a little” perspective may for Qt as well? Give it a try!

Android says:

Can you tell me how to lean the Qt app for Android? That’s huge and hard to deploy.

Fawzi Mohamed says:

Android deployment links dynamically by default. Static linking should make a single application smaller also in this case, but I never did it. Even with dynamic linking it has similar issues with respect to the plugins (which should be deployed and which shouldn’t).
I think that the next Qt release should use the same variables as qmake, so that setting QTPLUGIN.pluginType=… should help also there. In the meantime you can look at http://qt-project.org/doc/qt-5/deployment-android.html to see how to control that.

Commenting closed.