Speeding up Visual C++ Qt Builds

Published Friday March 27th, 2009 | by

Poor, poor Visual C++ developer. You see these MinGW or Linux guys using GNU make on their fancy quad core machines to compile Qt. They are already hacking again while you still sip at your coffee, watching the distressing 25% CPU usage in the task manager. And just because nmake doesn’t make use of all available processing power.

But good news, everyone! The misery is over. I’ve writte a nmake clone that can use an arbitrary number of processes concurrently. The thing is called jom and the sources are here, a ZIP with the binary is available for download here.

To install jom, just extract the zip file to a directory which is in your PATH. Now, instead of calling nmake to build Qt, call jom. It automatically detects the number of processor cores in your PC by calling QThread::idealThreadCount().
You also can use the -j command line argument to set the number of concurrent processes.

On a quad core machine with a Qt build takes half of the time it took using nmake. There’s still room for improvement. Jom is aimed to be compatible with decent nmake Makefiles. Its not complete yet but hey, you can already speed up your Qt builds!

Update:
This project moved to gitorious! The source can now be found here:
http://qt.gitorious.org/qt-labs/jom

Did you like this? Share it:

Posted in Labs, Qt

13 comments to Speeding up Visual C++ Qt Builds

Alexander K. says:

Slightly off-topic: Am I the only one that (since 4.5.0) has to wait for link.exe (MSVC2005) while it is in the “Generating code”-step for almost 2 minutes? This happens for all Qt-programs, even programs with less than 50 lines of code, link.exe just eats 50% with no reason at all (release-mode). I hadn’t had this issue with 4.4.2.

Special says:

There are a couple other options for this as well, though none as easy. The vcbuild tool (which works on a *.vsproj, rather than a makefile) supports building multiple projects at the same time (defaults to the number of available processors), but not multiple objects within a project. That applies to Visual Studio as well. There is also the /MP compiler option (at least on 2008, not sure when it was added), which forks the compiler (again, default to the number of processors) to build multiple objects at once. Neither is really an optimal solution, and both are a bit difficult to get in the context of building Qt. This is a nice little addition ;)

In response to Alexander K., “Generating code” is the Link-Time Code Generation step; it’s an optimization tool that works over the entire executable (rather than specific object files, which is how the compiler does things). It’s controlled by the /LTCG linker option and the /GL compiler option. It is pretty normal for this step to take ridiculously long on larger executables, especially since it can’t be done in parallel at all. If you’re linking statically with Qt, it will take FAR longer than a shared link would, as all of the parts of Qt being pulled in are also subject to the link-time optimizer. LTCG can provide major speed and efficiency improvements on a release build, so leaving it enabled for final releases is probably a good idea. My suggestion is to build in Debug whenever you’re not making or testing a release.

Bill Hoffman says:

Looks like a neat idea. I have been using gmake from cygwin with cmake generated makefiles and visual studio. I gave jom a try with a CMake generated nmake file, and the first thing I ran into, is that it does not seem to handle spaces in the PATH correctly:

$ ~/Tools/jom.exe
“C:Program FilesCMake 2.6bincmake.exe” -H”C:hoffmanMy BuildsCMake” -B”C:hoffmanMy BuildsCMake-nmake” –check-build-system CMakeFilesMakefile.cmake 0
‘C:Program’ is not recognized as an internal or external command,
operable program or batch file.
command failed with exit code 1
jom 0.6.2 – empower your cores

-Bill

Alexander K. says:

@Special: Thank you very much! This was it. I use Qt in static mode and I noticed the new “/GL” and “/LTCG” in the makespecs of 4.5.0. I’ll keep those switches enabled for those improvements. Thank you again for your quick and good answer.

Ganwell says:

As Special pointed out, you can use the -MP2 (-MP[N]) flag. It works with nmake since it passes multiple files to the compiler (cl.exe) so the compiler builds in parallel not the make-programm. I added this to the win32-msvc2008 qmake.conf:

“QMAKE_CFLAGS = -nologo -Zm200 -Zc:wchar_t- -MP2″

Of course there is no parallel qmake and other task.

Philippe says:

Under XP, I already use /MP on a MacPro 8 CPU and this much improves build times of Visual C++ 2008, but not as much as GNU does it when it gets more CPUs (compared with XCode on the same machine under Leopard).
But /MP only works for compiling, not for moc :-(
I also tried Incredibuild on this machine, but it did not give better result than /MP.
Using 1 or more properly set precompiled headers considerably helps, but I guess you all know that.

bratao says:

YEAH !
Thank you for Vc++ love !

RazrWire says:

/MP worked as far back as 2005, at least, and it is a big help.

What I would like would be for the .pro file to be able to specify things to the VC project file like: suppress warning 4305, /fp:fast, etc. I am switching back and forth between the Qt SDK and Visual Studio now, I get by by not running qmake in either side unless absolutely necessary, but it would be nice to be able to check in all of my compiler environment settings to svn via the .pro file.

Jörg says:

@Bill: hey thanks for reporting the spaces-in-filenames issue! Though I’ve never claimed that jom works for other projects than Qt ATM. ;-)

re LTCG: Indeed, we turned on link time code generation in MSVC builds with Qt 4.5.0. You can get back to the old build behaviour for your projects by modifying the mkspecs. Just remove /GL from the compiler options and /LTCG from the linker options. But I’d recommend to keep /GL and /LTCG when building Qt itself.

jespa says:

@Special: The /MP option is also available in VS 2005, so jom is not a essential tool. Anyway, congratulations and thanks to Jörg for his work

stefan says:

@jespa: /MP does work in VS 2005, but it occasionally corrupts pdb files, making it unusable in production environments. That’s from experience, and also confirmed in the comments of that blog-entry: http://blogs.msdn.com/somasegar/archive/2008/11/21/c-enhancements-in-vs-2010.aspx

@Joerg: thanks for your work, bring it on!

David Johnson says:

I remember when Visual Studio builds ran rings around GNU C++. My how times have changed.

Steve says:

Hi, would someone please advise if/how it is possible to take advantage of jom when building from within VS2005 using VS Integration? I have been using the -MP compile switch to speed up compile on my dual core, but moc’ing sits at 50% cpu.

It looks like devenv.exe launchs -> vcbuildhelper.exe -> cmd.exe -> moc.exe.
nmake isn’t involved.

Commenting closed.