Peek and Poke, Vol. 3

Published Thursday April 22nd, 2010 | by

With the feature work on Qt Creator 2.0 slowing down in anticipation of the beta release there is a bit of time to update the Tale of Debugging.

Executive Summary for the people not following the thread so far: Users of recent release of the GNU Project Debugger gdb can automate certain tasks using the Python scripting language. Qt Creator takes advantage of that to populate the Locals and Expressions view. While this could be done before using the gdb/MI protocol it is more robust and way more powerful nowadays. And, surprisingly, faster.

But let us start with a look at the outside world. Work is progressing nicely, albeit, unfortunately, not uniformly. The FSF gdb has seen two official releases (7.0.1 and 7.1) since December, and both releases decided to play nice with Qt Creator on Linux and Windows/MinGW. The Symbian folks managed to get hold of a pre-7.0 based port that’s capable of handling Python, so they are on the lucky side now, too, and so is, in theory, Maemo.

The other factions have been less lucky. Of course, Apple’s gdb is still fast and robust on Mac, but as it’s 6.3 based, it does not have Python, and the Python-enabled FSF-7.x-for-Darwin port happily crashes when only looking at something linked with QtCore. And then there was some vain attempt to improve the sad gdb-on-Windows/MSVC story.

 

Anyway, back to the bright side:

Qt Creator’s Python based ‘helpers’ to inspect objects of certain Qt and Standard data types now finally caught up with their older compiled cousins – and went a bit further. By now they cover:

QAtomicInt, QBasicAtomicInt, QByteArray, QChar, QAbstractItem, QAbstractItemModel, QModelIndex, QDateTime, QDir, QFile, QFileInfo, QFlags, QHash, QList, QImage, QLinkedList, QLocale, QMap, QMultiMap, QObject, QPixmap, QPoint, QPointF, QRect, QRectF, QSet, QSharedPointer, QSize, QSizeF, QStack, QString, QStringList, QTemporaryFile, QTextCodec, QUrl, QVariant, QVector, QWeakPointer, std::deque, std::list, std::map, std::set, std::string, std::vector, std::wstring, std::basic_string, gnu_cxx::hash_set, TBuf and TLitC.

Why QImage you may ask. Uuh… well. Later ;-)

For the beginning some boring stuff, like making gdb play nicely with stuff you’ll never need. Nested anonymous struct for instance. I never noticed this was a problem with the gdb/MI based versions until someone created a bug report for Qt Creator. The problem is exposed by the following code:

        union {
            struct { int i; int b; };
            struct { float f; };
            double d;
        } a = { 42, 43 };
        a.i = 1;
        a.i = 2;
        a.i = 3;

Put a breakpoint on the a.i = 1 line and press F5. When the program stops, expand the a, #1 and #2 nodes:

Anonymous Structs 1

Everything there. You can even modify the data in that structure: Change a.i in Locals view to 0. This changes f, d but, as expected, not b:

Anonymous Structs 2

 

Bored? Perhaps less boring is that you can now use scripting on the IDE side to do some fancy stuff with watched expressions, i.e. expressions that are evaluated each time the process stops, like at a break point or after a single step.

Have you ever had a large array of structs that contain lots of data but for the time being you are really only interest in a couple of members? I.e. something like the following, only a bit bigger?

        struct S { int a; double b; } s[10];
        for (int i = 0; i != 10; ++i)
            s[i].b = s[i].a = i;

Now, what if you are only interested in the a members of items 2 through 4?

Filtering to the Rescue! Create a new entry in the Watchers view (by selecting “Add New Watcher” from the Watchers view’s context menu or by “cloning” an expression from the Locals view or the main text editor) and edit it to

        ['s[2].a', 's[3].a', 's[4].a']

You can enter a Python list of strings there, and each will be evaluated as an expression by gdb in the context of the current stack frame. The result will show up as children of the list’s own node in the Watchers view, and as we specified the a member in all cases, we get a flat list of three ints.
Exactly what we were interested in.

But that’s not all. The more daring can even use Python to create the list of expressions by using e.g.

        ['s[%d].a' % i for i in range(2, 5)]

Filtered Display

and one can even obtain the range indices from other variable values or arithmetic operations thereof etc.

 

Still bored by looking at black-on-white numbers and strings? What about watching an image being painted? With real colors ;-) )?
Then try the following:

        QImage im(QSize(200, 200), QImage::Format_RGB32);
        im.fill(QColor(200, 10, 30).rgba());
        QPainter p;
        p.begin(&im);
        p.setPen(QPen(Qt::black, 5.0, Qt::SolidLine, Qt::RoundCap));
        p.drawEllipse(20, 20, 160, 160);
        p.drawArc(70, 115, 60, 30, 200 * 16, 140 * 16);
        p.setBrush(Qt::black);
        p.drawEllipse(65, 70, 15, 15);
        p.drawEllipse(120, 70, 15, 15);
        p.end();

Put a breakpoint on the first ‘drawEllipse’ line. Run and wait until the breakpoint is hit. Now go to the context menu of the im item in the Locals view and toggle between “Normal” and “Displayed” in the “Display of Type QImage” submenu.

Keep it at “Displayed”, press F10 a few more times, and watch the contents of the extra window:

QImage 2  QImage 3  QImage 4  QImage 5

 

So many possibilities ;-)

André

PS: A big “Thank you!” to everybody who was involved on making this possible by bringing Python scripting to gdb, especially to Thiago Jung Bauermann, Phil Muldoon and Tom Tromey.

Did you like this? Share it:

Posted in Qt, QtCreator

20 comments to Peek and Poke, Vol. 3

Sam says:

Great to see GDB getting into 1980 standard’s :)

rule says:

OOOO, drawing debugging is extremaly amazing :-)
And what about GDB 7xx + Python on Mac OS ? Is there any Ideas ?

RCL says:

I don’t really understand what bug you’re talking about in example with unions. You start with stating that the code exposes a bug, then you show two screenshots where everything seems to work. Or did you mean that the bug was in previous versions of gdb?

By the way, can we have the same functionality like in MSVC debugger, where you can append “,10″ to any pointer in Watches window to make it treated as an array of 10 elements instead? E.g. if you have int * pPtr and you type pPtr in Watch, you see only its value (an adress). But if you type pPtr,15 you see an array of 15 ints at the address that pPtr points to. Really handy.

Also, is casting variables to other types (specifically pointer types) supported in Watcher?

André says:

@Sam: Not bad, eh? ;-)

@RCL: I wrote “… a problem with the gdb/MI…”. Stress on the ‘MI’ part.

When trying to access the same structure with the MI protocol you’ll get a communication like the following:

35-var-list-children --all-values "local.a"
35^done,numchild="1",children=[child={name="local.a.public",exp="public",numchild="3",value=""}]
36-var-list-children --all-values "local.a.public"
36^error,msg="Duplicate variable object name"

Good luck with populating a treeview with output like that ;-)

One of the nicest aspects of the Python scripting is that you can create the data you need exactly the way you want it to have, not the way the MI protocol thinks it might be best for everyone. That saves a lot of bandwidth, eats less resources on the gdb side and even enables you to work around minor issues like the one mentioned above without having to wait for a fixed version of gdb popping up in your favourite distribution.

The “,10″ feature is there, but you just have to use gdb syntax, i.e. append “@10″.

Also, the Watchers expression take any expression that gdb understands on the command line. This is sometimes a bit different from C++. You’d need e.g. to use ‘foo::bar’* _including_ the quotes if you have a class bar in namespace foo. Apart from that there’s no magic there. So casting of pointers works.

André says:

@rule: The FSF gdb/Darwin crash is tracked as http://sourceware.org/bugzilla/show_bug.cgi?id=11488 and today there appeared a new comment: “It looks like this happens for binaries that are linked against a Framework.” So the problem is acknowledged and I don’t think it’s unfixable. Simple applications do work and Python works, so there’s hope.

RCL says:

@Andre Thanks for reply. Wasn’t aware of @10 syntax and ability to cast pointers :) By the way, I guess you would have made QtCreator 10x more popular with programmers having MSVC background by providing syntax sugar (e.g. replacing @10 with ,10 internally, the same for expressions) for these features. Right now MSVC syntax is de facto standard and I think it’s a bit early to set your own standards :) No offence meant!

André says:

It’s certainly not me wanting to set a standard here, rather using what gdb provides. And that is ‘@’. The nice thing about ‘@’ is that it cannot be legally part of a C++ expression. On the other hand, a comma can, so using it introduces a unnecessary ambiguity with the comma operator:

p f,2
 $4 = 2
p f@2
 $5 = {0x99e6d14, 0x808c4e0}

The first command evaluates (correctly!) an expression containing the comma operator, whereas the second uses the gdb’s ‘@’ “language extension” to create an array. So there’s no room to silently replace ,10 by @10 without losing functionality.

RCL says:

@Andre

Well, obviously it is less correct and probably some valid (but perhaps not practically usable) cases will be omitted. But my point is, if one wants to make MSVC-based developers (which are a majority right now) to consider/try/switch to QtCreator, it should hide its gdb-specific things and implement MSVC’s behavior (at least optionally). Sure, the @10 vs ,10 is may not look important, but it’s often these small things that cause frustration.

I’m not arguing against QtCreator – I like it and I like the very idea of having cross-platform IDE which is competitive to Microsoft tools (which are brilliant, but alas, absent on Mac/Linux). I just want it to behave closer to MSVC, which would make converting easier for us Windows coders. Don’t you want to attract more people at the expense of not being 100% correct? :)

razvanpetru says:

Interesting update. What’s happening to the CDB integration though? I’ve noticed that watches were disabled on gitorious a few days ago and CDB seems to lag behind when compared to MinGW’s GDB. e.g: 5s pause with 100% cpu on one of the cores when first breakpoint is hit (dumpers getting loaded?).
Having a complete package with MinGW is very nice, but on the other hand the Windows SDK is also free and so are the Windows debugging tools, with the important difference that they have better performance across the line.

My wish would be for Qt Creator to become an even more compelling alternative to Visual C++.

André says:

@razvanpetru: I’d certainly like to seem improvements on the cdb side, too, by right now there is, as you noticed, not much happening. There seem to be some hard limitations with the current approach. Anything involving function calls into the debugged application takes a lot of time, even for trivial functions, and I have no idea why. Also, unless one wants to manually set up stack frames, there are serious limitations on what kind of parameters can be passed into a function, and what kind of values can be returned, so the “binary” helpers concept does not scale at all on cdb.

Unfortunately, I don’t see many other options either. There seems to be no generic scripting available to cdb similar to what gdb has with python, and gdb itself cannot read .pdb debug information, so we can’t use gdb to handle binaries built by MSVC.

I readily admit that there’s a possibility that these problems can be easily solved by some “real Windows person”, but that’s certainly not me. Due to the closed nature of the software it’s also really hard to figure out where the actual bottleneck is (btw, in stark contrast to gdb where it’s comparatively easy to find and even solve such issues, see e.g. http://sourceware.org/bugzilla/show_bug.cgi?id=11198).

One possible way out might be kind of adapter, talking to cdb one one side, and the gdb serial protocol on the other side similar to what we did for Symbian where we interface AppTRK as the “native” debugging agent on the device and can use all the power of gdb/Python on the Creator side. But that would require gdb to be able to read .pdb, too, which is probably more work than the adapter itself. A second possibility would be making the cdb integration a python extension and mimic the interface the gdb python integration exposes. That would allow us to re-use all the python bits we’ve done for gdb, and give at least “feature parity” and probably improve performance as the python helpers use way less calls into the debugged process than the compiled helpers.

But even investigating the feasibility of these approaches alone takes a lot of time. And while Windows/MSVC certainly is one of the most popular target platforms for Qt development, there is some limit on the resources that can be spent for an inherently single-platform solution, even as part of the overall cross-platform experience.

I certainly would appreciate any effort someone interested in that area would put into such feasibility investigation, or into figuring out why our way of calling functions using cdb is so slow.

Alex K says:

Very cool! It would be very useful if I can write helpers to display an OpenCV matrix as an image, or as a table view listing its elements.

stephenju says:

I am using 1.8.3 snapshots and I can not find this “extra window” in the QImage example above. Is there a menu command to show it?

stephenju says:

Sorry. It’s 1.3.83 snapshots with Qt 4.6.2 binary release.

André says:

Windows? There you need git from “master” branch, as there have been two fixes after 2.0.0-beta was branched off. On Linux it should work with the snapshots.

Will Stokes says:

Debugging painting to QImage’s looks awesome. I can’t wait to upgrade to QtCreator 2.0 for this feature alone. It looks like this will work with QPixmap as well which is fantastic. :-)

stephenju says:

So it’s not in any of the 1.3.83 snapshots? I see the Display/Normal contextual menu items in the latest Windows snapshot but it does nothing. On the Mac snapshot from the same date, it doesn’t even have the menu items. Just the plain Bold Pointer… etc options.

Can’t build Creator from git since I can’t even properly build Qt from git. It’s beyond my pay grade. :)

André says:

@stephenju: well, the it is “in the snapshot” in the sense that the code has been there for a while. However, there have been two bugs that prevented it from working on Windows at the time the 2.0.0-beta branch was created, and the snapshots are currently built from that branch. It’s fixed in the master branch, though, so if you build from master, it works under Windows, too. That it does not work on Mac is due to Apple’s gdb version that does not have Python support at all. So unless the “real” gdb (i.e. FSF gdb 7.x) gains the ability to work with Qt on Mac, this feature won’t be available on Mac.

stephenju says:

@André: Thanks for the explanation. Too bad it doesn’t work on Mac which is my primary platform. But I am getting used to working on Windows thanks to Creator.

Which branch will the final 2.0 be created? Master or 2.0.0-beta?

André says:

Master.

ansmehta says:

Hi Andre

I installed Qt SDK on Linux_x86_64. I’m using Qt Creator 2.0.0 present in it. I built a python enabled gdb 7.1.To confirm whether my gdb is python enabled I did :

~/gdb7.1build/gdb/gdb –interpreter=mi
~”GNU gdb (GDB) 7.1n”
~”Copyright (C) 2010 Free Software Foundation, Inc.n”
~”License GPLv3+: GNU GPL version 3 or later n”
~”This is free software: you are free to change and redistribute it.n”
~”There is NO WARRANTY, to the extent permitted by law. Type “show copying”n”
~”and “show warranty” for details.n”
~”This GDB was configured as “x86_64-unknown-linux-gnu”.n”
~”For bug reporting instructions, please see:n”
~”.n”
(gdb)
-list-features
^done,features=["frozen-varobjs","pending-breakpoints","thread-info","python"]
(gdb)

Clearly I can see it is python enabled. I have python 2.4.3. My problem is that I can’t see the values of QString etc. I have unchecked “Use debugging helpers” in Tools->options->debugger since I wan’t to use python based debugging helpers and not the precompiled ones in qtc-debugging-helper. Do i need to add anything to my .gdbinit ? Currently i’m only looking to use python based debugging helpers of QString etc and don’t wish to write my own debugging helpers of custom classes(will try this later)

NOTE: I get a warning in application output window &”warning: GDB: Failed to set controlling terminal: Invalid argumentn ”
Do I need to rebuild Qt creator from sources after installing python enabled gdb or can I use directly the once present in SDK ??

Commenting closed.