Count with me: how many smart pointer classes does Qt have?

Published Tuesday August 25th, 2009 | by

On Friday, along with the Qt for Symbian integration, we got a new smart pointer class in Qt, called QScopedPointer. Harald, one of the class’s author, blogged about it, which prompted a lot of comments asking why we have those classes and what’s the difference between the ones we have.

Before we can find out why we have those classes, we need to know the classes we have. So, count with me, in chronological order:

  1. QPointer (4.0)
  2. QSharedDataPointer (4.0)
  3. QExplicitlySharedDataPointer (4.3/4.4)
  4. QtPatternist::AutoPtr (internal class, 4.4)
  5. QSharedPointer (4.5)
  6. QWeakPointer (4.5)
  7. QGuard (internal class, 4.6)
  8. QScopedPointer (4.6)

Note: QExplicitlySharedDataPointer was introduced in 4.3, but the API was made public and documented in 4.4

That many, huh?

Each and every case has its use and they all (except one) are still valid today.

Shared pointer versus shared data

First, let’s get one thing straight: there’s a difference between sharing pointers and sharing data. When you share pointers, the value of the pointer and its lifetime is protected by the smart pointer class. In other words, the pointer is the invariant. However, the object that the pointer is pointing to is completely outside its control. We don’t know if the object is copiable or not, if it’s assignable or not.

Now, sharing of data involves the smart pointer class knowing something about the data being shared. In fact, the whole point is that the data is being shared and we don’t care how. The fact that pointers are being used to share the data is irrelevant at this point. For example, you don’t really care how Qt tool classes are implicitly shared, do you? What matters to you is that they are shared (thus reducing memory consumption) and that they work as if they weren’t.

Strong versus weak pointer referencing

The difference between a strong and a weak reference is whether the existence of the smart pointer class on a given pointer guarantees that the object will not get deleted. In other words, if you have this smart pointer, are you sure that this will always remain valid (provided, of course, everyone is playing by the same rules)?

Some of the pointer classes above don’t guarantee that. If they don’t guarantee that the object remains valid, their main purpose in life is to tell you whether the object has been deleted already or not. Some classes may provide an additional feature that allows you to promote a weak pointer to a strong one, thus guaranteeing that it won’t get deleted anymore.

The Qt smart pointer classes

1. QPointer

QPointer is a weak pointer class and it shares the pointer value, not the data. It only operates on QObject and QObject-derived classes. This class was added in Qt 4.0 and is the direct upgrade of Qt 3′s QGuardedPtr (and Qt 2′s QGuardedPtr). Like its predecessors, QPointer suffers from broken constness support and shows its age.

Its sole purpose in life is to tell you whether the QObject has been deleted already or not. But, unlike Qt 2 and Qt 3, the QObject of Qt 4 can live in several threads. That means QPointer has one serious flaw: it lets you know whether the object has been deleted, but it makes no guarantee about the next line! For example, the following code could be in trouble:

    QPointer<QObject> o = getObject();

    // [...]
    if (!o.isNull())
        o->setProperty("objectName", "Object");

Even if isNull() returns false, there’s no guarantee that the object won’t get deleted by the next line.

Therefore, QPointer can only be used to access the object if you can guarantee, by external means, that the object won’t get deleted. For example, QWidget and its descendents can only be created, manipulated and deleted in the GUI thread. If your code is running on the GUI thread or has that thread blocked, then QPointer usage is safe.

2. QSharedDataPointer

Now this is a nice little class. It’s actually by far the most important of the smart pointer classes in Qt for its ingeniuty: it provides implicit sharing, with thread-safe copy-on-write. It requires that your class have a member called ref, which offers a function called ref() for increasing the reference count, and another called deref() that decreases that reference count and returns false when it drops to zero. If you derive your class from QSharedData, you get exactly that. Moreover, the size of a QSharedDataPointer object is exactly the size of a pointer. That means you can replace normal pointers with it in your code without breaking Binary Compatibility.

This class is the basis of all Qt value-type, implicit-shared, thread-safe copy-on-write recent classes, like QNetworkProxy. The only reason why it isn’t used in the base classes like QByteArray, QString and QList is that those classes were developed before this class was made. There’s nothing technically stopping the retrofitting of those classes with QSharedDataPointer.

So QSharedDataPointer is a strong smart pointer class, sharing data.

3. QExplicitlySharedDataPointer

This class is exactly like QSharedDataPointer (so it’s a a strong smart pointer class, sharing data), with the only difference that it never implicitly causes the detach. With QSharedDataPointer, any non-const access will cause the data to be copied. With QExplicitlySharedDataPointer, you have to call detach() for that to happen. This allows you to implement explicitly-shared data classes — which Qt doesn’t have anymore, but Qt 3 did in QMemArray (so it’s present in Qt4′s Qt3Support Q3MemArray).

But it also allows you to have finer-grained control of the detaching operation. In fact, if the Qt Tool classes were to be retrofitted with a smart pointer class, they’d be using QExplicitlySharedDataPointer instead. Using this class allows the code to delay the detaching until the very last moment, ensuring that no unnecessary memory access happens.

4. QtPatternist::AutoPtr

This is an internal class used by the QtXmlPatterns module. It’s basically your stock, dumb pointer wrapper. So it implements a strong pointer. It doesn’t share it, though.

The reason this class exists in the first place is that the QtXmlPatterns module makes extensive use of exceptions internally. To survive exceptions being thrown without leaking memory, a pointer wrapper is indicated. QtXmlPatterns also uses reference-counted classes, for which AutoPtr is not indicated — in that case, it uses QExplicitlySharedDataPointer.

5. QSharedPointer

This class was created as a response to QtPatternist::AutoPtr. When I started writing it, I intended for it to be ready for Qt 4.4 and replace the use of the internal class that Frans had written and what I perceived as a misuse of QExplicitlySharedDataPointer. QtXmlPatterns was using QExplicitlySharedDataPointer not for sharing data, but for sharing pointers. The objects it was sharing were not copiable. A later investigation, however, revealed that QtScript, Phonon, and Solid were using it for the same purpose. (In fact, QtScript introduced QExplicitlySharedDataPointer for that purpose in 4.3)

So QSharedPointer was shelved for 4.4, but was reborn in 4.5. It implements a strong smart pointer class, sharing the pointer. It has all the features you may want in a modern pointer class: it is polymorphic, it supports static, const, and dynamic casts, it implements atomic reference-counting and thread-safe semantics, it supports custom deleters. But note that, when I say it implements thread-safe semantics, it’s only for the pointer itself: remember it shares the pointer, not the data.

It comes with a cost, though: to support polymorphism correctly, the size of QSharedPointer is actually twice the size of a normal pointer. This means you cannot maintain binary compatibility while replacing a normal pointer with it in public parts of your API. You can use it internally in your code, though.

6. QWeakPointer

This is the companion class of QSharedPointer. If that implements a strong control of the pointer, QWeakPointer is a weak smart pointer class, sharing the pointer. It works in tandem with QSharedPointer: QWeakPointer can only be created from a QSharedPointer and they let you know when a QSharedPointer has been deleted.

They can be promoted to QSharedPointer, though, in a thread-safe manner. So it allows us to rewrite the code above to be safer:

    QWeakPointer<Data> weak(getSharedPointer());

    // [...]
    QSharedPointer<Data> ptr = weak;
    if (!ptr.isNull())
        ptr->doSomething();

In this case, the promotion of a QWeakPointer to a QSharedPointer will either succeed or it won’t. But that’s a thread-safe decision: if it does succeed, then the resulting object is guaranteed not to get deleted, while you hold the ptr reference (again, as long as everyone plays by the same rules).

With 4.6, I added a nifty new feature to QWeakPointer: its ability to track QObjects as well, without passing through a QSharedPointer. It can be used to determine whether a QObject-derived object has been deleted already or not. So it implements a weak pointer class sharing the pointer value for QObject-derived classes. Sounds familiar? Yes, that’s the idea: you can replace the old, slow QPointer with a faster, modern alternative. Just be careful that the size of QWeakPointer is not the same size of QPointer.

7. QGuard

This is another internal class. It was added to replace QPointer because that is very slow (it uses a global, mutex-protected QHash, which must be accessed by every QObject destructor). It’s actually what prompted me to write the QWeakPointer QObject-tracking feature. But it’s in a state of flux: we don’t know whether we’re going to keep or even use this class. Anyway, it’s internal, so you really don’t care about it.

8. QScopedPointer

This is the new kid in the block: it implements a non-shared strong pointer wrapper. It was created because of our attempt at handling the Symbian platform’s exceptions in our container classes: we needed a way to free resources without writing try/catch everywhere. A scoped pointer provides a very nice way to do RAII. In fact, QScopedPointer is actually a full replacement for QtXmlPattern’s QtPatternist::AutoPtr. Both implement the same functionality, so the internal one can be dropped.

Some people commented in Harald’s blog that we could’ve used QSharedPointer. Actually, we couldn’t: QSharedPointer has the size of two pointers, but we’re replacing Qt code that has the size of one pointer, so we needed a class that fits into that space. That’s also the reason why QScopedPointer has a custom deleter as a template parameter, as opposed to a parameter to the constructor (like QSharedPointer does): it has no space in those 4 or 8 bytes to store the custom deleter.

What’s more, QSharedPointer implements atomic reference-counting. Never mind the fact that it’s atomic: the reference counting is absolutely unnecessary for the cases that QScopedPointer is trying to solve.

Why not C++0x? Why not TR1? Why not Boost?

Some people in Harald’s blog suggested we should use std::shared_ptr (C++0x) or std::tr1::shared_ptr (TR1). I’m sorry, but those people didn’t see very far: we can’t use C++0x. It’s not even approved and there are only two compilers that implement initial support for it (GCC since 4.3 and MSVC 2010, which is in beta). It’s not even funny to suggest using C++0x for Qt at this point. You can use it for your own code, but we can’t use it in Qt.

TR1 has been implemented by more compilers. Unfortunately, not enough. We have to deal with compilers that haven’t implemented C++98 fully yet — or people who don’t bother to change their compiler settings. For example, the latest version of the Sun Studio compiler on Solaris (Sun Studio 12, with CC 5.10) still comes with the RogueWave implementation of pre-C++98 STL. If you read Sun’s article comparing RW stdlib to stlport4, you’ll see why they still keep the 11-year-old library as default. But the point is that they do, which means we have to deal with it. (Fortunately, other compiler vendors provide newer STL implementations, even though their compilers are sometimes far too picky)

That means the only smart pointer from STL we can use in Qt is std::auto_ptr. And even then there are issues (RW stdlib doesn’t implement member templates).

That leaves Boost. And there are some nice things in Boost: boost::shared_ptr, boost::intrusive_ptr, boost::scoped_ptr, etc. In fact, there are a lot of nice things in Boost. Very often I see things there that I’d like to have in Qt. Of course, that means I can just add said feature to Qt as well. There’s nothing stopping me, aside from, well, my day job :-)

One of the main problems with boost is that it provides an “un-Qt-ish” API — to say the least; I prefer calling it “horrible API”, but that’s a statement of opinion, not fact. Even if Boost’s API is intuitive to some people, it represents a departure from Qt’s API. That means those people using Qt and Boost need to learn Boost’s way of doing things as well, their naming of functions, etc.

At the very least, we’d have to wrap Boost’s API around with a Qt shell. But if we go further, we see that Qt loses control of an important piece of its technology. We then have to deal with whatever problems they have, at their schedules. Also, it adds a dependency to Qt, one we can’t justify because they don’t promise binary compatibility (cursory search over the web; please correct me if I’m wrong). Binary compatibility is the other of the main problems.

So, no, Boost is not an option either.

Conclusion

So Qt has too many smart pointer classes. Or does it?

In fact, Qt has only these pointer classes if you exclude the internal classes and you deprecate QPointer:

Class Description
QSharedDataPointer / QExplicitlySharedDataPointer Implements sharing of data (not of pointers), implicitly and explicitly, respectively
QSharedPointer Implements reference-counted strong sharing of pointers
QWeakPointer Implements reference-counted weak sharing of pointers
QScopedPointer / QScopedArrayPointer Implements non-reference-counted strong pointer wrapper (QSharedPointer’s little brother)

Update 1: QExplicitlySharedDataPointer can be used to implement reference-counted sharing of pointers when the target class includes the reference counter (similar to boost::intrusive_ptr)

Update 2: QScopedPointer is really based on the API of boost::scoped_ptr (but is not a copy); QSharedPointer and QWeakPointer were developed from the scratch.

Did you like this? Share it:
Bookmark and Share

Posted in Qt

21 comments to Count with me: how many smart pointer classes does Qt have?

Philippe says:

Great topic… to add to the Qt doc!
More generally, some topics of this blog, or excerpt of them, would be good additions to the Qt doc.

Andre says:

Thank you for this insightful overview. I think the Qt documentation would greatly benefit from having such a (this?) overview as an integral part of it! Overviews like this, explaining the differences between and usecases of classes that at first may seem similar are a valuable resource to understand what to use in what case. Qt has over 500 (public) classes by now, and that is a lot to take in.

Anarky says:

You nearly convinced me regarding the non use of C++0x or TR1. Some might argue that you could provide an implementations of std::shared_ptr for those compilers that don’t … But Ok, that’s not your job to implement the std library.
Concerning boost however … “if we go further, we see that Qt loses control of an important piece of its technology” … that sounds so NIHish to me. with the same reasoning you could say you shouldn’t reuse the std library (and that’s basically what you do in fact) and then why not that you can’t use C++ as is (ok with moc you nearly do that …)
The problem to me is that it makes it more and more difficult to use Qt together with another library. If I use a library that relies on boost and another that relies on Qt thay don’t share the same autoptr classes, the same collection classes, etc …
Luckyly, Qt provides so many things you can often build an application using mostly Qt (or kdelibs) only as dependancies, but when for a reason or another you can’t, it’s really a pain in the a**.

Birdy says:

>> One of the main problems with boost is that it provides an “un-Qt-ish” API — to say the least; I prefer calling it “horrible API”

Razvan Petru says:

Excellent post. I especially liked the small bits of implementation details about S60 or exceptions.

Is it possible to disable the atomic sharing in the shared pointer (through a policy, for instance)? If I want to use it in just one thread, I imagine that it would be faster this way.

Thiago Macieira says:

@Anarky: you’ll see that I mentioned two main problems with using boost: the un-Qt-ish API and the lack of binary compatibility. The control is not an issue — as long as it doesn’t affect our binary compatibility policies. We can give control away, as long as people promise not to break our binary compatibility promise.

But, in any case, we don’t like to. We pride ourselves in having a concise, elegant API. We have articles and manuals on writing good API. We spend a lot of time making sure our APIs are good. Giving away control to something as important as our API to third-parties who don’t subscribe to the same principles is not something we’ll do (principles that differ: naming convention, use of exceptions, sharing or not, etc.)

Johannes says:

Hi there. Nice overview… But here is one thing confusing me. You write about sharing pointers and data. I don’t quite get the difference. So if i have two QSharedPointer objects that both point to the same data (say, i created one, and then copied it) – then i would instinctively say “the pointers are different (not shared), but the data is the same (shared)” – but your description says that the pointer is shared, *not* the data.

I’m confused, can you please help out? Is this some fundamental Qt wording one should know?

Thiago Macieira says:

Johannes: this is the important point I tried to get through in the first section. When you use QSharedPointer, you’re sharing the ownership of the pointer. The class controls and handles the pointer only — anything else (like access to the data) is outside its scope. When you use QSharedDataPointer, you’re sharing the data. And that class is intended for implicit sharing: so it may split up.

This difference is clearly illustrated by the operator== functions. For QSharedPointer, two objects are equal if they have the same pointer. For QSharedDataPointer, two objects are equal if they have the same pointer or if they have the same data (i.e., d == other.d || *d == *other.d).

Anarky says:

@Thiago well … I guess you made your choice, and arguing is of no use now. Perhaps there were very good reasons not to choose boost implementation … binary compatibility might be one. However I don’t think there will be much change in the implementation of smart pointers and even in that case, you could stick to a given boost version. Anyway, the most important: I’m not involved at all in Qt development, you are, so you make the choices !

On the other hand, even if you get away from this particular issue, the argument you give (except for binary compatibility that I talked about earlier) sound wrong to me. The API is ugly/not Qtish: yeap, but then someone else with different taste will feel the Qt API is not to his taste and will start doing its own implementation ? When there is a working solution that brings no IP problem, it’s too bad to make library mixing difficult just because someone doesn’t like one kind of API. Sun did good with respect to that: they fixed API guidelines for java, even if you don’t like them, you follow them because it’s how it is. It doesn’t solve the problem, there are still people wanting to do things their own way, but it sure ease library mixing … In the worst cases, if the API is really a huge problem, developping a phonon-kind wrapper might be ok.

To summarize, I don’t want you to take that as a personal attack, especially beacause I really like Qt and the choices that have been made for it. But I think sharing instead of doing the same things over and over again is one of the main advantage of FOSS. That’s too sad it is sometimes not taken as an advantage because of relatively small glitches such as API taste and control over the code.

yours,
Anarky (Julien B.)

Chris says:

As a heavy duty boost user, the biggest thing I miss from Qt is ‘boost::enable_shared_from_this’ which allows you to retrieve the boost::shared_ptr from ‘this’. Equivalent functionality would be a welcome addition to QSharedPointer, although using QSharedDataPointer in place of QSharedPointer is an easy workaround that I hadn’t appreciated before this article today!

Thiago Macieira says:

@Chris: that’s in my plans. I didn’t have time to implement the “enable_shared_from_this” yet, but I know what it needs. I also need to go through a round of API review because that’s a horrible class name…

I’ve already received one implementation by email and I need to do the research before Qt 4.6 is out. The reason is that there’s a similar functionality added to QObject (the QWeakPointer tracking of QObjects) and I need to ensure that I can add the enable_shared_from_this functionality in 4.7 without breaking compatibility.

Thiago Macieira says:

@Anarky: I appreciate your comments and I don’t take it as a personal attack.

Many of our users — including Open Source users and even heavy users like KDE — would like to do what you’re describing. So I understand very well the need to share and API. But, fact is, Qt is still only C++ and there’s nothing stopping you from mixing containers. I’ve seen a lot of code doing it. And since STL containers don’t share anyway, the moment you use them you start making copies of the data. (Using boost::shared_ptr<std ::vector<Foo> > sounds like a horrible hack to me for something that the library should provide by default)

But that’s not the issue. Let me rephrase the two issues again:

First, the API. There are those who don’t like the Qt API (shocking, I know, but they exist :-P ). They will feel more comfortable with STL. Or with C. So Qt will represent a departure from what they’re used to, definitely. But if they use Qt, they need to understand the Qt style of programming anyway. So providing containers that subscribe to that style makes sense: there’s only one API style to learn. For those who, like me, like Qt API and don’t use STL in their projects, there’s no other style to learn.

This is not just about whether the API uses underscores or camelCase. This goes much further, like static polymorphism, the use of verbs/nouns/adjectives as function names, when exceptions are used (if ever), etc. I suggest taking a look at Matthias’s article on QQ13 about API Design: it’s quite a good read.

Second, the binary compatibility. We just cannot use anything in our public API that doesn’t have the same binary compatibility guarantees that Qt has, or stricter guarantees. For example, QFile and QTextStream have support for FILE *. That’s acceptable, because file streams are specified by ISO C and POSIX, so the compatibility guarantee is stricter. So we can’t use Boost containers in our API if they can’t promise us they’ll maintain compatibility.

You mentioned sticking to one version. Sure, we could do that in Qt. But then that means we’re imposing that version on you, the Qt user. Since we are using that version in our API, you must use that version and no other. So the net effect is that we enforce a binary compatibility on Boost by using it in our API. We even make the problem worse: instead of being able to use code from libraries using newer versions of Boost by copying the data, you simply cannot use those libraries.

We’ve been burned by this before, as recent as Qt 4.4. Apple, for example, has no qualms about breaking compatibility on their platforms (and Mac developers accept that — the Carbon/Cocoa debacle is the foremost example). In preparation for the 64-bit move, they changed the typedef of GLint from int to long (or the other way around, I don’t remember). That was fine for their API, but it broke Qt, because we had a few functions with GLint in the signature. Thankfully, we were able to restore compatibility by providing overloads for both int and long.

Writing a Phonon-style backend would make sense for something bigger. We’re considering, for example, boost::regex wrapped around in QRegExp. But for container classes, there’s just not much you can wrap around without losing all the inlining ability.

A third alternative would be to provide byte-compatible implementations. That is, be able to share the internals with Boost. Unfortunately, if they can’t guarantee binary compatibility, we can’t guarantee it either with them. So it would just be needlessly tying our hands.

The User says:

It shouldn’t be a problem to create policy-based pointers:
You could configure all the locking-, destruction-, sharing-, copying- and checking-features.

For example:
SharingPolicy: implementations would implement an setPtr-method and a destructor. Possible implementations: NormalPtr, RefCountedPtr, RefListPtr, AutoPtr (like std::auto_ptr), ScopedPtr, DynamicHierarchyPtr (can provide single-, double-, triple-…indirection), CheckedDynamicHierarchyPtr (against pointers pointing at them selves)
CheckingPoliy: NoCheck, AssertCheck, ExceptionCheck…
LockingPolicy: provides lock- and unlock-inline-methods
StoragePolicy: provides data-access, for example system-dependent pointers or pointers to arrays could be used
OperatorPolicy: policies to enable or disable operators like “operator=” or “operator pointertype”

=> endless possibilities ;)

See also “Modern C++ Design”.

The User

rule says:

Thiago, I think you right choosing the way of re-implementing boost smart pointers.
1. It’s a lowest part, other classes in the future (I think) will base on a top on this functionality, and Qt’s guys have to be sure of binary and functionality compatibility.
2. It’s help’s to make Qt more portable (for example I still use 2.95 gcc version on some projects, that designed for embedded devices)
3. it’s not make big overhead but makes a great functionality, I agree to pay this price.
4. It’s still C++, anyone still can make bridging between some 3-rd party lib and Qt. This is necessary not only for Qt. Mixing two independent framework’s functionality always need to make additional work on bridging. It’s C++ :-)
Grate work.

comment says:

Thanks for the boost bashing.
Glad to see someone has the heart to say HIS truth, even if it is not “politically correct”!

Don’t fear the flame-war!

comment says:

“static polymorphism”

Could you please explane why Qt is very conservative when using static polymorphism?
Now, having dropped support for msvc6 and other older compilers this reasons are obsolete.

Thanks Thiago.

Thiago Macieira says:

I think you misunderstand what static polymorphism is. See http://doc.trolltech.com/qq/qq13-apis.html#staticpolymorphism

comment says:

“static polymorphism”

OK, in the sense of “static type checking” at compile time versus “dynamic type checking” at runtime,
I thought you mean “polymorphism resolved at compile time”
http://en.wikipedia.org/wiki/Template_metaprogramming#Static_polymorphism

Therefore the polymorphism you talked about is maybe better called
“naming polymorphism” or “API polymorphism”

Thiago Macieira says:

We call that “static polymorphism” :-)

hippydream says:

Wonderful and extremely useful post.
The Qt docs could really need such an interesting overview of all the new (and old) Qt pointer classes :)

mongaulois says:

Hi.
About “But, unlike Qt 2 and Qt 3, the QObject of Qt 4 can live in several threads” for QPointer.
Could you explain why a QObject could be live in several threads???
For me QObject are not create for it and shared a QObject in several thread is absurde…

thanks

Commenting closed.