Retina display support for Mac OS, iOS and X11

Published Thursday April 25th, 2013 | by

Qt 5.0 added basic support for retina reasonable resolution displays. The upcoming Qt 5.1 will improve the support with new API and bug fixes. Qt 4.8 has good support, and backports of some of the Qt 5 patches are available.

While this implementation effort is mostly relevant to Mac and iOS developers, it is interesting to look at how other platforms handle high-dpi displays. There are two main approaches:

  • DPI-based scalingWin32 GDI and KDE. In approach the application works in the full physical device resolution and is provided with a DPI setting or scaling factor, which should be used to scale layouts. Fonts are automatically scaled by the OS (as long as you specify the font sizes in points and not pixels)
  • Pixels By Other Names. In this approach the physical resolution is (to various degrees) hidden to the application. Physical pixels are replaced with logical pixels:
    Platform/API Logical Physical
    HTML CSS pixel Device pixel
    Apple Point Pixel
    Android Density-independent pixel (dp) (Screen) Pixel
    Direct2D Device Independent Pixel (DIP) Physical Pixel
    Qt (past) Pixel Pixel
    Qt (now) Device-Independent Pixel Device Pixel

Qt has historically worked in device pixels with DPI scaling. Back in 2009 support for high DPI values on Windows was improved. The Qt layouts do however not account for increased DPI. Qt 5 now adds support of the “new pixels” type of scaling.

(Are there other high-dpi implementations out there? Use the comments section for corrections etc.)

Mac OS X High-dpi Support

The key to the OS X high-dpi mode is that most geometry that was previously specified in device pixels are now in device-independent points. This includes desktop geometry (which on the 15 inch retina MacBook Pro is 1440×900 and not the full 2880×1800), window geometry and event coordinates. The CoreGraphics paint engine is aware of the full resolution and will produce output at that resolution. For example, a 100×100 window occupies the same area on screen on a normal and high-dpi screen (everything else being equal). On the high-dpi screen the window’s backing store contains 200×200 pixels.

The main benefits of this mode is backwards compatibility and free high-dpi vector graphics. Unaware applications simply continue to work with the same geometry as before and can keep hardcoded pixel values. At the same time they get crisp vector graphics such as text for free. Raster graphics does not get an automatic improvement but is manageable. The downside is the inevitable coordinate system confusion when working with code that mixes points and pixels.

The scale factor between points and pixels is always 2x. This is also true when changing the screen resolution – points and pixels are scaled by the same amount. When scaling for “More Space” applications will render to a large backing store which is then scaled down to the physical screen resolution.

Scaling the user interface resolution on Mac OS

If you don’t have access to retina hardware there is also an emulation mode which can be useful when used on an extra monitor. Open Display Properties and select one of the HiDPI modes. (See this question on stack overflow if there are none.)

Enabling high-dpi for OS X Applications

High DPI mode is controlled by the following keys in the Info.Plist file:

<key>NSPrincipalClass</key>
<string>NSApplication</string>
<key>NSHighResolutionCapable</key>
<string>True</string>

Qmake will add these for you. (Strictly speaking it will only add NSPrincipalClass, NSHighResolutionCapable is optional and true by default).

If NSHighResolutionCapable is set to false, or the keys are missing, then the application will be rendered at the “normal” resolution and scaled up. This looks horrible and should be avoided, especially since the high-dpi mode is very backwards compatible and the application gets a lot of high-dpi support for free.

Scaled Qt Creator

High DPI Qt Creator

 

 

 

 

 

 

 

 

 
(Appart from a patch to update the “mode” icons, this an unmodified version of Qt Creator.)

Qt implementation details

Mac OS 10.8 (unofficially 10.7?) added support for high-dpi retina displays. Qt 4 gets this support for free, since it uses the CoreGraphics paint engine.

Qt 5 uses the raster paint engine and Qt implements high-dpi vector graphics by scaling the painter transform. HITheme provides high-dpi Mac style for both Qt 4 and 5. In Qt 5 the fusion style has been tweaked to run well in high-dpi mode.

OpenGL is a device pixel based API and remains so in high-dpi mode. There is a flag on NSView to enable/disable the 2x scaling – Qt sets it in all cases. Shaders run in device pixels.

Qt Quick 1 is built on QGraphicsView which is a QWidget and gets high-dpi support through QPainter.

Qt Quick 2 is built on Scene Graph (and OpenGL) which has been updated with high-dpi support. The Qt Quick Controls (née Desktop Components) has also been updated to render in high-dpi mode, including using distance field text rendering.

The take-away point here is that for app developers this doesn’t matter, you can do most of your work in the comfort of the device-independent pixel space while Qt and/or the OS does the heavy lifting. There is one exception which is raster content – high-dpi raster content needs to be provided and correctly handled by application code.

Widgets and QPainter

QPainter code can mostly be kept as is. As an example lets look at drawing a gradient:

QRect destinationRect = ...
QGradient gradient = ...
painter.fillRect(rect, QBrush(gradient));

On high-dpi displays the gradient will have the same size on screen but will be filled with more (device) pixels.

Drawing a pixmap is similar:

QRect destinationRect = ...
QPixmap pixmap = ...
painter.drawPixmap(destinationRect, pixmap);

To avoid scaling artifacts on high-dpi displays the pixmap must contain enough pixels: 2x the width and height of destinationRect. The application can either provide one directly or use QIcon to manage the different resolutions:

QRect destinationRect = ...
QIcon icon = ...
painter.drawPixmap(destinationRect, icon.pixmap(destinationRect.size()));

QIcon::pixmap() has been modified to return a larger pixmap on high-dpi systems. This is a behavior change and can break existing code, so it’s controlled by the AA_UseHighDpiPixmaps application attribute:

qApp->setAttribute(Qt::AA_UseHighDpiPixmaps);

The attribute is off by default in Qt 5.1 but will most likely be on by default in a future release of Qt.

Edge cases and devicePixelRatio

Qt Widgets has some edge cases. Ideally it would pass QIcons around and the correct pixmap would be select at draw time, but in reality Qt API often produces and consumes pixmaps instead. This can cause errors when the pixmap size is used for calculating layout geometry – the pixmap should not use more space on screen if it’s high-resolution.

To indicate that a 200×200 pixmap should occupy 100×100 device-independent pixels use QPixmap::devicePixelRatio(). Pixmaps returned from QIcon::pixmap() will have a suitable devicePixelRatio set.

QLabel is one “pixmap consumer” example:

QPixmap pixmap2x = ...
pixmap2x.setDevicePixelRatio(2.0);
QLabel *label = ...
label->setPixmap(pixmap2x);

QLabel then divides by devicePixelRatio to get the layout size:

QSize layoutSize = pixmap.size() / pixmap.devicePixelRatio();

Several issues like this has been fixed in Qt, and application code can have similar code that needs to be corrected before enabling AA_UseHighDpixmaps.

The devicePixelRatio() accessor is available on several Qt classes:

Class Note
QWindow::devicePixelRatio() Preferred accessor
QScreen::devicePixelRatio()
QGuiApplication::devicePixelRatio() Fallback if there is no QWindow pointer
QImage::[set]devicePixelRatio()
QPixmap::[set]devicePixelRatio()

Text

Font sizes can be kept as-is, and produce similarly-sized (but crisp) text on high-dpi displays. Font pixel sizes are device-independent pixel sizes. You never get tiny text on high-dpi displays.

QGlWidget

OpenGL operates in device pixel space. For example, the width and height passed to glViewport should be in device pixels. QGLWidget::resizeGL() gives the width and height in device pixels.

However, QGLWidget::width() is really QWidget::width() which returns a value in device-independent pixels. Resolve it by multiplying with widget->windowHandle()->devicePixelRatio() if needed.

Qt Quick 2 and controls

Qt Quick 2 and the Qt Quick Controls work well out-of-the box. As with widgets coordinates are in device-independent pixels. Qt Quick has fewer raster-related edge cases, since the QML Image element specifies the image source as a url which avoids passing around pixmaps.

Qt Quick Controls

One exception is OpenGL shaders that run in device pixel space and see the full resolution. This is usually not a problem, the main thing to be aware of is that mouse coordinates are in device-independent pixels and may need to be converted to device pixels.

shadereffects example in action

Managing high-resolution raster content

As we have seen, raster content won’t look nice when scaled and high-resolution content should be provided. As an app developer you have two options: (ignoring the “do-nothing” option)

  • Replace existing raster content with a high-resolution version
  • Provide separate high-resolution content

The first option is convenient since there is only one version of each resource. However, you may find (or your designer will tell you) that resources like icons look best when created for a specific resolution. To facilitate this, Qt as adopted the “@2x” convention for image filenames:

foo.png
foo@2x.png

High-resolution content can be provided side-by-side with the originals. The “@2x” version will be loaded automatically when needed by the QML Image element and QIcon:

Image { source = “foo.png” }
QIcon icon(“foo.png”)

(remember to set AA_UseHighDpiPixmaps for QIcon)

Experimental cross-platform high-dpi support:

QPA allows us to relatively easily make a cross-platform implementation. The Qt stack can be divided into three layers:

  1. The Application layer (App code and Qt code that uses the QPA classes)
  2. The QPA layer (QWindow, QScreen, QBackingStore)
  3. The platform plugin layer (QPlatform* subclasses)

Simplified, the application layer operates in the device-independent pixel space and does not know about device pixels. The platform plugins operates in device pixel space and does not know about device-independent pixels. The QPA layer sits in between and translates, based on a scale factor set by the QT_HIGHDPI_SCALE_FACTOR environment variable.

In reality the picture is a little bit more complicated, with some leakage between the layers and the special Mac and iOS exception that there is additional scaling on the platform.

Code is on github. Finally, screenshots of Qt Creator on XCB:

DPI scaled Qt Creator

QT_HIGDPI_SCALE_FACTOR=2 Scaled Qt Creator

 

 

Did you like this? Share it:
Bookmark and Share

Posted in Graphics, Mac OS X, QPA, Qt

14 comments to Retina display support for Mac OS, iOS and X11

Thomas says:

I’d like to think that the proper way to use differnt form factors is to accept that they have different DPIs.
I think its buying into Apples marketing a bit too much to say “high resolution” like it introduced a second DPI, and the rest of the world is still looking at ‘low’ resolution.
The truth is naturally that there are an almost infiinite amout of resolutions, many of them higher than what Apple ships (bb10 for instance).

I’m a bit confused about the (short) reference to QtQuick. I was quite comfortable with the idea that a certain device I target has a certain DPI and certain amount of pixels. I can design for that, life is good.
So I’m wondering if that no longer is the case, is this change going to mean some QML elements get twice as big somehow? Is that only on Apple products?

In Qt4 times I often used code that essentially takes the user configured font, calculates the x-height and uses the pixel-size as a standard size to determine things like spacing in my UI elements.
I have the impression that this is the proper way to do it, having a magick “multiply-by-two” seems limited to only one vendors hardware, as I mentioned above ;)

ps. the screenshots of creator show a bug in the scaling of the current-build-icon.

I fully agree to you Thomas! Only supporting double-scaled images does not fulfill today’s requirements, especially not on mobiles where the device’s resolutions vary from 480×320 up to 2048×1536 like the iPad 3 or even more some Android tablets have. Thus we added a custom content-scaling possiblity as part of the V-Play game engine on top of Qt, which allows to have arbitrary scale factors and custom definable suffixes. In we explain how to achieve content-scaling across devices and how to handle different aspect ratios with the same UI layout.

It would be great to see a similar concept built-in Qt though!

Morten Johan Sørvig says:

Looks interesting! And I agree on the goal of making it easier to work with a wide array of resolutions.

Morten Johan Sørvig says:

Well, I think the documentation linked to in the first table to makes a good argument that the “new pixel type” approach is not an Apple-only thing. The fixed 2x scale factor is as far as i have seen exclusive to Apple, and makes developers life easier on those platforms by reducing resolution fragmentation.

The changes to Qt 5.1 are only relevant on OS X and iOS, and they are as such targeted at making Qt work well on those platforms. This also includes Qt Quick, were we have had good results making the desktop controls work on both retina and non-retina hardware (meaning you can re-use the same layouts and pixel positioning tweaks for both). We’re also using the same approach on iOS, although we don’t have much experience with control sets there yet.

I don’t think this would prevent us from letting the application see the full resolution if it wants to – it could be a mode that the app sets at startup. We can then argue about which should be the default :)

“Free scaling” to non-2x factors is in fact possible with research code i talk about in the final paragraph – you can set QT_HIGHDPI_SCALE_FACTOR=1.8 for example. In the current implementation it’s quite usable but not pixel perfect.

Fritzt says:

Many thanks Morten for also providing a backport to Qt 4.8

Qt4iOS says:

Nice work! I’ve been playing around with this on iOS (having backported it to 5.0.2), and it works really well. There certainly are times when you want to have it turned off (and use the full resolution of the display), and other times when you want to turn both ‘retina’ and ‘highres’ off and run apps in ‘non-retina’ display mode. So you need to have those options available (currently they are compile time for retina/highres, and configuration driven for ‘non-retina’).

Thanks for some other informative website. The place else may I get that kind of information written in such a perfect way?
I’ve a mission that I am simply now working on, and I’ve been on the
glance out for such info.

Florian Boucault says:

Ubuntu has implemented an approach based on a unit equivalent to the logical pixel (called dp in Ubuntu):

http://developer.ubuntu.com/api/ubuntu-12.10/qml/mobile/resolution-independence.html

There is also a multiple of that unit called grid unit that makes it more likely that UI elements align gracefully.

SciK says:

In the “Desktop Components” screenshot, most components look fine, except for the corners of the tab bar. How come?

Alexander says:

There’s a problem in Qt4 under X11 when used with high-ish DPI values (greater than 120), which is quite unfortunate. I wonder why it didn’t get any attention before, considering there’s no such bug on Windows or OS X.

See the bug report – Qt doesn’t scale the widgets according to DPI under X11 (unlike under Windows and OSX)
https://bugreports.qt-project.org/browse/QTBUG-27389

Robin Lobel says:

OpenGL support with hidpi (OSX 10.7.5 with hidpi) is broken with Qt 5.0.2.
Just try to run Hello GL: it’s pixelated and the render window goes outside the drawing area.
Will this be fixed in Qt 5.1 ? Is there any beta coming soon ? This is a blocking bug for my application..

Romuald C. says:

What aboutQt Stylesheets ? Is there a way to specify images devicePixelRatio ?

Peter Housel says:

It would be nice if the eglfs (embedded OpenGL ES) backend supported some way of configuring the display DPI.

Commenting closed.