Morten Johan Sørvig

Retina display support for Mac OS, iOS and X11

Published Thursday April 25th, 2013 | by

Qt 5.0 added basic support for retina reasonable resolution displays. The upcoming Qt 5.1 will improve the support with new API and bug fixes. Qt 4.8 has good support, and backports of some of the Qt 5 patches are available.

While this implementation effort is mostly relevant to Mac and iOS developers, it is interesting to look at how other platforms handle high-dpi displays. There are two main approaches:

  • DPI-based scalingWin32 GDI and KDE. In approach the application works in the full physical device resolution and is provided with a DPI setting or scaling factor, which should be used to scale layouts. Fonts are automatically scaled by the OS (as long as you specify the font sizes in points and not pixels)
  • Pixels By Other Names. In this approach the physical resolution is (to various degrees) hidden to the application. Physical pixels are replaced with logical pixels:
    Platform/API Logical Physical
    HTML CSS pixel Device pixel
    Apple Point Pixel
    Android Density-independent pixel (dp) (Screen) Pixel
    Direct2D Device Independent Pixel (DIP) Physical Pixel
    Qt (past) Pixel Pixel
    Qt (now) Device-Independent Pixel Device Pixel

Qt has historically worked in device pixels with DPI scaling. Back in 2009 support for high DPI values on Windows was improved. The Qt layouts do however not account for increased DPI. Qt 5 now adds support of the “new pixels” type of scaling.

(Are there other high-dpi implementations out there? Use the comments section for corrections etc.)

Mac OS X High-dpi Support

The key to the OS X high-dpi mode is that most geometry that was previously specified in device pixels are now in device-independent points. This includes desktop geometry (which on the 15 inch retina MacBook Pro is 1440×900 and not the full 2880×1800), window geometry and event coordinates. The CoreGraphics paint engine is aware of the full resolution and will produce output at that resolution. For example, a 100×100 window occupies the same area on screen on a normal and high-dpi screen (everything else being equal). On the high-dpi screen the window’s backing store contains 200×200 pixels.

The main benefits of this mode is backwards compatibility and free high-dpi vector graphics. Unaware applications simply continue to work with the same geometry as before and can keep hardcoded pixel values. At the same time they get crisp vector graphics such as text for free. Raster graphics does not get an automatic improvement but is manageable. The downside is the inevitable coordinate system confusion when working with code that mixes points and pixels.

The scale factor between points and pixels is always 2x. This is also true when changing the screen resolution – points and pixels are scaled by the same amount. When scaling for “More Space” applications will render to a large backing store which is then scaled down to the physical screen resolution.

Scaling the user interface resolution on Mac OS

If you don’t have access to retina hardware there is also an emulation mode which can be useful when used on an extra monitor. Open Display Properties and select one of the HiDPI modes. (See this question on stack overflow if there are none.)

Enabling high-dpi for OS X Applications

High DPI mode is controlled by the following keys in the Info.Plist file:

<key>NSPrincipalClass</key>
<string>NSApplication</string>
<key>NSHighResolutionCapable</key>
<string>True</string>

Qmake will add these for you. (Strictly speaking it will only add NSPrincipalClass, NSHighResolutionCapable is optional and true by default).

If NSHighResolutionCapable is set to false, or the keys are missing, then the application will be rendered at the “normal” resolution and scaled up. This looks horrible and should be avoided, especially since the high-dpi mode is very backwards compatible and the application gets a lot of high-dpi support for free.

Scaled Qt Creator

High DPI Qt Creator

 

 

 

 

 

 

 

 

 
(Appart from a patch to update the “mode” icons, this an unmodified version of Qt Creator.)

Qt implementation details

Mac OS 10.8 (unofficially 10.7?) added support for high-dpi retina displays. Qt 4 gets this support for free, since it uses the CoreGraphics paint engine.

Qt 5 uses the raster paint engine and Qt implements high-dpi vector graphics by scaling the painter transform. HITheme provides high-dpi Mac style for both Qt 4 and 5. In Qt 5 the fusion style has been tweaked to run well in high-dpi mode.

OpenGL is a device pixel based API and remains so in high-dpi mode. There is a flag on NSView to enable/disable the 2x scaling – Qt sets it in all cases. Shaders run in device pixels.

Qt Quick 1 is built on QGraphicsView which is a QWidget and gets high-dpi support through QPainter.

Qt Quick 2 is built on Scene Graph (and OpenGL) which has been updated with high-dpi support. The Qt Quick Controls (née Desktop Components) has also been updated to render in high-dpi mode, including using distance field text rendering.

The take-away point here is that for app developers this doesn’t matter, you can do most of your work in the comfort of the device-independent pixel space while Qt and/or the OS does the heavy lifting. There is one exception which is raster content – high-dpi raster content needs to be provided and correctly handled by application code.

Widgets and QPainter

QPainter code can mostly be kept as is. As an example lets look at drawing a gradient:

QRect destinationRect = ...
QGradient gradient = ...
painter.fillRect(rect, QBrush(gradient));

On high-dpi displays the gradient will have the same size on screen but will be filled with more (device) pixels.

Drawing a pixmap is similar:

QRect destinationRect = ...
QPixmap pixmap = ...
painter.drawPixmap(destinationRect, pixmap);

To avoid scaling artifacts on high-dpi displays the pixmap must contain enough pixels: 2x the width and height of destinationRect. The application can either provide one directly or use QIcon to manage the different resolutions:

QRect destinationRect = ...
QIcon icon = ...
painter.drawPixmap(destinationRect, icon.pixmap(destinationRect.size()));

QIcon::pixmap() has been modified to return a larger pixmap on high-dpi systems. This is a behavior change and can break existing code, so it’s controlled by the AA_UseHighDpiPixmaps application attribute:

qApp->setAttribute(Qt::AA_UseHighDpiPixmaps);

The attribute is off by default in Qt 5.1 but will most likely be on by default in a future release of Qt.

Edge cases and devicePixelRatio

Qt Widgets has some edge cases. Ideally it would pass QIcons around and the correct pixmap would be select at draw time, but in reality Qt API often produces and consumes pixmaps instead. This can cause errors when the pixmap size is used for calculating layout geometry – the pixmap should not use more space on screen if it’s high-resolution.

To indicate that a 200×200 pixmap should occupy 100×100 device-independent pixels use QPixmap::devicePixelRatio(). Pixmaps returned from QIcon::pixmap() will have a suitable devicePixelRatio set.

QLabel is one “pixmap consumer” example:

QPixmap pixmap2x = ...
pixmap2x.setDevicePixelRatio(2.0);
QLabel *label = ...
label->setPixmap(pixmap2x);

QLabel then divides by devicePixelRatio to get the layout size:

QSize layoutSize = pixmap.size() / pixmap.devicePixelRatio();

Several issues like this has been fixed in Qt, and application code can have similar code that needs to be corrected before enabling AA_UseHighDpixmaps.

The devicePixelRatio() accessor is available on several Qt classes:

Class Note
QWindow::devicePixelRatio() Preferred accessor
QScreen::devicePixelRatio()
QGuiApplication::devicePixelRatio() Fallback if there is no QWindow pointer
QImage::[set]devicePixelRatio()
QPixmap::[set]devicePixelRatio()

Text

Font sizes can be kept as-is, and produce similarly-sized (but crisp) text on high-dpi displays. Font pixel sizes are device-independent pixel sizes. You never get tiny text on high-dpi displays.

QGlWidget

OpenGL operates in device pixel space. For example, the width and height passed to glViewport should be in device pixels. QGLWidget::resizeGL() gives the width and height in device pixels.

However, QGLWidget::width() is really QWidget::width() which returns a value in device-independent pixels. Resolve it by multiplying with widget->windowHandle()->devicePixelRatio() if needed.

Qt Quick 2 and controls

Qt Quick 2 and the Qt Quick Controls work well out-of-the box. As with widgets coordinates are in device-independent pixels. Qt Quick has fewer raster-related edge cases, since the QML Image element specifies the image source as a url which avoids passing around pixmaps.

Qt Quick Controls

One exception is OpenGL shaders that run in device pixel space and see the full resolution. This is usually not a problem, the main thing to be aware of is that mouse coordinates are in device-independent pixels and may need to be converted to device pixels.

shadereffects example in action

Managing high-resolution raster content

As we have seen, raster content won’t look nice when scaled and high-resolution content should be provided. As an app developer you have two options: (ignoring the “do-nothing” option)

  • Replace existing raster content with a high-resolution version
  • Provide separate high-resolution content

The first option is convenient since there is only one version of each resource. However, you may find (or your designer will tell you) that resources like icons look best when created for a specific resolution. To facilitate this, Qt as adopted the “@2x” convention for image filenames:

foo.png
foo@2x.png

High-resolution content can be provided side-by-side with the originals. The “@2x” version will be loaded automatically when needed by the QML Image element and QIcon:

Image { source = “foo.png” }
QIcon icon(“foo.png”)

(remember to set AA_UseHighDpiPixmaps for QIcon)

Experimental cross-platform high-dpi support:

QPA allows us to relatively easily make a cross-platform implementation. The Qt stack can be divided into three layers:

  1. The Application layer (App code and Qt code that uses the QPA classes)
  2. The QPA layer (QWindow, QScreen, QBackingStore)
  3. The platform plugin layer (QPlatform* subclasses)

Simplified, the application layer operates in the device-independent pixel space and does not know about device pixels. The platform plugins operates in device pixel space and does not know about device-independent pixels. The QPA layer sits in between and translates, based on a scale factor set by the QT_HIGHDPI_SCALE_FACTOR environment variable.

In reality the picture is a little bit more complicated, with some leakage between the layers and the special Mac and iOS exception that there is additional scaling on the platform.

Code is on github. Finally, screenshots of Qt Creator on XCB:

DPI scaled Qt Creator

QT_HIGDPI_SCALE_FACTOR=2 Scaled Qt Creator

 

 

14 Comments


Posted in Graphics, Mac OS X, QPA, Qt

Qt for iOS Preview

Published Tuesday March 5th, 2013 | by

We are very excited to be able to bring Qt to a new platform. Qt for iOS is planned to be a supported part of Qt 5.2, scheduled for release late 2013. The scope of that release is not completely determined: available resources, platform/app store restrictions and Qt legacy set constraints on the project. This blog outlines the current plan.

Qt 5.1 will contain a preview – which was in fact merged last Friday. It’s also possible to take a look today by checking out the Qt source code. See “Getting Started” for detailed instructions below.

Development and Deployment
Development and especially deployment is done using Xcode. The supported workflow is to maintain a .pro file based project, export it to Xcode (and re-export when the project setup changes), and then build and deploy using Xcode. Source code editing can as usual be done in any editor.

Qt 5 Architecture
Understanding the level of support various Qt modules will get, requires a brief detour into Qt 5 architecture. With Qt 5 there is now a common API that all platforms implement (the Qt platform abstraction – QPA). Most of the Qt for iOS project work will happen here, and this is the code base the team will support. The QPA layer powers both widgets and QML (1 and 2).

Styling
The Qt Mac style uses the HITheme API provided by OS X to draw native-looking UI elements. There is no such API on iOS, which means creating a QiOSStyle similar to QMacStyle is not possible. Cross-platform styles such as the new Fusion style will however be available. Future styling efforts will focus on controls for Qt Quick 2.

Qt Quick 2: JavaScript engines and JIT
Qt Quick 2 uses the V8 javascript engine, which cannot be deployed on iOS due to policy and technical limitations related to V8′s use of a just-in-time compiler. This means no Qt Quick 2 on iOS for the time being. We are aware of the problem and are working on a solution.

What works today

  • Widgets
  • Graphics View
  • Qt Quick 1
  • OpenGL
  • Touch events
  • Orientation events
  • ++

Qt5 Cinematic Experience by QUIt Coding, ported to Qt Quick 1 and running on an iPad

 

Getting started

  1. Homework: Setup Xcode for development (acquire certificates, configure devices). Test-deploy one of the standard Xcode app templates. Make sure you have Git installed.
  2. Clone qtbase
    git clone git://gitorious.org/qt/qtbase.git qtbase-ios
    cd qtbase-ios/
    git checkout dev

    [Optional: git checkout ios for the development branch]
  3. Build QtBase for either device or simulator. Note that unlike Qt 4, Qt 5 does not support multi-architecture builds.
    /configure -xplatform unsupported/macx-ios-clang -developer-build -nomake examples -nomake tests -release [-sdk iphonesimulator]
    make
  4. Get the Simple Demo:
    git clone git://github.com/msorvig/qt-ios-demo.git
    cd qt-ios-demo
    ../qtbase-ios/bin/qmake
    open qt-ios-demo.xcodeproj

Update: Building Qt Quick 1
You need the full Qt source code, see Building Qt 5 from Git. Then build the QtScript and QtQuick1 modules:

cd qtscript; ../qtbase/bin/qmake; make; cd ..
cd qtquick1; ../qtbase/bin/qmake; make; cd ..

101 Comments


Posted in Uncategorized