A quick look at UIKit on macOS

June 7, 2018 9 minute read

We all saw it coming, and this week, it happened. Apple announced the iOSMac project (a codename — also better known by another codename, Marzipan), although not in the way we were hoping for…

Coming in 2019

…but what did come in the initial beta of macOS Mojave were some real, working, iOSMac apps. News, Stocks, Home, and Voice Memos are all iOS apps ported to macOS and included in the initial beta of Mojave.

Some of this has already been covered by people faster than me, but I’m posting research from the way I approached this.

An effective way to get started investigating new frameworks like this is to review a real use case and work our way through the most obvious things. We’ll look at the simplest of the initial four apps — Voice Memos.

Starting with the obvious

otool -L shows a list of frameworks the app links against:

$ otool -L /Applications/VoiceMemos.app/Contents/MacOS/VoiceMemos
VoiceMemos:
	/System/Library/Frameworks/AVFoundation.framework
	/System/Library/Frameworks/CloudKit.framework
	/System/Library/Frameworks/CoreData.framework
	/System/Library/Frameworks/CoreFoundation.framework
	/System/Library/Frameworks/CoreGraphics.framework
	/System/Library/Frameworks/CoreMedia.framework
	/System/Library/Frameworks/CoreSpotlight.framework
	/System/Library/Frameworks/Foundation.framework
	/System/Library/Frameworks/QuartzCore.framework
	/System/Library/PrivateFrameworks/AppSupport.framework
	/System/Library/PrivateFrameworks/FrontBoardServices.framework
	/System/Library/PrivateFrameworks/MediaRemote.framework
	/System/Library/PrivateFrameworks/MediaServices.framework
	/System/iOSSupport/System/Library/Frameworks/UIKit.framework
	/System/iOSSupport/System/Library/PrivateFrameworks/NetAppsUtilitiesUI.framework
	/System/iOSSupport/System/Library/PrivateFrameworks/VoiceMemos.framework
	/usr/lib/libSystem.B.dylib
	/usr/lib/libobjc.A.dylib

(Output shortened so it’s easier to read on smaller screens.)

Interesting! While it’s linked against many of macOS’s core frameworks, there’s also some new iOS-alike frameworks found in /System/iOSSupport, and some of iOS’s core frameworks found in /System/Library/PrivateFrameworks.

Particularly of interest is that it links against FrontBoardServices, the client framework to the FrontBoard system of iOS. With the release of iOS 8, Apple began splitting out parts of SpringBoard — the “system app”, in FrontBoard nomenclature — so that different user interfaces and behaviors can be used for the growing array of iOS-based products such as Apple Watch, Apple TV, MacBook Pro Touch Bar, and HomePod. iOS still uses SpringBoard, watchOS uses Carousel, etc. The frameworks are identical across all of these platforms — particularly they all use the UIKit framework. This includes watchOS, where Apple has decided to keep UIKit private, exposing some of it through a rather limited wrapper framework (WatchKit). To see an iOSMac app linking against FrontBoardServices gives a clear indication that we’re very likely dealing with a one-to-one API-compatible system for iOS on top of macOS, with zero overhead such as running iOS in a separate subsystem from macOS.

Launching the Voice Memos app and checking the task list reveals a few processes running:

  • VoiceMemos.app (/Applications/VoiceMemos.app/Contents/MacOS/VoiceMemos)
  • voicememod (/System/iOSSupport/System/Library/PrivateFrameworks/VoiceMemos.framework/Support/voicememod)
  • UIKitSystem.app (/System/Library/CoreServices/UIKitSystem.app/Contents/MacOS/UIKitSystem system_app_start)
  • UIKitHostApp.xpc (/System/Library/PrivateFrameworks/UIKitHostAppServices.framework/Versions/A/XPCServices/UIKitHostApp.xpc/Contents/MacOS/UIKitHostApp)

The first two seem obvious — the app itself, and a daemon that does whatever necessary background work. Comparing to a jailbroken iPhone, we can see the same voicememod running when the Voice Memos app is in use.

The next two are what we’re interested in. We’ve very likely confirmed that there is a FrontBoard system app, named UIKitSystem. We also see the XPC agent UIKitHostApp, which we’ll delve into shortly.

UIKitSystem

$ /System/Library/CoreServices/UIKitSystem.app/Contents/MacOS/UIKitSystem
UIKitSystemApp is the system app for iosmac applications. It cannot be started directly.

Well, that solves that mystery, but that’s no fun. Let’s see what it links against:

$ otool -L /System/Library/CoreServices/UIKitSystem.app/Contents/MacOS/UIKitSystem
UIKitSystem:
	/System/Library/CoreServices/UIKitSystem.app/Contents/Frameworks/UIKitSystemAppCore.framework
	/System/Library/Frameworks/CoreFoundation.framework
	/System/Library/Frameworks/CoreServices.framework
	/System/Library/Frameworks/Foundation.framework
	/System/Library/Frameworks/QuartzCore.framework
	/System/Library/PrivateFrameworks/AssertionServices.framework
	/System/Library/PrivateFrameworks/BackBoardServices.framework
	/System/Library/PrivateFrameworks/BaseBoard.framework
	/System/Library/PrivateFrameworks/FrontBoardServices.framework
	/System/Library/PrivateFrameworks/Swift/libswift….dylib (etc, etc)
	/System/Library/PrivateFrameworks/UIKitSystemAppServices.framework
	/System/iOSSupport/System/Library/Frameworks/UIKit.framework
	/System/iOSSupport/System/Library/PrivateFrameworks/AccessibilityPlatformTranslation.framework
	/System/iOSSupport/System/Library/PrivateFrameworks/FrontBoard.framework
	/System/iOSSupport/System/Library/PrivateFrameworks/UIKitCore.framework
	/System/iOSSupport/System/Library/PrivateFrameworks/UIKitServices.framework
	/usr/lib/libAccessibility.dylib (compatibility version 1.0.0, current version 1.0.0)
	/usr/lib/libSystem.B.dylib
	/usr/lib/libobjc.A.dylib

It links against FrontBoard.framework! We are definitely looking at the system app here. This is a little different, though, because there’s no “home screen” or other UI we can necessarily associate with it. The key feature of iOSMac is that the user never needs to know they’re using an app that’s “different” from traditional Mac apps built with AppKit — much like the early days of Mac OS X with the Carbon framework that provided the ability for classic Mac OS apps to run on the new, completely different OS. One of the roles of the system app on iOS, watchOS, and tvOS is to be the “host” of the window (“context”) of apps. This allows it to do fun things such as displaying live-updating snapshots of apps in the app switcher, support split-screen on iPad and the gesture-based switcher on iPhone X, and so on. (You’ll see why this is relevant shortly.)

It’s also interesting to observe that it links against libswift. Apple has slowly ramped up its usage of Swift internally since iOS 10 and macOS 10.12, and the promise that Swift’s interface will be standardised with Swift 5 is a sure sign that Apple will continue using Swift.

Finally, this is a launch agent, just as SpringBoard is on iOS. You can find its plist at /System/Library/LaunchAgents/com.apple.uikitsystemapp.plist.

<plist>
<dict>
	<key>Label</key>
	<string>com.apple.uikitsystemapp</string>
	<key>MachServices</key>
	<dict>
		<key>PurpleSystemAppPort</key>
		<dict>
			<key>ResetAtClose</key>
			<true/>
		</dict>
		<key>com.apple.frontboard.systemappservices</key>
		<true/>
		<key>com.apple.frontboard.workspace</key>
		<true/>
		<key>com.apple.uikitsystemapp.services</key>
		<true/>
		<key>com.apple.UIKit.KeyboardManagement.hosted</key>
		<true/>
		<key>com.apple.UIKit.statusbarserver</key>
		<true/>
	</dict>
	<key>ProgramArguments</key>
	<array>
		<string>/System/Library/CoreServices/UIKitSystem.app/Contents/MacOS/UIKitSystem</string>
		<string>system_app_start</string>
	</array>
</dict>
</plist>

(Some bits I’m not covering removed for brevity.)

Another correlation we can find here is that each of these except for com.apple.uikitsystemapp.services can be found in the launch daemon plist for SpringBoard. Here is a shortened version of the plist from on iOS 10.3:

<plist>
<dict>
	<key>Label</key>
	<string>com.apple.SpringBoard</string>
	<key>MachServices</key>
	<dict>
		<key>PurpleSystemAppPort</key>
		<dict>
			<key>ResetAtClose</key>
			<true/>
		</dict><key>com.apple.UIKit.KeyboardManagement.hosted</key>
		<true/>
		<key>com.apple.UIKit.statusbarserver</key>
		<true/><key>com.apple.frontboard.systemappservices</key>
		<true/>
		<key>com.apple.frontboard.workspace</key>
		<true/></dict>
	<key>KeepAlive</key>
	<true/>
	<key>RunAtLoad</key>
	<true/>
</dict>
</plist>

One difference we can see is that SpringBoard has RunAtLoad and KeepAlive enabled. While iOS truly needs SpringBoard for your iPhone to be useful at all, UIKitSystem only needs to be launched and stay running for at least the duration of an iOSMac app. Hence, it won’t start till an iOSMac app is launched, and may be terminated in low memory conditions as long as no iOSMac apps have been used for a while.

UIKitHostApp

Since iOSMac apps intend to not seem obviously different from an AppKit app, forcing them to live in a separate “world” with its own home screen and app switching functionality — like the iOS Simulator — would be a dealbreaker. Indeed, at the WWDC keynote the four iOSMac apps were demoed and there was no indication that they were anything other than typical AppKit apps until iOSMac was announced later in the keynote.

Craig announces macOS News app

How Apple has tackled this is intriguing — displaying of windows in fact seems to not be handled by the app at all, rather being offloaded to UIKitHostApp. In fact, launching an iOSMac app from the command line reveals that the app is actually UIKitHostApp disguising itself as the app you launched:

UIKitHostApp.xpc in the Dock

When launching the app through LaunchServices (Finder, Dock, Launchpad, etc.), it correctly gets named “Voice Memos” in the Dock.

Peeking at UIKitHostApp’s entitlements — a plist embedded in most modern Mach-O binaries, used by Apple to control the use of risky features — gives it away:

$ ldid -e /System/Library/PrivateFrameworks/UIKitHostAppServices.framework/Versions/A/XPCServices/UIKitHostApp.xpc/Contents/MacOS/UIKitHostApp
<plist>
<dict>
	<key>applicationservices.allowedtowrapanotherprocess</key>
	<true/>
	<key>com.apple.authkit.client.private</key>
	<true/>
	<key>com.apple.private.defaults-impersonate</key>
	<true/>
	<key>com.apple.uikitsystemapp.bundlehost</key>
	<true/>
	<key>com.apple.uikitsystemapp.client</key>
	<true/>
</dict>
</plist>

applicationservices.allowedtowrapanotherprocess likely allows UIKitHostApp to take the role of displaying windows on behalf of other apps.

As an interesting test, you can confirm this by sending pause and resume signals to the Voice Memos and UIKitHostApp processes:

When killall -STOP VoiceMemos is executed, the app interface becomes unresponsive. This is expected as events (clicks, typing, etc.) must be delivered to the app — the app is what’s in control of how its interface works. Notice that the title bar reacts to the window coming into focus, and the traffic lights also react, just like normal. Now resume Voice Memos with killall -CONT VoiceMemos, and pause UIKitHostApp with killall -STOP UIKitHostApp. Unfortunately the QuickTime screen recorder doesn’t catch the wait cursor (the beachball), but in this situation the app is instantly unresponsive to all events, including in the title bar area. macOS notices that the app didn’t respond to the “come into focus” event and displays the wait cursor. This subtle difference makes it pretty obvious what’s going on — the UI is rendered by the Voice Memos app, and displayed by UIKitHostApp.

One interesting difference from AppKit apps is that iOSMac apps simply do not display at all when GPU acceleration isn’t present, such as when running macOS in VMware or VirtualBox, as noticed by @zhuowei:

Ok, but how can I make an iOSMac app?

Early on, the community realised making an iOSMac app of their own isn’t as straightforward as they wished. Looking at the entitlements of the Voice Memos binary, we see a few interesting private entitlements:

$ ldid -e /Applications/VoiceMemos.app/Contents/MacOS/VoiceMemos
<plist>
<dict>
	<key>com.apple.QuartzCore.secure-mode</key>
	<true/>
	<key>com.apple.UIKit.vends-view-services</key>
	<true/>
	<key>com.apple.private.iosmac</key>
	<true/>
	<key>com.apple.private.mobileinstall.xpc-services-enabled</key>
	<true/>
	<key>com.apple.private.security.container-required</key>
	<true/>
	<key>com.apple.private.security.system-application</key>
	<true/>
	<key>com.apple.private.tcc.allow</key>
	<array>
		<string>kTCCServiceMicrophone</string>
	</array>
	<key>platform-application</key>
	<true/>
</dict>
</plist>

(Again, cut down for brevity.)

The one that sticks out is com.apple.private.iosmac. It’s not possible to execute a binary using this entitlement, even using a certificate obtained via a paid Apple developer account, unless the binary is signed by Apple or distributed through the App Store (where Apple carefully scrutinizes app entitlements). The iOSMac system indeed checks for this and won’t let you get by without it:

@zhuowei, @biscuitehh, @HamzaSood have all found different ways to work around this, each involving some sort of workaround to sidestep the restrictions globally (regardless of the entitlement and regardless of the app being used — so a fair security risk).

By disabling System Integrity Protection, setting the boot argument amfi_get_out_of_my_way=0x1 (AMFI is AppleMobileFileIntegrity), and rebooting the Mac, I was able to get as far as seeing the initial interface of my iOS terminal app NewTerm, unfortunately with an error indicating forking failed due to lack of permission:

Fortunately, explicitly setting the “App Sandbox” entitlement to false allowed it to successfully fork and execute my shell:

(For comparison, see the NewTerm homepage for how the app looks on iOS.)

I’ve poked around trying different ideas to make NewTerm work, and none have made a difference (in fact I occasionally made it worse, causing a window server crash before the window even shows up…), so I’m going to leave it here for now and see if any of these issues improve in the coming macOS Mojave betas.

You can see my kinda-sorta working demo of NewTerm running under iOSMac at github.com/kirb/iOSMac-Demo.

iOSMac is definitely not ready for primetime — that should have been obvious when Apple indicated this is only “phase one”. It is very possible for an iOSMac app to crash the window server, causing all of your running apps to be terminated and kicking you back to the login window. Sometimes you get even more unlucky and the window server hangs, forcing you to either use another device to SSH into your Mac and killall UIKitSystem, or simply hold down the power button. Such is beta life — I’d be certain this will be heavily improved in the coming betas, because these issues could certainly arise while using Apple’s own iOSMac apps. There are many frameworks missing; it’s rather clear that Apple has only included frameworks used by the four apps, plus a few extras whose API/ABI map one-to-one with macOS frameworks. Still, a great demo for now, fascinating how closely related it is to true iOS, and it’s amazing that its release to developers is only a year from now!