Back in March, while everyone was waiting for the new iPad with bated breath, we heard rumblings of a 3rd-generation Apple TV. We’re always trying to find new ways to wow our users, and creating a dual-screen AirPlay experience with Apple TV seemed like a great opportunity to do just that. In the last major release of our iOS app, we added a new feature that allows users who own an Apple TV and an iOS device to view full-screen slideshows of homes.
Hold on…what is AirPlay exactly?
AirPlay is Apple’s proprietary protocol stack for wireless streaming of media. Since this is a developer blog, we’ll cut through the marketing-speak for you and cut right to the heart of what AirPlay does.
Streaming audio/video – On iOS, applications can stream audio or video to an AirPlay-enabled receiving device like an Apple TV or AirPort Express with connected speakers (audio only, of course). This feature has been available to third-party apps since version 4.3. Most apps that utilize AirPlay fall into this category. Computers running iTunes have this capability as well.
Screen Mirroring – Screen mirroring has been iOS’s confusing little secret since iOS 5 arrived last year. This feature allows second and third-generation iPads and the iPhone 4S to stream content wirelessly to an Apple TV. As the name implies, you can mirror what appears on your iOS device, which makes sense given the definition of the verb to mirror. Enabling screen mirroring on a device also allows apps to send an entirely different view. To alleviate confusion when discussing this feature, we use the term “screen mirroring” to refer to the use of the Apple TV as a second display and “actual screen mirroring” to refer to displaying the same display on both the iOS device and the Apple TV. Needless to say, there will still be some confusion.
As a real estate technology company, we have a lot more pretty pictures than streaming media lying around. AirPlay’s streaming capabilities are well documented and not useful to us. Despite being introduced in the second-generation Apple TV, screen mirroring is considerably less documented. We hope the lessons we learned developing a screen mirroring experience for iOS prove useful to others.
The Design
Imagine that you’re sitting at home on your couch and flipping through homes on Redfin on your iPad. While your significant other is cooking in the kitchen, you find a drop dead gorgeous home in your price range in the perfect neighborhood and want to share it from across the room.
This is the scenario we designed for. It’s useful to remember how different this medium is from the iPhone or iPad. Your Apple TV display cannot detect touch gestures, is considerably larger than an iPad (size-wise, probably not in terms of resolution), and is typically viewed from at least a few feet away. As users, we’re very used to reading on smartphones, tablets, and computers, but TV is what we go to when we need to get away from all those words. Minimize the amount of text to display. When you do display text, make sure it’s BIG.
Our AirPlay slideshow has two features that supplement our app:
- When a user a selects a home, the Apple TV display will play an automated slideshow of the home’s photos.
- When a user enters the home’s photo gallery, the slideshow will sync with the photo that the user has selected.
Having screen mirroring enabled in most apps will perform “actual screen mirroring” on the Apple TV. To make it clear to the user to expect a different experience right from the start, we display a splash screen on the external display with instructions on how to start a slideshow. We also give the option to disable the slideshow in case the user prefers to see the same screen in a larger format.
The Code
Apple’s documentation on displaying content on an external display is very thorough and should be your primary source for developing a dual-screen experience. This post focuses on some of the lessons we learned that we haven’t found documented elsewhere. It is assumed you have some experience with developing iOS apps in Objective-C.
Tip #1: There is no way to detect AirPlay capabilities for a device.
When screen mirroring is enabled and an app is opened, it simply appears in the SDK as a second UIScreen. It is not possible to distinguish between a display connected via HDMI out and one connected via AirPlay.
Apple recommends detecting capabilities instead of determining the device type as it helps makes apps future-proof for new devices. Unfortunately, it isn’t possible to detect potential AirPlay screen mirroring displays if one is not currently enabled. The table below shows which iOS devices support external displays.
| Devices | HDMI-out | Screen Mirroring |
|---|---|---|
| iPod Touch (All Generations) | No | No |
| iPhone 3GS and earlier | No | No |
| iPhone 4 | Yes | No |
| iPhone 4S | Yes | Yes |
| iPad (1st Gen.) | Yes | No |
| iPad (2nd Gen.) | Yes | Yes |
| iPad (3rd Gen.) | Yes | Yes |
A helpful Stack Overflow user told us about Erica Sadun’s UIDevice extensions that abstracts some of the device detection code for you. We don’t use it in our code as we discovered it a little late in the game.
We support the HDMI-only devices but display different instructions as not to mislead users into thinking their devices can do something they cannot.
Tip #2: When an external screen is detected, create a view controller specifically for that screen
Chances are that only a small percentage of your app’s users have external display capabilities. You shouldn’t increase the complexity of your code to handle such a small case.
Handle the display of the external display in a single controller to keep this functionality separate from the rest of your codebase.
|
1
2
3
4
5
6
7
8
9
10
11
12
13
|
UIScreen *externalScreen = [[UIScreen screens] objectAtIndex:1];UIWindow *externalWindow = [[UIWindow alloc] initWithFrame:externalScreen.bounds];externalWindow.screen = externalScreen;// This is a custom constructor that takes in the window size and uses// it to size the controller's view on load. Accessing the view's// window size on load didn't work for us.YourExternalDisplayViewController *externalViewController = [[YourExternalDisplayViewController alloc] initWithFrame:externalWindow.frame];externalWindow.rootViewController = externalViewController;externalWindow.hidden = NO; |
Tip #3: Use NSNotificationCenter to sync events between your external view’s controller and the rest of your app
The other controllers in your application do not need to call the external view’s controller directly. Instead, post a notification when you’d like make a change to your external view. If there is no external display, the notifications will float off into the ether. If the controller exists, it will listen for the notification and handle it appropriately.
In your external view controller:
|
1
2
3
4
5
6
7
|
-(void)viewDidAppear { [super viewDidAppear]; [[NSNotificationCenter defaultCenter] addObserver:self selector:@selector(handleSyncSlideshowNotification:) name:@”SyncSlideshowNotification” object:nil];} |
In your main display’s view controller:
|
1
2
3
4
|
[[NSNotificationCenter defaultCenter] postNotificationName:@”SyncSlideshowNotification” object:nil userInfo:yourParameters]; // as (NSDictionary*) |
You can run into race conditions using notifications if you’re not careful. Still, we feel this is the right way to keep this feature from turning your code into a rat’s nest.
Tip #4: Don’t use a XIB file to create your view
Once you leave the walled garden of developing views for iOS devices, you’re dealing with a wide variety of resolutions. XIB files are convenient when creating a view of fixed size. We recommend creating and positioning your subviews in your viewDidLoad method according to the size of your external UIWindow.
Tip #5: No one knows about screen mirroring. Make it as obvious as possible how to turn it on.
For such a cool feature, it is somewhat astounding how few people know that it exists or how to turn it on. When we announced screen mirroring on our blog, one of the first comments in the post was asking how to enable it. And we don’t blame these users in the slightest. There’s no way to turn screen mirroring on from within an app, but Apple provides instructions to turn it on. As it turns out, this setting is located in one of the most inconspicuous spots in the entire operating system. After telling one of our friends at Apple about the new feature, he promptly emailed us back asking how to turn it on. This is one area where Apple has let down their users and developers. We hope to make up for it by giving users more instruction on how to enable screen mirroring in the next release of our own app.
Those are some of the lessons we learned developing our own screen mirroring experience for AirPlay. If you have any tips of your own, please leave a comment below.
United States
Canada