The Core Magic Behind visionOS's Future Killer Apps

I was recently asked a simple question: Where do you think the future of augmented reality software is headed?

My brain momentarily froze. I am (unhealthily? let’s say: comfortably) obsessed with augmented reality. I often jot down potential AR use cases and spatial computing app ideas to reevaluate or tinker with or launch at some later point. But I rarely distill those myriad, disparate thoughts and concepts into a simple, ordered list of undergirding themes.

Let’s do that now.

I’ll preface this list with: Today’s Apple Vision Pro is not a consumer device. It’s a glorified developer kit. It’s too heavy, too expensive, and too isolating to ever gain any semblance of consumer traction. I’m presupposing, and still anticipating, a future where the Apple Vision product line has exciting, accessible, competitive consumer products for sale. When we arrive at that threshold may be subjective; my guess is that time is 2.5 - 5 years away.

That’s not to say Vision Pro apps haven’t already seen pockets of success with enterprises and early adopters, with more on the way. I recently developed an immersive training app for a large enterprise client – I expect such corporate pilots and rollouts to quietly gain steam where no one can see them: Off the App Store in the bowels of Corporate America.

I’ve always been more interested in consumer apps over enterprise apps, so let’s focus on the former. And the conceptual buckets below aren’t mutually exclusive. And there’s the very real chance there never is a visionOS killer app…but enough hedging.

In drastically increasing levels of coolness:

My Predicted Themes for visionOS Killer Apps

Big Screens

We love our screens. Let’s put a screen in every room, in our pockets, on our wrists, on our laps, and at least two on our desks. Bigger is usually “better” (as determined by display size trends of TVs, monitors, smartwatches, smartphones, movie theater screens, portable game consoles, and car dashboard displays over the years).

But once we put a couple small displays right in front of our eyeballs…do we need all the others? Do I need to buy, unpack, lug around, hook up, and be careful not to destroy a fragile slate? With nothing to say for the manufacturing, assembly, transportation, and logistical resources expended to get that screen from somewhere else to delivered on my doorstep.

Why wouldn’t I just create a virtual screen viewable through my headset, move it around with a gesture of my hand, and scale it up or down at will? Why would I limit myself to a singular, “in the billions of places on Earth, I can only write code in this specific spot because it has my displays and cables and machines set up the way I want” workspace when I can just take that gear with me?

If I work in virtual, floating visionOS windows, my setup is no longer a mess of wires and ports and connectors and power plugs – it’s a config setting saved on my Vision Pro.

I think this is the most obvious use case that is already 50% of the way towards being a killer app in visionOS 2.2, and in fact isn’t a separate, third-party app but rather a feature of the OS itself. The problem is the cost and weight of the hardware needed to run this killer functionality.

This theme is incumbent on Apple to make happen in visionOS. It’s their hardware, their software, their OS. A light, comfortable, social, affordable headset is required here – but I think this use case, built into the OS, is what will eventually lead to more consumers buying future headsets. More consumers with visionOS devices (be they headsets or glasses) will in turn unlock the opportunity for third-party developers to meaningfully monetize consumer visionOS apps.

The ability to freely create and manipulate big virtual screens is an obvious thing people love about their spatial computing headsets today; and many more will discover that love in the future. But creating giant, virtual, floating screens doesn’t utilize any of the capabilities of spatial computing devices that really excite me. Let’s move on to those.

Real-Time Holograms

One of my favorite X-Men growing up was Nightcrawler, primarily because he could teleport. It tracks that one of my personal favorite features of visionOS is Environments. The ability to instantly teleport to a tropical beach or a beautiful mountaintop is different parts gratifying, soothing, and fascinating.

Similarly, having the Persona of someone who lives thousands of miles away join you in your living room as you gather around a meme, or review a mockup, or sketch out an idea on a shared canvas – all while being able to see their gestures and expressions in real-time – is ethereal. These are powerful experiences that are definitive steps above a video chat or changing your device wallpaper.

Whereas visionOS has some related elements built into the OS in the form of Environments, Personas, SharePlay, and FaceTime, I think there are massive opportunities to hack together products that:

1. Teleport users’ holograms (i.e. Personas) somewhere new, old, or impossible
2. Gather multiple users’ Personas in the same spot
3. Beam real-time holograms to users

Let me expand on that last one.

What I want to do is watch the Chicago Bears as tiny, little holograms on my coffee table. I want to be able to rotate and zoom into and pan around the field as I see fit. I want to be able to scale up the holograms and resituate them so I’m watching a play from on the field behind the offensive line. I think we are several years, sensor generations, camera generations, display generations, and machine learning model iterations away from that being possible.

But…I do think it’s coming. Some real-time sports visionOS apps that recreate the action in 3D already exist. It will take time for more real-time 3D recreations (or “real-time holograms,” which is more fun) of sports to materialize. When they do, there will be a contingent of fans, followers, and professionals that prefer to watch the hologram version.

Visualization & Customization

Things I find interesting about augmented reality are things you can’t do on a typical screen. When you’re interacting with an iOS app you’re still fundamentally tapping on a 2D plane – iPhone’s display – to do things. You can’t reach into your iPhone’s screen to grab a virtual beer that just materialized on the table in front of you.

In visionOS you can grab that (virtual) beer, even if you won’t feel it. You can move it around. You can knock it over. You can pick it up and throw it. visionOS and its apps can understand your physical space. This enables you to augment reality – manipulating both real and virtual things in your surroundings.

It’s this ability to visualize customizations to our space that I find immensely intriguing. People love to customize their stuff: their desktops, their homescreens, their Control Center, their playlists, their wardrobe, their homes, their bodies. So if I’m a temporarily embarrassed capitalist, why shouldn’t I be able to look out my window and see a view covetable by a billionaire? It may be virtual and less convincing than reality, but it’s a bit more accessible as it sidesteps that whole “earn billions of dollars first” thing.

Let’s take money out of it: I love Yosemite, but the NIMBYs are preventing new homes from being built smack dab in the middle of Yosemite Valley. With spatial computing I can have a virtual, impossible view of Yosemite from my living room windows. Or a view of The Shire. Or a view of some artist’s soothing, abstract, hallucinatory work.

With spatial computing I can tap a wall to change its color. I can be a goth for a day and change my floors, cabinets, and ceilings to onyx. I can make my IKEA chair take on the virtual appearance of a Herman Miller classic. I can don a Wario moustache on my Persona for a Spatial FaceTime with friends and have a more professional Mario moustache thirty minutes later for a business call with a client.

There are tons of hurdles to overcome (hardware, cost, software, third-party support), but once we get there I expect more people to be obsessed with spatial computing the way I am due to:

• Big virtual screens that facilitate existing content and habits
• Real-time holograms that facilitate new experiences and engaging social interactions
• Customization that facilitates people’s desires to change the world around them