Look Back With Creepy Eyes

I guess there was an anniversary for the Apple Vision Pro a while ago, because I kept seeing people doing retrospectives on it, and I figured I should probably write about my own experiences. If only because I spent so much time leading up to the launch writing about how hyped I was, but then never did a follow-up after the honeymoon period was over.

But that idea, like the headset itself, went back to sitting unused on a shelf.

Even though I almost never use it, I don’t regret getting it, and I don’t have any real intention of trying to sell it.1And if any early adopters are still considering selling theirs: good luck with that! It sure feels like all the interest dried up not long after the initial hype did. With all my back-and-forth trying to talk myself into getting one, the thing that finally pushed me over the edge was simply owning the device itself. Like the Apple Watch “series 0,” the first iPad, the first iPhone, one of the original iPods, and the used Macintosh SE I’ve got sitting on my desk, I like having it as a kind of landmark of consumer technology history more than for its practical utility. In other words: I still just think it looks neat.

I still think that a lot of the negative reviews early on had too much of a We were promised jetpacks! vibe to them. There’s still an amazing amount of groundbreaking technology crammed into the headset, all in service of trying to make the experience of using it as frictionless and approachable as possible, and it still seems short-sighted to be dismissive of all that to focus on frivolous complaints.2Such as, of course, the complaint that it messes up your hair. Essentially complaining that it’s only 10 years ahead of its time instead of 20.

Ultimately, pretty much everything I like and dislike about the Vision Pro is a result of one key decision Apple made when positioning it as a product: they want it to be a “lifestyle” device instead of an “enthusiast” one.

There’s nothing surprising about that, and it was pretty evident even before the device launched. The Vision Pro in its first incarnation feels like something that could only be made by Apple, because of the industrial design, the focus on interconnected devices, vertical integration, tight connection between hardware and software, and the years spent advancing and refining the state of ARKit on iOS devices. But that also means that Apple would need the device to sell at iPad or Mac levels to make any sense as an Apple product, and it’s probably impossible to get those numbers from being limited just to gamers or fans of the Quest.3And Apple just doesn’t sell hardware at a loss, which Facebook is more willing to do.

What did surprise me was how much that one decision rippled out into everything else, and how much it’s keeping the platform from feeling like it’ll find its audience.

The Anti-VR VR Club

For starters, there’s such a strong mandate to avoid all of the things that keep normal people from getting into VR: the hassle of setup, the disorientation of being in a virtual environment, and the feeling of isolation from wearing a headset.

So you start by seeing your actual environment4After the initial setup process, of course, and you use hand gestures for everything. No trying to find your way around or re-orient the room, no worrying about tripping over obstacles, no process of setting up a room boundary, or finding where you left the controllers, or making sure the controllers are charged up. Most of that is good and necessary. It’s just that it feels like they took the idea so far that it almost became reactionary.

Most obvious is the front display showing your eyes, to simulate having actual eye contact with someone else in the room. It made such a bad first impression in the marketing material that it felt tone deaf, does no one at Apple realize how creepy this is?! But it’s been used so often as part of the “brand” of the Vision Pro that now it feels almost arrogant. I appreciate that it’s part of Apple’s philosophy to take big swings and solve unsolvable problems, but this just feels like it backfired. And it’s become so associated with the platform, if not just this initial device, that it doesn’t seem like it will quickly be forgotten as another case of “well-intentioned attempts to humanize tech,” like the sharing-your-heartbeat feature from the Apple Watch.

(And not to keep harping on it unnecessarily, but even with all the system updates, I still can’t get it to make a good “Persona” for me. Once again, I’m baffled as to how a tech company founded in Silicon Valley in the 1970s has failed to accommodate an old man with a big white beard).

Indirect Control

But for me, it’s been other things that fall out of that decision that make the Vision Pro a lot less interesting for me. The system always prioritizes your real-world scene over virtual elements, meaning that when it detects your hand or another real-world object colliding with something virtual, it’ll fade out the virtual object.

This means that almost all of your interaction with the device is indirect: you look at something from some distance away, and you use hand gestures to interact with it. Most of the app ideas that I’d had, and the types of games and apps that I’d been looking forward to, get their appeal from being able to interact with objects directly. After all, using my hand on a mouse to indirectly manipulate things on a distant screen is 90% of how I already interact with computers.5Largely thanks to Apple in the first place. Why would I go through the hassle of putting on a headset just to get the same experience, with the only difference that I’ve got a larger distant screen?

I have just started trying an app called Simply Piano that can project tutorial info either directly onto your desk, or as an overlay on a real-world piano or keyboard. So far, this seems like an ideal use case for the headset, but the pure-virtual keyboard seems to have a problem figuring out exactly which key I’m pressing. I’ll have to see if it has the same issue with a real keyboard.

That kind of direct interaction seems to be a rarity, though. I’d been looking forward to the Game Room app on the Vision Pro, just as a no-brainer demonstration of how tactile and immersive a board or card game could feel in a virtual environment. So it was a big disappointment to try it out and learn that it does pretty much everything possible to keep you from being able to touch any of the game components.

App Development

I’ve done just enough development with the Vision Pro to know that it supports multiple modes of interaction, the two key ones being self-contained 3D scenes you interact with from a distance, and “immersive” scenes that you can in theory interact with directly. “In theory” because from what I’ve seen, there’s still a system-level mandate to avoid all of the kinds of things that make room-scale VR possible, in favor of completely stationary (standing or seated) experiences.

The development tools are still evolving, but at least as of the last time I checked, Apple had made it much more straightforward to make self-contained 3D experiences and windowed apps than fully immersive ones. I get too quickly frustrated with SwiftUI to have spent a lot of time with it6Everything reminds me too much of CSS and web development, but even I can tell that they’ve put a ton of effort into making the skills from iOS and iPad development carry over seamlessly to the Vision platform.

For more complicated 3D scenes, and especially fully immersive ones, I have a hard time imagining how to pull that together without a dedicated game engine. Apple’s Reality Composer Pro does a lot for a tool included for free with Xcode, but it still feels pretty bare-bones for someone who’s used to Unity, Unreal, or even Godot. It seems like it would’ve been a better move to make an easier pipeline for devs to use the tools they’re already comfortable with, so you really could just choose the headset as another build target.

And in the most frustrating move for somebody in my position (as a hobbyist developer who works in game engines at his day job), the only option for using Unity is with an expensive professional license. I can completely understand why Unity isn’t supporting the free version7Possibly the only place where I can see Unity’s licensing and pricing making sense?, since it’s a ton of work to support a platform that hasn’t proven itself to give a good return on investment. But it does mean a much larger barrier to entry for hobbyists and indie developers to just try something goofy for the Vision Pro to see if it takes off.

I’d been hoping that Apple would just acquire an existing game engine and work on giving it “official” support for the Vision platform, but the window of opportunity for that seems to have passed.

All of this means that the majority of the available apps for the Vision platform are going to be “windowed” or based directly on iPad or iOS apps, using 3D sparingly if at all. That’s even in Apple’s recommended app design guidelines, to use 3D sparingly. It feels very frustrating to me, because 3D scenes and immersive environments are the biggest draw for putting on a headset in the first place. Why am I strapping a computer to my face just to keep using windows?

Big Screens

I installed the 2.5 version of the OS before I started writing this, to check out how things have progressed since the last time I used the Vision Pro. And I’ve been writing a lot of this blog post in a big virtual desktop screen mirroring my MacBook’s. It’s neat, having a giant Mac desktop that fills up an entire wall. But it goes back to that same question: is it neat enough to warrant strapping a computer to my face, instead of just using my monitor?

So far, the best use of the Vision Pro for me has been (unsurprisingly) to watch movies. But even there, there’s only a subset of movies and media that make sense to strap on the headset to watch. It’s got to be something that I either don’t mind watching alone or prefer to, and it’s got to be something that really benefits from a huge screen. I avoided seeing Dune 2 and Furiosa in theaters specifically for the opportunity to watch them at home alone on a huge 3D screen.

Of course, I haven’t actually watched either one of those yet, since there never seems to be a good opportunity to sit by myself in a room for 2+ hours with a computer strapped to my face. But I paid for the privilege of knowing that I could.

And it’s not really related to the design of the headset or the philosophy of the platform, but still: even if you are in the mood to strap a screen and start watching, there’s no guarantee it’ll be available. YouTube and Netflix still don’t have official apps, and watching them in Safari isn’t great. And with the fragmented state of streaming these days, it’s more likely that the weird thing I’m in a sudden mood to watch — like when I spontaneously decided I had to see Black Sunday — has a fairly slim chance of being available.

Recent Additions

Apple has recognized that media is probably the most obvious killer feature of the platform, though. They’ve released quite a few entires in their “Immersive” series, at this point including the expected documentaries along with some concert movies and one fictional movie that I know of.

The Photos app on the Vision Pro now lets you generate a “spatial” version of most photos in your library, even if they weren’t taken with a 3D camera or with that mode on the iPhone Pro models.8I think it requires depth information, but I haven’t experimented to be sure. It works surprisingly well, and it’s a neat encouragement to look through photos on the headset, even if I’m not confident that the novelty will last long. Viewing panorama photos with the scene stretched around your room remains fantastic, though, and might be the biggest bang for the buck on the whole device.

They’ve also added a dedicated Spatial Gallery app, which has an updated set of 3D photos and short video clips, at least a third of which seem to be extended ads for Apple TV programming, in the form of behind-the-scenes photos. It’s obviously not something that warrants getting an expensive headset, but it’s a good sign that Apple at least recognizes that the people who bought an expensive headset are eager to have more reasons to use it.

I was prepared that the Vision Pro would become something that just sat around collecting dust apart from watching the occasional movie; I just expected that I’d have gotten more development-experimentation time out of it before then. Esoteric technical limitations9Including the fact that the amazing image tracking supported on the iPhone is limited to 1Hz updates on the Vision Pro, specialized development tools that encourage simple windowed apps, and opinionated decisions at the platform level, have all thrown a bucket of cold water on my goal of making simple games and goofy special-purpose apps for the headset.

The stuff that it does well, it still does shockingly well. When I put it on last night after such a long period of not using it, I was stunned all over again by just how good the gesture recognition and gaze detection are. At this stage, though, it just seems like an astonishing amount of groundbreaking technical work was applied to the most mundane parts of the experience, like moving windows around and mirroring your desktop. The show-stopping experiences and things you can only do on the Vision Pro platform don’t seem to have materialized yet.

1
And if any early adopters are still considering selling theirs: good luck with that! It sure feels like all the interest dried up not long after the initial hype did.

2
Such as, of course, the complaint that it messes up your hair.

3
And Apple just doesn’t sell hardware at a loss, which Facebook is more willing to do.

4
After the initial setup process, of course

5
Largely thanks to Apple in the first place.

6
Everything reminds me too much of CSS and web development

7
Possibly the only place where I can see Unity’s licensing and pricing making sense?

8
I think it requires depth information, but I haven’t experimented to be sure.

9
Including the fact that the amazing image tracking supported on the iPhone is limited to 1Hz updates on the Vision Pro

1 And if any early adopters are still considering selling theirs: good luck with that! It sure feels like all the interest dried up not long after the initial hype did.

2 Such as, of course, the complaint that it messes up your hair.

3 And Apple just doesn’t sell hardware at a loss, which Facebook is more willing to do.

4 After the initial setup process, of course

5 Largely thanks to Apple in the first place.

6 Everything reminds me too much of CSS and web development

7 Possibly the only place where I can see Unity’s licensing and pricing making sense?

8 I think it requires depth information, but I haven’t experimented to be sure.

9 Including the fact that the amazing image tracking supported on the iPhone is limited to 1Hz updates on the Vision Pro