My Life Unfolded in Immersive iPhone Spatial Video on Apple Vision Pro

I saw Apple’s Vision Pro for a third time in 2023. This time, I wasn’t looking at Avatar, or Apple’s test photos or videos. I was looking at my mom and sister at Thanksgiving, and my kids playing guitar and piano with my nephew, and me running after my son at the Museum of Natural History into a room full of luminescent rocks.

Apple updated iOS 17.2 this week, adding the ability to record 3D “spatial” videos if you have an iPhone 15 Pro. You won’t get a chance to see what those 3D videos look like on a Vision Pro headset until next year, but I did. Let me tell you how it went.

I already got to look at Apple’s own spatial video samples a few weeks ago, and even test-recorded one of my own. I was impressed back then, in part by the quality of the video recordings and partly by the Vision Pro’s excellent display. My second time around had its own level of uncanniness; this was stuff from my own life, my own memories. I was playing back experiences I had already lived.

Despite the “spatial” name, these videos are stereoscopic 3D, not full scenes you can move around in. I can’t get closer to these recorded memories or view them from the side, but there were moments that shined and others that showed me where spatial recording and playback have limits. I took dozens of 3D clips but only got a chance to play back a handful.

My photos and videos were AirDropped to a phone that had an account linked to the Vision Pro for my demo, but the cool part is that anyone who has a Vision Pro will already have their photo library – and everything else in Apple’s ecosystem – synced. It gave me a weird sense of familiar surprise to glance down and tap my fingers to accept an AirDropped set of files in a Vision Pro headset.

I looked at 10 of my own spatial video samples from dozens I shot, plus a couple of panoramic photos from Storm King Arts Center and southwest England, and some HDR photos of my family in the Meadowlands at DeKorte Park. The photos looked great, easily as good as or better than on my Mac or a TV.

Spatial videos have their own specific framing, in a fuzzy border that helps mask that the video isn’t quite as screen-filling as you might expect. I noticed this more during my second demo. Regular non-spatial videos can be enlarged by pinching and zooming, but not 3D spatial ones. I wish they could be, just a little bit.

Apple has its own recording recommendations for spatial video, suggesting not to move too much and stay in well-lit areas. I broke a lot of those rules, floating around and sometimes going into dim spaces (a dark bar with CNET colleagues, or the museum and its glow-in-the-dark rocks). The results still worked well, but brighter places pop more.

The 30-frames-per-second frame rate limitation was also visibly apparent, especially with movement. I wish 60 fps were possible. If someone’s running by fast, like my kids as I followed them in Central Park as they climbed a rock next to a pond near The Ramble, that frame rate can sometimes feel too choppy for my tastes in an extremely fast refresh-rate headset like Vision Pro.

The most compelling experiences for me were playing clips where I held the iPhone near my own line of sight with family. In one clip, I’m sitting around a table at Thanksgiving with my mom, telling her I’m recording in 3D for a future project. She sort of raised her eyebrows a bit, and the way she looked at me in 3D – at a scale close to normal size, with her seeming to make eye contact – made me feel like I was almost there. It made me want to climb through that fuzzy-bordered window and join my family again on the other side.

The Vision Pro is expected to arrive in early 2024, so we’ll know more next year – sooner than later. Hopefully, this becomes a bit of helpful advice as you take spatial videos of your own on an iPhone 15 Pro, not knowing what the final results will look like later.