Experiencing the Journey of My Life in iPhone Spatial Video with Apple Vision Pro

I saw Apple’s Vision Pro for a third time in 2023. This time, I wasn’t looking at Avatar, or Apple’s test photos or videos. I was looking at my mom and sister at Thanksgiving, and my kids playing guitar and piano with my nephew, and me running after my son at the Museum of Natural History into a room full of luminescent rocks.

Apple updated iOS 17.2 this week, adding the ability to record 3D “spatial” videos if you have an iPhone 15 Pro. You won’t get a chance to see what those 3D videos look like on a Vision Pro headset until next year, but I did. Let me tell you how it went.

I already got to look at Apple’s own spatial video samples a few weeks ago, and even test-recorded one of my own. I was impressed back then, in part by the quality of the video recordings and partly by the Vision Pro’s excellent display. My second time around had its own level of uncanniness; this was stuff from my own life, my own memories. I was playing back experiences I had already lived.

Despite the “spatial” name, these videos are stereoscopic 3D, not full scenes you can move around in. I can’t get closer to these recorded memories or view them from the side, but there were moments that shined and others that showed me where spatial recording and playback have limits. I took dozens of 3D clips but only got a chance to play back a handful.

Dropping into my own life

My photos and videos were AirDropped to a phone that had an account linked to the Vision Pro for my demo, but the cool part is that anyone who has a Vision Pro will already have their photo library – and everything else in Apple’s ecosystem – synced. It gave me a weird sense of familiar surprise to glance down and tap my fingers to accept an AirDropped set of files in a Vision Pro headset.

I looked at 10 of my own spatial video samples from dozens I shot, plus a couple of panoramic photos from Storm King Arts Center and southwest England, and some HDR photos of my family in the Meadowlands at DeKorte Park. The photos looked great, easily as good as or better than on my Mac or a TV. HDR really popped, and it was fun to zoom in big. As I said the last time, panoramic photos have their own immersive appeal when viewed in the headset, and the display resolution helps photos shine.

Spatial videos have their own specific framing, in a fuzzy border that helps mask that the video isn’t quite as screen-filling as you might expect. I noticed this more during my second demo. Regular non-spatial videos can be enlarged by pinching and zooming, but not 3D spatial ones. I wish they could be, just a little bit.

Apple has its own recording recommendations for spatial video, suggesting not to move too much and stay in well-lit areas. I broke a lot of those rules, floating around and sometimes going into dim spaces (a dark bar with CNET colleagues, or the museum and its glow-in-the-dark rocks). The results still worked well, but brighter places pop more.

The 30-frames-per-second frame rate limitation was also visibly apparent, especially with movement. I wish 60 fps were possible. If someone’s running by fast, like my kids as I followed them in Central Park as they climbed a rock next to a pond near The Ramble, that frame rate can sometimes feel too choppy for my tastes in an extremely fast refresh-rate headset like Vision Pro.

Intimate moments shine

The most compelling experiences for me were playing clips where I held the iPhone near my own line of sight with family. In one clip, I’m sitting around a table at Thanksgiving with my mom, telling her I’m recording in 3D for a future project. She sort of raised her eyebrows a bit, and the way she looked at me in 3D – at a scale close to normal size, with her seeming to make eye contact – made me feel like I was almost there. It made me want to climb through that fuzzy-bordered window and join my family again on the other side.

With less motion, the frame rate limit doesn’t stand out as much. The more intimate framing is better suited for the contained nature of the spatial video playback frame, as well.

I also got a kick out of seeing some of my favorite exhibits at the Museum of Natural History in 3D. I went to the Hall of Ocean Life, where a trio of little tanks on the top floor house dioramas of prehistoric sea creatures that I’ve loved since I was a kid. I captured the ammonites and trilobites, gliding from case to case. In 3D on Vision Pro, it felt like a little home diorama revisited. I could see the joy of capturing little immersive installations, or spaces, that aren’t easy to go back to, and having them in front of you again. Given the subject matter, that smaller scale in that fuzzy frame was lovely.

See you again someday, Vision Pro

That was it: my third Vision Pro demo, but brief. It felt like I got to live a tiny bit with some of my own stuff on the headset, a precursor to how the Vision Pro will likely feel for anyone who’s been deep in Apple’s ecosystem already. The Vision Pro will be ready to connect with your own things right out of the box.

The Vision Pro’s display still strikes me as beautiful, although the fit and feel are very much similar to Meta’s Quest headsets. The easy interface is continually refreshing, but I wonder how Apple can improve its spatial video recording and playback even further, allowing for more controls, higher resolutions and faster frame rates. In addition, it would be nice to see it allow for more playback control options down the road.

The Vision Pro is expected to arrive in early 2024, so we’ll know more next year – sooner than later. Hopefully, this becomes a bit of helpful advice as you take spatial videos of your own on an iPhone 15 Pro, not knowing what the final results will look like later.