Technologies

Apple iPhone Spatial Video Arrives in Beta and Looks Amazing on Vision Pro

I tried the new iPhone 15 Pro camera feature on Apple’s upcoming AR-VR headset. The 3D is undeniably vivid.

I’m looking at a plate of sushi hovering in front of me in 3D. The chef finishes off toppings on yellowtail rolls and tuna, talking to me as she works. It looks vivid. It looks real. The amazing part is that I just shot this video myself, moments earlier, on an iPhone 15 Pro. And now it’s a VR experience I’m watching in beautiful 3D on Apple’s Vision Pro headset.

I swipe with my fingers and watch other ghostly videos Apple provided. Families in a home, walking through grass. Cuddling together. All in lifelike 3D. I feel like I’m peeking in on their lives, which is weird and intimate. But the vividness is undeniable.

I’m experiencing Apple’s new Spatial Video iPhone camera feature, now available in Apple’s new iOS 17.2 public beta, with the final version rolling out in 17.2 later this year. It allows you to record 3D videos, but to use it you’ll need Apple’s best phone, the $1,000 iPhone 15 Pro. And to watch the videos in 3D you’ll need the $3,500 Vision Pro headset, which launches early next year.

The videos look great and the 3D is compellingly realistic. They’re also easy to record, and can save as videos that will play back in 2D in a normal video format. But ultimately this feature is made for a 2024 product that, at $3,500, it’s safe to say most people won’t buy anytime soon. Still, the experience is impressive.

Vision Pro impressed me once again

I first used the Vision Pro at Apple’s WWDC conference in June, and once again I’m reminded of its extraordinary display capabilities. Photos and video look fantastic on Apple’s headset. I’d even venture to say the Vision Pro beats any display I have anywhere in my home right now. Yes, I’d want to look at videos and photos — 3D and otherwise — if I had a Vision Pro.

My second dive into the Vision Pro, complete with fitted prescription lenses that matched my needs, reminds me how much smaller the headset was than I remembered. And also, how effortless the interface is.

Setting up eye tracking was a rapid process; I looked at dots around a circle and tapped them. I also tried a new set of zoom gestures, pinching my fingers and stretching them to expand photos. The gesture works with eye tracking, so wherever I looked, the image expanded. It felt like telepathy. I got to look through a few test photos in the Photos app, and play back photos in Memories. After having used a Quest 3 recently, Apple’s passthrough cameras and display resolution are on another planet.

Panoramic photos were a surprise. They opened up and wrapped around me, felt like windows into other places — almost 3D, in fact. And spatial videos look nice, too. Their 3D, almost ghostly playback quality feels like it’s aiming for an immersive memory more than a straight-up video playback. But I found some limits even in my brief demo of the experience.

Unfortunately, there aren’t any captures of what I saw or recorded here: the photos are all ones Apple provided, so you’ll have to just follow along and read about my experience.

02-iphone-15-pro-cameras-for-spatial-video

Spatial videos on iPhone 15 Pro: How they work

Spatial videos need to be recorded by holding the phone horizontally. The iPhone 15 Pro uses its main and ultrawide cameras side-by-side in landscape mode to create the 3D video. It records two 1080p, 30fps videos at once, resolving the lens and distance differences with computational photography. The file is saved in HEVC format, similar to other Apple video files, with a storage size of about 130MB a minute. That’s not too bad, likely because the files are limited to that relatively low 1080p and 30fps resolution and framerate.

Spatial video clips can be shared via Messages or AirDrop, but there could be cases where files are compressed down in other apps, losing the 3D data. Still, the video clips should be usable anywhere as an everyday video file, which is great news for anyone who wondered (myself included) whether you’d have to choose a «special» 3D video format.

You do have to toggle Spatial Video mode on, either inside Apple’s Camera app settings or with a Vision Pro-mask-shaped toggle in the Camera app itself. That limits your video resolution and frame rate, and also means you have to shoot in landscape. Sorry, no 3D vertical videos.

The camera app makes recommendations on turning the camera sideways, and staying a certain distance from a subject. I was told to stay within 3 to 8 feet of what I was shooting for a good spatial video, but when I shot my test recording of someone making sushi at a table I got up closer and it looked perfectly fine. I also recorded in a well-lit room, but apparently the spatial video recording mode prevents adjustments on brightness and contrast, which means low-light recording may end up grainier than normal videos.

03-spatial-video-capture-on-iphone-15-pro

How will spatial video evolve or be edited?

There’s no API for Apple’s 3D HEVC video format, which means the files are not designed so that third-party apps can recognize them yet. But it’s possible app developers will figure out some sort of a solution. Also, it’s disappointing that there’s no other way to view the videos in 3D than on a Vision Pro.

The stereo 3D recording theoretically sounds like a process that could be adapted for other VR headsets like the $500 Meta Quest 3. It’s straightforward stereo video, and doesn’t use lidar or depth sensing for any sort of full-movement spatial capture. Unfortunately that’s not currently in Apple’s plans.

There’s no recording limit on the videos, so you could theoretically make a spatial video 3D feature-length film. But the clips will only be able to be trimmed, either in the iPhone or Vision Pro. Final Cut Pro, Apple’s video editing software, will get Spatial Video editing support, but not until sometime next year. Meanwhile if you do end up trying to edit these video files in a Mac or iOS video editor, they’ll end up being converted over into 2D-only files.

It’s also a tiny bit disappointing that the videos can’t be recorded in 4K resolution. Apple’s Vision Pro headset has astounding quality and resolution, which became clear once again as I looked at iPhone photos in the headset and zoomed in on them, or viewed panoramic photos in a wraparound mode that made it feel like I was in a vivid immersive recreation of a location, similar to a 360-degree photo. The spatial videos look really nice, but I felt the desire to see them in more fluid 60fps, 4K or both. Maybe someday.

Memories for a 3D future

Apple often seems like it’s trying to become not just a lifestyle, but a memory company. Automated photo collections are already called «Memories.» And that’s how 3D spatial videos feel on Vision Pro: hazily bordered, like digital recollections in some Minority Report or Bladerunner future. It reminded me, too, of some of the immersive 3D realism I had using Google’s real-time 3D light field conversations in Project Starline, but in this case I was just watching videos.

Apple’s choice to frame the spatial videos in a fuzzy sort of frame makes it look more like the videos are almost holographically being cast into the room, and the edges sort of dissolve a bit. But I’d love to see a way for the videos to have a standard frame around them, too.

Apple’s spatial video format shows up in a new category of its own on the Photos app, and won’t show up in Memories yet. Maybe this new spatial video tab in Photos becomes the go-to repository for new 3D experiences. Right now, Apple isn’t allowing 3D spatial photos on the iPhone 15 Pro, just videos, although the Vision Pro can capture photos itself. But Apple’s clearly aiming for everyone to hopefully start recording in the format before the headset arrives, building a library of capable videos.

That means you’ll have to remember to toggle spatial video on, and care to use it. Future Vision Pro owners will have that feature toggled on by default on the iPhone 15 Pro, but that’s a tiny portion of iPhone owners. I’ll be turning it on, because I want to see what my videos will look like next year on a Vision Pro. But unless you’re planning on buying a Vision Pro yourself, there’s little reason to record spatial video now, even though it’s possible, someday way down the road, you might regret not shooting that birthday party footage in 3D. I’m already thinking about I shot video footage at my niece’s bat mitzvah last week, and what it would have been like if I could see it in 3D, too. That new spatial video camera toggle is already playing on my future FOMO.

I Took 600+ Photos With the iPhone 15 Pro and Pro Max. Look at My Favorites

+31 more

See all photos

Technologies

We Played Nintendo Switch 2: Mario, Donkey Kong, Mouse, Camera and a Lot More

Technologies

Nintendo Switch 2: Every Reveal About the Console, New Games, Price, Release Date

The $450 console launches June 5, with Mario Kart World the highlight of its launch day game lineup.

The Nintendo Switch 2 console’s biggest reveal yet arrived Wednesday as part of the company’s latest Nintendo Direct event. This teed up a day of Switch 2 reveals that include its June 5 release date, a $450 price, its specs and its initial game lineup that extends from launch into 2026.

The console will be further buoyed by having Mario Kart World as a launch title, which will go on sale in either a $500 bundle with the Switch 2 or on its own for $80 — making the open-world racing game one of Nintendo’s most expensive yet. Other games announced, such as Donkey Kong Bananza, are being priced around $70, which matches the cost of 2023’s The Legend of Zelda: Tears of the Kingdom. That Zelda game and Breath of the Wild are among the original Switch titles that will get enhanced Nintendo Switch 2 Edition upgrades, a premium upgrade allowing players to get new features, modes and graphical enhancements that take advantage of the newer system’s capabilities.

Here are the top highlights from Nintendo’s console event, all of the games we’ve heard about so far (including new titles, Nintendo Switch 2 Editions and GameCube games coming to Nintendo Switch Online) and all of our coverage so far. You can also check out our Nintendo Switch 2 live blog for even more updates about the Switch 2 as we learn them.

Nintendo Switch 2 console

The Nintendo Switch 2’s specs saw a noticeable technical bump over the Switch. The Nintendo Switch 2 has a 7.9-inch, 1080p resolution LCD that supports a 120Hz refresh rate — matching what we now see on most Android phones. When the Switch 2 is docked, compatible games can run in 4K resolution. The new dock also includes a cooling fan.

The Switch 2 comes with 256GB of internal storage, and the new Switch 2 Game Cards will load games faster. For digital libraries, however, the Switch 2 will only work with microSD Express cards, which are different from the microSD cards that are compatible with the prior Nintendo Switch. Nintendo will have a Software Transfer feature available to help move games and data from the original Switch to the Switch 2.

The new Joy-Con for the controllers will support mouse controls, and a new C button will be part of a new GameChat communication feature that allows both voice and video chat. There’s also a Nintendo Switch 2 Camera, allowing players to see each other.

The Switch 2 also adds a second USB-C port to the top of the system, which Nintendo says can help connect its new camera accessory or charge the console when playing in tabletop mode. Nintendo also revealed a new Switch 2 Pro Controller with the C button and customizable GL and GR buttons on the back.

Following the presentation, Nintendo unveiled the system’s price of $450 in the US. It will also sell the Switch 2 bundled with a digital version of Mario Kart World for $500.

Mario Kart World

Mario Kart World removes its traditional boundaries and lets drivers roam freely across an entire world of race courses. The game will get its own Nintendo Direct later this month, where we’ll see additional details, but we already know it’ll include traditional races and a Free Roam mode, much like in the Forza Horizon series.

Getting the game bundled for an extra $50 on the cost of the Switch 2 might be the move if you are interested in the game, because Nintendo announced on its website that standalone copies of Mario Kart World will cost $80.

Joy-Con 2 C button and GameChat

Nintendo’s rolling out its new C button across several new Switch 2 controllers. The button will be used for the new GameChat communication features without a headset. The button will be used alongside a microphone on the console itself, which Nintendo says can be used whether it’s docked to a TV or in handheld mode.

In its teaser video, Nintendo promises the microphone will be able to cancel out loud background noises. GameChat will also work with a Nintendo Switch 2 Camera, allowing video chat and various camera-based game modes in supported titles. GameChat will be free at launch through March 31, 2026.

Nintendo Switch 2 Edition games upgrade titles

The Nintendo Switch 2 will play three types of games: original Switch games, Switch 2 games and Switch 2 Edition games that will receive substantial enhancements. For many of these Switch 2 Edition games, you’ll need to buy an upgrade pack if you own the original for Switch. Many of them will get more than just enhanced graphics in the upgrade; for example, Super Mario Party Jamboree will get new games that support the new Joy-Cons’ mouse controls, audio recognition and video camera gameplay options through the Switch 2 Camera.

Other Nintendo Switch games that are getting Switch 2 Edition options include The Legend of Zelda Breath of the Wild, The Legend of Zelda Tears of the Kingdom, Kirby and the Forgotten Land, Metroid Prime 4 Beyond and Pokemon Legends: Z-A. Enhancements vary: The Zelda games will start working with a companion phone app for maps and sending schematics to friends, while Kirby will get a new story that’s exclusive to the Switch 2 Edition. On the third-party side, Civilization 7 will get mouse controls.

Nintendo did not announce what upgrade packs will cost. However, some Switch games will get free updates that will improve performance or enhance features when playing them on the Switch 2. The Switch games getting these updates include:

Hyrule Warriors: Age of Imprisonment

Hyrule Warriors: Age of Imprisonment is a new game set in the world of Zelda that tells the story that leads into The Legend of Zelda: Tears of the Kingdom. This appears to be similar to how Hyrule Warriors: Age of Calamity told the story of a war that led to the events of Breath of the Wild. The teaser shows Zelda discovering that she’s arrived in the past of Hyrule, and the game will expand on how she gets involved in the corresponding Imprisonment War.

Nintendo Switch 2 games

Several other first- and third-party games were spotlighted during the Switch 2 Direct. These include Donkey Bananza, one of the first 3D platforming games featuring DK since 1999’s Donkey Kong 64. Kirby will also return to the racing genre in Kirby Air Riders, which comes more than 20 years after the GameCube racer Kirby’s Air Ride. DragXDrive will use mouse controls to control a futuristic wheelchair basketball game in which players will simulate push and pull motions to control their character.

An onslaught of Switch 2 third-party games were quickly shuffled through during the Direct, which I list below. An unnamed James Bond game is in development at Hitman studio IO Interactive, as is a darker title from Elden Ring creator FromSoftware called The Duskbloods, which will be exclusive to the Switch 2.

The full list of announced Nintendo Switch 2 games includes:

Borderlands 4
Bravely Default Flying Fairy HD Remaster
Cyberpunk 2077: Ultimate Edition
Daemon X Machina: Titanic Scion
Deltarune
Donkey Kong Bananza
Drag x Drive
EA Sports FC
EA Sports Madden NFL
Elden Ring Tarnished Edition
Enter the Gungeon 2
Fast Fusion
Final Fantasy 7 Remake Intergrade
Fortnite
Hades 2
Hitman World of Assassination — Signature Edition
Hogwarts Legacy
Hollow Knight: Silksong
Hyrule Warriors: Age of Imprisonment
Kirby Air Riders
Kirby and the Forgotten Land — Nintendo Switch 2 Edition and Star-Crossed World
Kunitsu-Gai: Path of the Goddess
Mario Kart World
Metroid Prime 4: Beyond — Nintendo Switch 2 Edition
NBA 2K
Nintendo Switch 2 Welcome Tour
Nobunaga’s Ambition: Awakening Complete Edition
Pokemon Legends: Z-A — Nintendo Switch 2 Edition
Project 007
Puyo Puyo Tetris 2S
Rune Factory: Guardians of Azuma — Nintendo Switch 2 Edition
Sid Meier’s Civilization 7 — Nintendo Switch 2 Edition
Split Fiction
Star Wars Outlaws
Starseeker: Astroneer Expeditions
Street Fighter 6
Super Mario Party Jamboree — Nintendo Switch 2 Edition and Jamboree TV
Survival Kids
The Duskbloods
The Legend of Zelda: Breath of the Wild — Nintendo Switch 2 Edition
The Legend of Zelda: Tears of the Kingdom — Nintendo Switch 2 Edition
Tony Hawk’s Pro Skater 3 and 4
WWE 2K
Yakuza 0 Definitive Edition

Among these titles, you can see the full list of June 5 Switch 2 launch day games here.

Nintendo GameCube library coming to Switch 2

The Nintendo Switch Online game library will add GameCube games to the Switch 2. On launch day, these games will initially include The Legend of Zelda: The Wind Waker, SoulCalibur 2 and F-Zero GX, with each game getting enhanced graphics. Online multiplayer will also be added to certain titles. Super Mario Sunshine, Super Mario Strikers and Luigi’s Mansion are among the games set to arrive later. At launch, Nintendo will also sell a GameCube controller, which will be wireless and include a C button for GameChat.