Technologies

Apple iPhone Spatial Video Arrives in Beta and Looks Amazing on Vision Pro

I tried the new iPhone 15 Pro camera feature on Apple’s upcoming AR-VR headset. The 3D is undeniably vivid.

I’m looking at a plate of sushi hovering in front of me in 3D. The chef finishes off toppings on yellowtail rolls and tuna, talking to me as she works. It looks vivid. It looks real. The amazing part is that I just shot this video myself, moments earlier, on an iPhone 15 Pro. And now it’s a VR experience I’m watching in beautiful 3D on Apple’s Vision Pro headset.

I swipe with my fingers and watch other ghostly videos Apple provided. Families in a home, walking through grass. Cuddling together. All in lifelike 3D. I feel like I’m peeking in on their lives, which is weird and intimate. But the vividness is undeniable.

I’m experiencing Apple’s new Spatial Video iPhone camera feature, now available in Apple’s new iOS 17.2 public beta, with the final version rolling out in 17.2 later this year. It allows you to record 3D videos, but to use it you’ll need Apple’s best phone, the $1,000 iPhone 15 Pro. And to watch the videos in 3D you’ll need the $3,500 Vision Pro headset, which launches early next year.

The videos look great and the 3D is compellingly realistic. They’re also easy to record, and can save as videos that will play back in 2D in a normal video format. But ultimately this feature is made for a 2024 product that, at $3,500, it’s safe to say most people won’t buy anytime soon. Still, the experience is impressive.

Vision Pro impressed me once again

I first used the Vision Pro at Apple’s WWDC conference in June, and once again I’m reminded of its extraordinary display capabilities. Photos and video look fantastic on Apple’s headset. I’d even venture to say the Vision Pro beats any display I have anywhere in my home right now. Yes, I’d want to look at videos and photos — 3D and otherwise — if I had a Vision Pro.

My second dive into the Vision Pro, complete with fitted prescription lenses that matched my needs, reminds me how much smaller the headset was than I remembered. And also, how effortless the interface is.

Setting up eye tracking was a rapid process; I looked at dots around a circle and tapped them. I also tried a new set of zoom gestures, pinching my fingers and stretching them to expand photos. The gesture works with eye tracking, so wherever I looked, the image expanded. It felt like telepathy. I got to look through a few test photos in the Photos app, and play back photos in Memories. After having used a Quest 3 recently, Apple’s passthrough cameras and display resolution are on another planet.

Panoramic photos were a surprise. They opened up and wrapped around me, felt like windows into other places — almost 3D, in fact. And spatial videos look nice, too. Their 3D, almost ghostly playback quality feels like it’s aiming for an immersive memory more than a straight-up video playback. But I found some limits even in my brief demo of the experience.

Unfortunately, there aren’t any captures of what I saw or recorded here: the photos are all ones Apple provided, so you’ll have to just follow along and read about my experience.

Spatial videos on iPhone 15 Pro: How they work

Spatial videos need to be recorded by holding the phone horizontally. The iPhone 15 Pro uses its main and ultrawide cameras side-by-side in landscape mode to create the 3D video. It records two 1080p, 30fps videos at once, resolving the lens and distance differences with computational photography. The file is saved in HEVC format, similar to other Apple video files, with a storage size of about 130MB a minute. That’s not too bad, likely because the files are limited to that relatively low 1080p and 30fps resolution and framerate.

Spatial video clips can be shared via Messages or AirDrop, but there could be cases where files are compressed down in other apps, losing the 3D data. Still, the video clips should be usable anywhere as an everyday video file, which is great news for anyone who wondered (myself included) whether you’d have to choose a «special» 3D video format.

You do have to toggle Spatial Video mode on, either inside Apple’s Camera app settings or with a Vision Pro-mask-shaped toggle in the Camera app itself. That limits your video resolution and frame rate, and also means you have to shoot in landscape. Sorry, no 3D vertical videos.

The camera app makes recommendations on turning the camera sideways, and staying a certain distance from a subject. I was told to stay within 3 to 8 feet of what I was shooting for a good spatial video, but when I shot my test recording of someone making sushi at a table I got up closer and it looked perfectly fine. I also recorded in a well-lit room, but apparently the spatial video recording mode prevents adjustments on brightness and contrast, which means low-light recording may end up grainier than normal videos.

How will spatial video evolve or be edited?

There’s no API for Apple’s 3D HEVC video format, which means the files are not designed so that third-party apps can recognize them yet. But it’s possible app developers will figure out some sort of a solution. Also, it’s disappointing that there’s no other way to view the videos in 3D than on a Vision Pro.

The stereo 3D recording theoretically sounds like a process that could be adapted for other VR headsets like the $500 Meta Quest 3. It’s straightforward stereo video, and doesn’t use lidar or depth sensing for any sort of full-movement spatial capture. Unfortunately that’s not currently in Apple’s plans.

There’s no recording limit on the videos, so you could theoretically make a spatial video 3D feature-length film. But the clips will only be able to be trimmed, either in the iPhone or Vision Pro. Final Cut Pro, Apple’s video editing software, will get Spatial Video editing support, but not until sometime next year. Meanwhile if you do end up trying to edit these video files in a Mac or iOS video editor, they’ll end up being converted over into 2D-only files.

It’s also a tiny bit disappointing that the videos can’t be recorded in 4K resolution. Apple’s Vision Pro headset has astounding quality and resolution, which became clear once again as I looked at iPhone photos in the headset and zoomed in on them, or viewed panoramic photos in a wraparound mode that made it feel like I was in a vivid immersive recreation of a location, similar to a 360-degree photo. The spatial videos look really nice, but I felt the desire to see them in more fluid 60fps, 4K or both. Maybe someday.

Memories for a 3D future

Apple often seems like it’s trying to become not just a lifestyle, but a memory company. Automated photo collections are already called «Memories.» And that’s how 3D spatial videos feel on Vision Pro: hazily bordered, like digital recollections in some Minority Report or Bladerunner future. It reminded me, too, of some of the immersive 3D realism I had using Google’s real-time 3D light field conversations in Project Starline, but in this case I was just watching videos.

Apple’s choice to frame the spatial videos in a fuzzy sort of frame makes it look more like the videos are almost holographically being cast into the room, and the edges sort of dissolve a bit. But I’d love to see a way for the videos to have a standard frame around them, too.

Apple’s spatial video format shows up in a new category of its own on the Photos app, and won’t show up in Memories yet. Maybe this new spatial video tab in Photos becomes the go-to repository for new 3D experiences. Right now, Apple isn’t allowing 3D spatial photos on the iPhone 15 Pro, just videos, although the Vision Pro can capture photos itself. But Apple’s clearly aiming for everyone to hopefully start recording in the format before the headset arrives, building a library of capable videos.

That means you’ll have to remember to toggle spatial video on, and care to use it. Future Vision Pro owners will have that feature toggled on by default on the iPhone 15 Pro, but that’s a tiny portion of iPhone owners. I’ll be turning it on, because I want to see what my videos will look like next year on a Vision Pro. But unless you’re planning on buying a Vision Pro yourself, there’s little reason to record spatial video now, even though it’s possible, someday way down the road, you might regret not shooting that birthday party footage in 3D. I’m already thinking about I shot video footage at my niece’s bat mitzvah last week, and what it would have been like if I could see it in 3D, too. That new spatial video camera toggle is already playing on my future FOMO.

I Took 600+ Photos With the iPhone 15 Pro and Pro Max. Look at My Favorites

+31 more

See all photos

Technologies

iOS 17 Cheat Sheet: Your Questions on the iPhone Update Answered

Here’s what you need to know about new features and upcoming updates for your iPhone.

Apple’s iOS 17 was released in September, shortly after the company held its Wonderlust event, where the tech giant announced the new iPhone 15 lineup, the Apple Watch Series 9 and the Apple Watch Ultra 2. We put together this cheat sheet to help you learn about and use the new features in iOS 17. It’ll also help you keep track of the subsequent iOS 17 updates.

iOS 17 updates

Using iOS 17

Getting started with iOS 17

Make sure to check back periodically for more iOS 17 tips and how to use new features as Apple releases more updates.

17 Hidden iOS 17 Features You Should Definitely Know About

+15 More

See all photos

Technologies

Get Ready for a Striking Aurora That Could Also Disrupt Radio Communications

Don’t expect the storm to cause a lingering problem, though.

A geomagnetic storm is threatening radio communications Monday night, but that doesn’t mean you should be concerned. In fact, it may be an opportunity to see a colorful aurora in the night sky.

The National Oceanic and Atmospheric Administration has issued a geomagnetic storm watch after witnessing a coronal mass ejection from the sun on Saturday. The watch, which was issued over the weekend and will expire after Monday, said the onset of the storm passing over Earth on Sunday night represented a «moderate» threat to communications. As the storm continues to pass through, it could deliver a «strong» threat on Monday night that could cause radio communications to be temporarily disrupted during the worst of it.

Even so, NOAA said, «the general public should not be concerned.»

A coronal mass ejection occurs when magnetic field and plasma mass are violently expelled from the sun’s corona, or the outermost portion of the sun’s atmosphere. In the vast majority of cases, the ejection occurs with no real threat to Earth. However, in the event the ejection happens in the planet’s direction, a geomagnetic storm occurs, and the Earth’s magnetic field is temporarily affected.

In most cases, geomagnetic storms cause little to no disruption on Earth, with radio communications and satellites affected most often. In extreme cases, a geomagnetic storm can cause significant and potentially life-threatening power outages — a prospect that, luckily, the planet hasn’t faced.

Switching poles

Every 11 years, the sun’s magnetic poles switch, with the north pole and south pole swapping positions. During those cycles, the sun’s activity ramps up as it gets closer to pole-switching time. The height of its activity is called solar maximum, and scientists believe we either may be entering the solar maximum or may be already in it.

During periods of heightened solar activity, sunspots increase on the sun and there’s an increase in coronal mass ejections, among other phenomena. According to NOAA, solar maximum could extend into October of this year before the sun’s activity calms and it works towards its less-active phase, solar minimum.

Even when geomagnetic storms hit Earth and disrupt communications, the effects are usually short-lived. Those most affected, including power grid operators and pilots and air traffic controllers communicating over long distances, have fail-safe technologies and backup communications to ensure operational continuity.

But geomagnetic storms aren’t only about radios. In most cases, they also present unique opportunities to see auroras in the night sky. When the storms hit, the plasma they carry creates a jaw-dropping aurora, illuminating the night sky with brilliant colors. Those auroras can be especially pronounced during the most intense phases of the storm, making for nice stargazing.

If you’re interested in seeing the aurora, you’ll need to be ready. The NOAA said the «brunt of the storm has passed» and even if it lingers into Tuesday, there won’t be much to see after Monday night.

Technologies

Last Total Solar Eclipse for 20 Years Is Coming: How to See and Photograph It

It’s your last chance until 2044.

Get your eclipse glasses ready, Skygazers: the Great American Eclipse is on its way. On April 8, there’ll be a total eclipse over North America, the last one until 2044.

A total solar eclipse happens when the moon passes between the Earth and the sun, blocking the sun and turning an otherwise sunny day to darkness for a short period of time. Depending on the angle at which you’re viewing the eclipse, you may see the sun completely shrouded by the moon (called totality) or some variation of it. The more off-angle you are and the further you are from the path of the eclipse, the less likely you’ll be to see the totality.

The 2024 total solar eclipse will happen on Monday, April 8. The Great American Eclipse will reach the Mexican Pacific coast at 11:07 a.m. PT (2:07 p.m. ET), and then traverse the US in a northeasterly direction from Texas to Maine, and on into easternmost Canada. If you want a good look at it, but don’t live in the path of totality, you shouldn’t wait much longer to book accommodation and travel to a spot on the path.

Or how about booking a seat in the sky? Delta Airlines made headlines for offering a flight that allows you to see the entire path of totality. Its first eclipse flight, from Austin, Texas, to Detroit sold out quickly. But as of Monday, Delta has added a second flight from Dallas to Detroit, which also covers the path of totality. The airline also has five flights that will offer prime eclipse viewing.

Not everyone can get on one of those elusive eclipse-viewing flights. Here’s a look at other options to nab a chance to see this rare sight and what to know about it.

Total solar eclipse path

The eclipse will cross over the Pacific coast of Mexico and head northeast over mainland Mexico. The eclipse will then make its way over San Antonio at approximately 2:30 p.m. ET on April 8 and move through Texas, over the southeastern part of Oklahoma and northern Arkansas by 2:50 p.m. ET.

By 3 p.m. ET, the eclipse will be over southern Illinois, and just 5 minutes later, will be traveling over Indianapolis. Folks in northwestern Ohio will be treated to the eclipse by 3:15 p.m. ET, and it will then travel over Lake Erie and Buffalo, New York, by 3:20 p.m. ET. Over the next 10 minutes, the eclipse will be seen over northern New York state, then over Vermont. By 3:35 p.m. ET, the eclipse will work its way into Canada and off the Eastern coast of North America.

Best places to watch the Great American Eclipse

When evaluating the best places to watch this year’s total eclipse, you’ll first want to determine where you’ll have the best angle to see the totality. The farther off-angle you are — in other words, the farther north or south of the eclipse’s path — the less of an impact you can expect.

Therefore, if you want to have the best chance of experiencing the eclipse, you’ll want to be in its path. As of this writing, most of the cities in the eclipse’s path have some hotel availability, but recent reports have suggested that rooms are booking up. And as more rooms are booked, prices are going up.

So if you want to be in the eclipse’s path, and need a hotel to do it, move fast. And Delta’s eclipse-viewing flight from Dallas to Detroit has just four seats left at the time of publication.

Eclipse eye safety and photography

As with any solar eclipse, it’s critical you keep eye safety in mind.

During the eclipse, and especially during the periods before and after totality, don’t look directly at the sun without special eye protection. Also, be sure not to look at the sun through a camera (including the camera on your phone), binoculars, a telescope or any other viewing device. This could cause serious eye injury. Sunglasses aren’t enough to protect your eyes from damage.

If you want to view the eclipse, you’ll instead need solar viewing glasses that comply with the ISO 12312-2 safety standard. Anything that doesn’t meet that standard or greater won’t be dark enough to protect your eyes. Want to get them for free? If you’ve got a Warby Parker eyeglasses store nearby, the company is giving away free, ISO-certified solar eclipse glasses at all of its stores from April 1 until the eclipse, while supplies last.

If you don’t have eclipse viewing glasses handy, you can instead use indirect methods for viewing the eclipse, like a pinhole projector.

Read more: A Photographer’s Adventure With the Eclipse

In the event you want to take pictures of the eclipse, attach a certified solar filter to your camera. Doing so will protect your eyes and allow you to take photos while you view the eclipse through your lens.

There’s also a new app to help you both protect your eyes and take better photos of the eclipse on your phone. Solar Snap, designed by a former Hubble Space Telescope astronomer, comes with a Solar Snap camera filter that attaches to the back of an iPhone or Android phone, along with solar eclipse glasses for protecting your eyesight during the event. After you attach the filter to your phone, you can use the free Solar Snap Eclipse app to zoom in on the eclipse, adjust exposure and other camera settings, and ultimately take better shots of the eclipse.

2024 eclipse compared to 2017

The last total solar eclipse occurred in 2017, and many Americans had a great view. Although there are plenty of similarities between the 2017 total solar eclipse and the one coming April 8, there are a handful of differences. Mainly, the 2024 eclipse is going to cover more land and last longer.

The 2017 eclipse started over the northwest US and moved southeast. Additionally, that eclipse’s path was up to 71 miles wide, compared with a maximum width of 122 miles for this year’s eclipse. Perhaps most importantly, the moon completely covered the sun for just 2 minutes, 40 seconds in 2017. This year, maximum totality will last for nearly four-and-a-half minutes.