Connect with us

Technologies

Apple Vision Pro’s Biggest Missing Pieces

Commentary: Apple’s AR/VR «spatial computer» pushes the upper limits of immersive tech. But it has some notable omissions.

The evolution of VR and AR is in major flux, and right now Apple’s bleeding-edge, ultra-expensive Vision Pro headset is sitting at the top of the heap — and it’s not even expected to arrive until 2024.

After a demo at WWDC, I came away instantly impressed at how the Vision Pro hardware synthesized so much of what I’ve seen in VR and AR over the last five years. But this time it was all done with Retina Display-level resolution and smooth, easy hand-tracking finesse. At $3,499 (around £2,800 or AU$5,300 converted), Apple’s hardware is priced far beyond VR headsets like the Meta Quest 2, and also aims to be a full computer experience in AR (as well as VR, even though Apple doesn’t outright acknowledge it).

Even so, there are notable absences from the Vision Pro, at least based on what Apple presented at WWDC. I had expectations as to what Apple might make the killer apps and features for its spatial computer headset, and only some of them materialized. Maybe others will emerge as we get closer to Apple’s 2024 headset release, or get introduced via software updates much like Meta has done with the Quest over time.

Still, I’m surprised they’re not already part of the Vision Pro experience. To me, they’ll eventually make everything I saw work even better.

Read moreBest VR Headsets of 2023

Fitness

The Meta Quest’s best feature, other than games, is its ability to be a portable exercise machine. Beat Saber was my pandemic home workout, and Meta’s acquisition of Within (maker of Supernatural, a subscription fitness app that pairs with the Apple Watch) indicates how much fitness is already a part of the VR landscape.

Apple is a prime candidate to fuse VR, AR, fitness and health and take the experience far beyond what Meta has done. Apple already has the Apple Watch and Apple Health and Fitness Plus subscription workouts. And yet, the Vision Pro has no announced fitness or health apps yet, except for a sitting-still Meditation app that’s more of a breathing prompt. 

When will the Apple Watch become part of the Vision Pro experience?

James Martin/CNET

Even more puzzling: The Vision Pro seemingly doesn’t work with the Apple Watch at all. This could change. Maybe Apple is waiting to discuss this aspect next year. Or, maybe, it will arrive with a future version of the Vision hardware.

Some VR sporting game app makers are already announcing ports for the Vision Pro, including Golf Plus, an app that works in VR with controllers. The assumption, for now, is that these apps will find a way to work just using eye and hand tracking.

Apple didn’t even demonstrate that much active motion inside the Vision Pro; my demos were mostly seated, except for a final walk-around experience where I looked at a dinosaur up close.

Is the dangling battery pack part of the concern? The headset’s weight? Or is Apple starting with computing interfaces first and adding fitness later?

The iPhone in your pocket should ideally interface with Vision Pro, too.

Lexy Savvides/CNET

iPhone, iPad and Watch compatibility

Speaking of fitness and the Apple Watch, I always imagined Apple’s AR headset would emphasize seamless compatibility with all of its products. Apple didn’t exactly do that with the Vision Pro, either.

The Vision Pro will work as a monitor-extending device with Macs, providing high-res virtual displays in a similar way that headsets like the Quest 2, Quest Pro and others already do. I didn’t get to try using the Vision Pro with a Mac, and I didn’t get to use a trackpad or keyboard, either. The Vision Pro will work with Magic Trackpads and Magic Keyboards to add physical trackpad/typing input options, again, like other VR/AR headsets, in addition to onboard eye- and hand-tracking.

And yet, the Vision Pro won’t interface directly with iPhones, iPads or the Apple Watch. Not yet, at least.

The Vision Pro primarily runs iPad-type apps. This is why the iPad Pro seems to be the best computer companion to the Vision Pro: it has a keyboard, a trackpad, built-in motion tracking that’s already AR-friendly, front and rear depth-sensing cameras that could possibly help with 3D scanning environments or faces, and it has a touchscreen and Pencil stylus.

Qualcomm’s software tools for AR glasses extend phone apps to headsets. The Apple Vision Pro bypasses the phone and works on its own.

Qualcomm

Apple is emphasizing that the Vision Pro is a self-contained computer that doesn’t need other devices. That’s understandable, and most of Apple’s cloud services, like FaceTime, will work so that the Vision Pro will essentially absorb most iPhone and iPad features. Yet I don’t understand why iPhones, iPads and Watches wouldn’t be welcome input accessories. Their touchscreens and motion controls could help them act as remotes or physical-feedback devices, in a similar way to how Qualcomm is already looking at the relationship between phones and AR glasses. I hold up my iPhone all the time to enter passwords on the Apple TV. I seamlessly drop photos, links and text from my iPhone over to my Mac.

Touchscreens could act as virtual keyboards. Drawing on the iPad could mirror a 3D art interface. With Apple’s already excellent passthrough cameras, iPhone, iPad and Watch displays could become interactive second screens, tactile interfaces that sprout extra parts in AR. Also, there’s the value of haptics and physical feedback.

The PSVR 2 controller: One advantage to physical devices is physical feedback.

James Martin/CNET

No haptics

The buzzing, tapping and rumbling feelings we get on our phones, watches and game controllers, those are feedback tools I’ve really connected with when I go into VR. The PlayStation VR 2 even has rumbling feedback in its headset. The Vision Pro, with eye and hand tracking, has no controllers. And no haptic feedback. I’ve been fascinated by the future of haptics — I saw a lot of experimental solutions earlier this year. For now, Apple is sitting out on haptic solutions for Vision Pro.

When I use iPhones and the Watch, I feel those little virtual clicks as reminders of when I’ve opened something, or when information comes in. I feel them as extensions of my perceptual field. In VR, it’s the same way. Apple’s pinch-based hand tracking technically has some physical sensation when your own fingers touch each other, but nothing will buzz or tap to let you know something is happening beyond your field of view — in another open app, for instance, or behind you in an immersive 3D environment.

Microsoft made a similar decision with the HoloLens by only having in-air hand tracking, but former AR head Alex Kipman told me years ago that haptics were part of the HoloLens roadmap. 

Apple already has haptic devices; the Apple Watch, for example. All those iPhones, too. I’m surprised the Vision Pro doesn’t already have a solution for haptics. But maybe it’s also on its roadmap? 

Logitech’s VR Ink, released in 2019, is an in-air 3D stylus. How will Apple handle creative tools in 3D?

Logitech

Will there ever be other accessories like the Pencil?

One of the wildest parts about a mixed-reality future is how it can blend virtual and real tools together, or even invent tools that don’t exist. I’ve had my VR controllers act like they’re morphing into objects that feel like they’re an extension of my body. Some companies like Logitech have already developed in-air 3D styluses for creative work in VR and AR.

Apple’s Vision Pro demos didn’t show off any creative apps beyond the collaborative Freeform, and nothing that showed how 3D inputs could be improved with handheld tools. 

Maybe Apple is starting off by emphasizing the power of just eyes and hands here, similar to how Steve Jobs initially refused to give the iPad a stylus. But the iPad has a Pencil now, and it’s an essential art tool for many people. Dedicated physical peripherals are helpful, and Apple has none with its Vision Pro headset (yet). I do like VR controllers, and Meta’s clever transforming Quest Pro controllers can be flipped around to become writing tools with an added stylus tip. As a flood of creative apps arrive on the Vision Pro in 2024, will Apple address possibilities for dedicated accessories? Will the Vision Pro allow for easy pairing of them? Hopefully, yes.

The Apple Vision Pro is a long way from arriving, and there’s still so much we don’t know. As Apple’s first AR/VR headset evolves, however, these key aspects should be kept in mind, because they’ll be incredibly important ways to expand how the headset feels useful and flexible for everyone.

Technologies

Wild Weather Ahead: Here’s How 2024 Is Shaping Up After the Hottest Year on Record

The climate crisis is impacting communities around the world. Here’s what to know about dealing with extreme weather in 2024.

We just lived through the hottest year since recordkeeping began more than a century ago, but before too long when we look back at 2023, it might not stand out as the pinnacle of extreme heat. 

That’s because it’s unlikely to be the only hottest year that we experience. Our climate is changing, growing warmer due to the emissions from burning fossil fuels, and our weather is changing with it. It’s possible that this year may turn out to be hotter still.

In March, scientists from the EU’s Copernicus Climate Change Service said February 2024 was the hottest February according to records that stretch back to 1940. The news came on the heels of their report in early January that, as expected, 2023 was indeed the hottest year on record. Temperatures closed in on the critical 1.5-degree Celsius rise above preindustrial levels, after which we will see irreversible damage to the planet. These aren’t freak outliers: The extreme heat we’re experiencing is something we’ll need to be prepared to deal with on a much more regular basis, along with storms, floods and drought.

Later in March, the US National Oceanic and Atmospheric Administration issued its spring outlook, predicting that most of the continental US and Alaska will see above-average temperatures from April through June. The risk of flooding, it said, will ease during the three-month period because of «historically low winter snow cover» in large parts of the country.

A key trend highlighted by the US government’s Fifth National Climate Assessment, published in November, was that climate change is provoking extreme weather events across the country that are both more frequent and more severe. It pointed to an increase in heatwaves and wildfires in the West over the past few decades, the increased drought risk in the Southwest over the past century and more extreme rainfall east of the Rockies. Hurricanes have also been intensifying, as those who have found themselves in the path of a storm know all too well.

You’ll need to be prepared. Extreme weather is going to have a widespread impact on industry, society and individuals. Last year in the US there were 25 extreme weather events with losses amounting to over $1 billion that resulted in the deaths of 464 people. People lost their homes, saw personal property damaged or suffered mental and physical health issues.

Three months into 2024, we’re staring down the barrel of another potentially record-setting hot year. If there’s a silver lining, it’s that the US is now better prepared than ever and we know what steps you can take to better deal with these unwelcome events. When it comes to weather, forewarned is forearmed. 

The US has been taking active steps. The Biden administration has provided funding to build resilient communities, and a new (as of September 2023) National Climate Resilience Framework, which should provide the US with a whole range of protections. These include conserving water resources, modernizing and strengthening the electric grid against weather and disasters and building infrastructure to protect communities and ecosystems from sea level rise, tidal flooding, hurricanes and storm surges.

At home and in your community, you can take steps, too, including preparing your home for wildfires and flooding and recognizing signs of heat-related health issues. This way, when wild weather comes calling, its impact on our homes, health and livelihoods is minimized.

Forecast 2024

Last year’s heat was no anomaly. It’s part of a long-term trend: The last 10 years have been the 10 warmest on record, according to NASA, with most of the Earth’s warming taking place over the last 40 years. Most forecasters are anticipating yet another year of extreme heat ahead.

«If we look at the forecast for the next three months in the long range, it’s suggesting that the trend that we’re seeing in baseline warming could continue, and so 2024 could rival 2023 for being the hottest year on record, which is very scary,» says Chloe Brimicombe, a heatwave researcher at the University of Graz.

Some of the extreme weather we experienced in the latter half of last year and will continue to experience in the first half of this year is a result of El Niño, a cyclical climate event that sees unusually warm ocean waters that has a knock-on effect of warmer temperatures and increased rainfall across the southern part of the US. For instance, temperatures in Death Valley, California, peaked at 128 degrees Fahrenheit in July, while forecasters predicted warmer temperatures in northern parts of the US stretching into February and a colder, wetter winter for Southern states.

While meteorologists are able to make long-term predictions about El Niño, other climate-related predictions are trickier. «All things told, we’re going to see an increased prevalence of heat events across the globe, but we can’t tell right now exactly where that will be,» says Andy Hoell, a climate scientist at NOAA.

What we do know, he adds, is that the climate crisis can compound events such as extreme heat or extreme rainfall to make them more likely or more severe. 

In the past, it wasn’t always easy to draw direct links between extreme weather events and climate change. But huge improvements in attribution science (the ability to specifically identify emissions as the cause for unusually dramatic weather) in recent years have changed the game. The World Weather Attribution program, based at Imperial College London, has now completed nine studies on droughts, heatwaves, wildfires and heavy rainfall in North America. «Every study found that climate change made the event more intense and more likely,» says Ben Clarke, a researcher at WWA.

The speed at which climate scientists are able to identify human-caused climate change as the culprit for extreme weather has also dramatically improved. Last year alone, Climate Central was able to attribute record-breaking spring heat in the western US, and ongoing extreme heat stretching through the summer in Texas and Florida, to climate change as it was happening. «It’s much more impactful as far as our understanding of what climate change really is if we can make that connection in real time,» says Andrew Pershing, vice president of science at Climate Central, a climate science analysis non-profit.

Thanks to attribution science, we can confidently point to a heatwave we’ve experienced and say whether climate change played a role in making it happen. But it also helps us to recognize that extreme weather events we’re experiencing are part of a pattern – one that can’t be broken without tackling the root causes of the climate crisis. «Until the world moves away from fossil fuels and reduces emissions to net zero,» says Clarke, «extreme weather events in North America will continue to become more intense, more dangerous and more deadly.»

Even if you live in a region that hasn’t yet directly been impacted by a climate-linked weather event, you’re not off the hook.

«As the climate continues to warm, most areas will be at an increased risk of some types of climate-linked extreme weather,» says Russell Vose, chief of the Monitoring and Assessment Branch at NOAA’ National Centers for Environmental Information and one of the NCA’s authors. «Perhaps the best example is extreme heat – it can occur anywhere.»

He points to the scorching heat dome that descended on the Pacific Northwest in June and July 2021, which was unprecedented in the historical record. The unpredictable nature of such extreme heat means no regions are marked as safe.

In fact, a region that’s been lucky enough to not yet experience an extreme heat event is more likely to experience one in the future and suffer more greatly due to lack of preparedness, according to a study published by scientists from Bristol University last April.

Scientists are more concerned about the ability of people in areas that don’t usually get intensely hot to cope when their turn comes. «What worries me would be something in the Upper Midwest or the Northeast that just hasn’t had a major heat event for a few years,» says Pershing. «I think we kind of lose a little bit of that muscle memory.»

Weather’s unequal impacts

The weather might not discriminate when it comes to who gets hit, but that doesn’t mean its impacts are experienced equally by all groups across American society.

«Certain groups are simply more vulnerable to extreme events due to geographic, socioeconomic or demographic factors,» says Vose. He points to the extreme rainfall brought by Hurricane Harvey in 2017, which led to a large number of homes being flooded in Harris County, Texas, with a disproportionate impact on low-income Hispanic neighborhoods.

When a heatwave hits, it will feel hotter in high-density urban environments that are more likely to be occupied by people of color or people living in poverty than in more spread-out neighborhoods or rural areas. Then some are homeless and can’t access health care. They have little ability to protect themselves, no matter how much warning they get about an incoming heatwave. This makes these groups much more vulnerable to the health risks of extreme heat.

Heat researchers are extremely concerned about people who live in housing not resistant to warm temperatures, says Brimicombe, who points out that those who rent are especially at risk. «If you’re a tenant, you have less ability to adapt your house to extreme heat than if you’re a homeowner,» she says. «And that also means young families, because babies are vulnerable to extreme heat.»

Not only are economically disadvantaged communities in the US more susceptible to feeling the worst impacts of extreme weather, but they have also done the least to contribute towards the climate crisis in the first place. A study published last August revealed that the wealthiest households in the US are historically responsible for 40% of the country’s climate emissions.

Meanwhile, these same households have more tools at their disposal to protect themselves from the impact of climate-related weather events. In 2019, The New York Times reported that wealthy California residents were banding together to hire private firefighters to protect them from the impacts of wildfires.

The Biden administration is well aware that marginalized and minority groups are hardest hit by climate change, including extreme weather. At the beginning of his term, the president set up the White House Environmental Justice Advisory Council, made up of leading experts from the US climate justice community.

Last September the group published its policy recommendations urging the government to ensure climate disasters do not further or exacerbate harm to vulnerable populations and communities. 

«Disaster relief should never be the cause of deepening inequality in any neighborhood, region, or Tribal community,» the council wrote in its recommendations. «When disaster hits, the goal of government should be that the people hit the hardest should emerge stronger and more secure than before, not the opposite.»

It recommended a number of measures that would help protect people in case of extreme weather including the creation of a low-cost national flood insurance and the establishment of a «Just Relocation Fund» that would provide communities hit by climate impacts with a relocation process based on a dignity framework with respect for their human rights. 

The White House has yet to respond to the recommendations, but if it does act on them this would hopefully prevent a repeat of the aftermath of Hurricane Katrina in 2005, in which Black communities were allocated less money to rebuild their housing, resulting in a lawsuit against the federal government.

Through the Bipartisan Infrastructure Law and other initiatives, the Biden administration is investing heavily in adaptation, mitigation and resilience measures designed to protect all Americans from the impacts of climate-linked extreme weather. As with all funding, people may have to wait some time to feel the full impact of that funding. In the meantime, there are a number of steps you can take to keep yourself safe in the months ahead.

How to weather the weather, whatever the weather

Summer’s not so far off, meaning sizzling days are on the horizon. 

Intense heat poses some scary risks to our health, including heat cramps, heat exhaustion and heat stroke, which can be life-threatening. It’s important to familiarize yourself with the signs so that you’ll recognize them in yourself and others, and can therefore seek medical attention if necessary.

Remember that heat is more likely to adversely affect older people, children and babies, and those with preexisting health conditions. There may be cooling centers or other well-air-conditioned places in your community where you can take refuge – if you do, consider taking elderly or vulnerable neighbors with you. «Look out for friends and families,» said Brimicombe. «Don’t be complacent.»

The British writer and fellwalker Alfred Wainwright is widely credited as coining the phrase, «there’s no such thing as bad weather, only unsuitable clothing.» Wainwright, who died in 1991, didn’t live through the kind of consistently bad weather we’re experiencing in this era of extreme heat, but that doesn’t mean we have nothing to learn from him. In the midst of a heatwave, it’s best to wear loose-fitting clothes in light colors, rather than black, which absorbs the heat.

Make sure you stay hydrated and try to spend as little time as possible outside in the sun. Try to block sunlight from warming your house, and consider buying reflectors to place in your windows that can help keep the heat out. At nighttime, take note of when it might be cooler outside than in, and use this to your advantage by opening doors and windows to let the internal temperature of your house regulate. Fans can be effective, but at very high temperatures they’re likely to just start pushing the hot air around – in which case you should, sparingly and without putting too much pressure on the grid, resort to air conditioning, or moving to your local cooling center.

Remember that global warming is worldwide, so the same heat warnings apply even if you plan to travel to other parts of the world over the summer. The heat waves that hit the US in the summer of 2023 also impacted areas of Europe, including popular vacation spots in the Mediterranean. Countries including Greece, Spain and Italy were all affected by wildfires that resulted in the evacuation of locals and tourists alike from some areas and islands.

The surge in Europe-bound American tourists that occurred in 2023 is expected to continue this year, but if you’re planning to be among them it’s important not to travel without comprehensive insurance. Likewise, if you’re traveling in the peak months of July and August, be prepared to adjust your itinerary in case of extreme heat to ensure you’re not putting your health at risk. This may mean spending more time indoors than you’d planned for the sake of your health.

For other types of extreme weather that may hit your property such as wildfires, storms or floods, it may be useful to have an evacuation plan. You should prepare an emergency evacuation bag, also known as a go bag or a bug-out bag. Don’t forget to plan for your pets. The National Fire Protection Association has a handy guide on how to prepare your home for wildfires

One of the easiest but most important things you can do is keep an eye on long- and short-term weather forecasts. The silver lining for people in the US, says Pershing, is that the country has great weather forecasting capabilities and the channels to communicate incoming events to people so you can prepare. «The gaps are really whether you take it seriously yourself,» he says.

So for anyone who does take it seriously, be sure to read our tips on how to prepare yourself and your home for wildfires, hurricanes, floods and storms.

Here are some additional resources:

For even more details on natural disasters and how to prepare beforehand or respond after an event takes place, check out https://www.ready.gov/.

Correction, March 15: This story originally misstated the name of the National Fire Protection Association.

Continue Reading

Technologies

SpaceX Calls Mission 3 a Success, Despite Losing Starship: How to Rewatch

On its third attempt, SpaceX launched its Starship and cruised into space, but lost the rocket after reentry to Earth.

SpaceX launched its third Starship mission on Thursday, with the space exploration company owned by Elon Musk forging ahead after the first two attempts exploded after takeoff. SpaceX considers those first two missions successful, thanks to the data it was able to collect, and the third mission was the most successful of the bunch.

Shortly before 9:30 a.m. ET Thursday, the company posted three words to the Musk-owned X social media site: «Liftoff of Starship!» A 36-second video showed the rocket engines igniting and then Starship rising amid a cloud of exhaust smoke and up into the sky.

But the rocket did not complete the round trip, as you can see by rewatching the full test flight. «The ship has been lost. No splashdown today,» Dan Huot of SpaceX communications said on the stream. «But we were able to get through some of the early phases of reentry.»

SpaceX quality engineering manager Kate Tice noted on the stream that SpaceX wasn’t intending to recover Starship anyway, and had been planning to crash it into the ocean. 

Starship is arguably the most ambitious effort for Musk, who owns the satellite-based internet company Starlink along with X, EV maker Tesla and the neurotechnology company Neuralink. The Starship missions are critical to SpaceX’s — and Musk’s — goal of getting to and eventually settling the moon and Mars. 

With a new flight trajectory and hopes for new data insights, the space company’s third mission may prove to be its most important yet.

When did the Starship mission launch?

Starship’s third mission launched on March 14. It had been pending favorable weather and a license from the Federal Aviation Administration.

How to rewatch Starship mission 3

SpaceX set up livestreams for watching the third mission. One was the SpaceX account on X, and another was the SpaceX third mission landing page. You could also watch via CNET’s YouTube channel.

You can rewatch the launch on X, on the mission 3 page, or via CNET’s YouTube stream embedded here.

What SpaceX achieved in Starship mission 3

SpaceX’s third Starship mission was designed to test whether the spacecraft can complete certain tasks. After liftoff, the company planned to open Starship’s payload door and transfer its propellant from one part of the vehicle to another. For the first time, SpaceX also attempted to relight its Raptor engine while in space, a test that could be critical for future missions as it eventually tries to propel Starship through space.

Starship took a different flight path this time around, and had planned to land in the Indian Ocean instead of the Pacific Ocean until it lost the vehicle after reentry. In a statement, SpaceX said that the new flight path was designed to maximize «public safety,» but the company didn’t discuss how. 

The new flight path also paved the way for SpaceX to try «in-space engine burns,» a reference to the company attempting to reignite the Raptor engine in space.

«Huge congratulations to the entire team for this incredible day: clean count (glad the shrimpers could get out in the nick of time!), liftoff, hot staging, Super Heavy boost back and coast (and likely a couple engines making mainstage during landing burn!), clean ship ‘insertion’ and coast, payload door cycling and prop transfer demo (to be confirmed!), and ship entry!» SpaceX President Gwynne Shotwell posted on X following the launch, naming the successful components of the test.

The mission was slated to last for 1 hour, 15 minutes. Previous missions, if completed, would have lasted 90 minutes.

What happened in the previous missions?

April mission: Forced detonation

The first Starship mission launched in April 2023. Early on in the mission, the two stages of Starship — the reusable upper stage, called Starship, and its Super Heavy first-stage booster — were supposed to separate. That didn’t happen, and for safety reasons, the SpaceX team was forced to detonate the vehicle just 4 minutes into the mission.

November mission: Explosion due to liquid oxygen

In November 2023, Starship launched on its second mission. That time around, Starship was able to separate its two stages and it reached nominal first-stage engine burn. However, Starship exploded 8 minutes after launch when it tried to vent its liquid oxygen. Oddly, the explosion may not have needed to happen. Earlier this year, Musk said on a real mission carrying payload — meaning the materials a spaceship carries to perform its scientific mission — liquid oxygen wouldn’t be onboard.

«Starship’s second flight test achieved a number of major milestones and provided invaluable data to continue rapidly developing Starship,» the company wrote on its site. «Each of these flight tests continue to be just that: a test. They aren’t occurring in a lab or on a test stand, but are putting flight hardware in a flight environment to maximize learning.»

Corinne Reichert contributed to this report.

Continue Reading

Technologies

iOS 17 Cheat Sheet: All Your Questions on the iPhone Update Answered

We explain everything from new features to upcoming updates.

Apple’s iOS 17 was released in September, shortly after the company held its Wonderlust event, where the tech giant announced the new iPhone 15 lineup, the Apple Watch Series 9 and the Apple Watch Ultra 2. We put together this cheat sheet to help you learn about and use the new features in iOS 17. It’ll also help you keep track of the subsequent iOS 17 updates.

iOS 17 updates

Using iOS 17

Getting started with iOS 17

Make sure to check back periodically for more iOS 17 tips and how to use new features as Apple releases more updates.

17 Hidden iOS 17 Features You Should Definitely Know About

See all photos

Continue Reading

Trending

Exit mobile version