Technologies

Apple Vision Pro’s Biggest Missing Pieces

Commentary: Apple’s AR/VR «spatial computer» pushes the upper limits of immersive tech. But it has some notable omissions.

The evolution of VR and AR is in major flux, and right now Apple’s bleeding-edge, ultra-expensive Vision Pro headset is sitting at the top of the heap — and it’s not even expected to arrive until 2024.

After a demo at WWDC, I came away instantly impressed at how the Vision Pro hardware synthesized so much of what I’ve seen in VR and AR over the last five years. But this time it was all done with Retina Display-level resolution and smooth, easy hand-tracking finesse. At $3,499 (around £2,800 or AU$5,300 converted), Apple’s hardware is priced far beyond VR headsets like the Meta Quest 2, and also aims to be a full computer experience in AR (as well as VR, even though Apple doesn’t outright acknowledge it).

Even so, there are notable absences from the Vision Pro, at least based on what Apple presented at WWDC. I had expectations as to what Apple might make the killer apps and features for its spatial computer headset, and only some of them materialized. Maybe others will emerge as we get closer to Apple’s 2024 headset release, or get introduced via software updates much like Meta has done with the Quest over time.

Still, I’m surprised they’re not already part of the Vision Pro experience. To me, they’ll eventually make everything I saw work even better.

Read more: Best VR Headsets of 2023

Fitness

The Meta Quest’s best feature, other than games, is its ability to be a portable exercise machine. Beat Saber was my pandemic home workout, and Meta’s acquisition of Within (maker of Supernatural, a subscription fitness app that pairs with the Apple Watch) indicates how much fitness is already a part of the VR landscape.

Apple is a prime candidate to fuse VR, AR, fitness and health and take the experience far beyond what Meta has done. Apple already has the Apple Watch and Apple Health and Fitness Plus subscription workouts. And yet, the Vision Pro has no announced fitness or health apps yet, except for a sitting-still Meditation app that’s more of a breathing prompt.

Apple Watch Ultra — When will the Apple Watch become part of the Vision Pro experience?

James Martin/CNET

Even more puzzling: The Vision Pro seemingly doesn’t work with the Apple Watch at all. This could change. Maybe Apple is waiting to discuss this aspect next year. Or, maybe, it will arrive with a future version of the Vision hardware.

Some VR sporting game app makers are already announcing ports for the Vision Pro, including Golf Plus, an app that works in VR with controllers. The assumption, for now, is that these apps will find a way to work just using eye and hand tracking.

Apple didn’t even demonstrate that much active motion inside the Vision Pro; my demos were mostly seated, except for a final walk-around experience where I looked at a dinosaur up close.

Is the dangling battery pack part of the concern? The headset’s weight? Or is Apple starting with computing interfaces first and adding fitness later?

iPhone 14 Pro vs. iPhone 13 Pro — The iPhone in your pocket should ideally interface with Vision Pro, too.

Lexy Savvides/CNET

iPhone, iPad and Watch compatibility

Speaking of fitness and the Apple Watch, I always imagined Apple’s AR headset would emphasize seamless compatibility with all of its products. Apple didn’t exactly do that with the Vision Pro, either.

The Vision Pro will work as a monitor-extending device with Macs, providing high-res virtual displays in a similar way that headsets like the Quest 2, Quest Pro and others already do. I didn’t get to try using the Vision Pro with a Mac, and I didn’t get to use a trackpad or keyboard, either. The Vision Pro will work with Magic Trackpads and Magic Keyboards to add physical trackpad/typing input options, again, like other VR/AR headsets, in addition to onboard eye- and hand-tracking.

And yet, the Vision Pro won’t interface directly with iPhones, iPads or the Apple Watch. Not yet, at least.

The Vision Pro primarily runs iPad-type apps. This is why the iPad Pro seems to be the best computer companion to the Vision Pro: it has a keyboard, a trackpad, built-in motion tracking that’s already AR-friendly, front and rear depth-sensing cameras that could possibly help with 3D scanning environments or faces, and it has a touchscreen and Pencil stylus.

A man with AR glasses on, holding a phone, seeing a floating window with a person speaking to him — Qualcomm’s software tools for AR glasses extend phone apps to headsets. The Apple Vision Pro bypasses the phone and works on its own.

Qualcomm

Apple is emphasizing that the Vision Pro is a self-contained computer that doesn’t need other devices. That’s understandable, and most of Apple’s cloud services, like FaceTime, will work so that the Vision Pro will essentially absorb most iPhone and iPad features. Yet I don’t understand why iPhones, iPads and Watches wouldn’t be welcome input accessories. Their touchscreens and motion controls could help them act as remotes or physical-feedback devices, in a similar way to how Qualcomm is already looking at the relationship between phones and AR glasses. I hold up my iPhone all the time to enter passwords on the Apple TV. I seamlessly drop photos, links and text from my iPhone over to my Mac.

Touchscreens could act as virtual keyboards. Drawing on the iPad could mirror a 3D art interface. With Apple’s already excellent passthrough cameras, iPhone, iPad and Watch displays could become interactive second screens, tactile interfaces that sprout extra parts in AR. Also, there’s the value of haptics and physical feedback.

Sony Playstation VR 2 virtual reality headset — The PSVR 2 controller: One advantage to physical devices is physical feedback.

James Martin/CNET

No haptics

The buzzing, tapping and rumbling feelings we get on our phones, watches and game controllers, those are feedback tools I’ve really connected with when I go into VR. The PlayStation VR 2 even has rumbling feedback in its headset. The Vision Pro, with eye and hand tracking, has no controllers. And no haptic feedback. I’ve been fascinated by the future of haptics — I saw a lot of experimental solutions earlier this year. For now, Apple is sitting out on haptic solutions for Vision Pro.

When I use iPhones and the Watch, I feel those little virtual clicks as reminders of when I’ve opened something, or when information comes in. I feel them as extensions of my perceptual field. In VR, it’s the same way. Apple’s pinch-based hand tracking technically has some physical sensation when your own fingers touch each other, but nothing will buzz or tap to let you know something is happening beyond your field of view — in another open app, for instance, or behind you in an immersive 3D environment.

Microsoft made a similar decision with the HoloLens by only having in-air hand tracking, but former AR head Alex Kipman told me years ago that haptics were part of the HoloLens roadmap.

Apple already has haptic devices; the Apple Watch, for example. All those iPhones, too. I’m surprised the Vision Pro doesn’t already have a solution for haptics. But maybe it’s also on its roadmap?

logitech-vr-ink-lifestyle-final — Logitech’s VR Ink, released in 2019, is an in-air 3D stylus. How will Apple handle creative tools in 3D?

Logitech

Will there ever be other accessories like the Pencil?

One of the wildest parts about a mixed-reality future is how it can blend virtual and real tools together, or even invent tools that don’t exist. I’ve had my VR controllers act like they’re morphing into objects that feel like they’re an extension of my body. Some companies like Logitech have already developed in-air 3D styluses for creative work in VR and AR.

Apple’s Vision Pro demos didn’t show off any creative apps beyond the collaborative Freeform, and nothing that showed how 3D inputs could be improved with handheld tools.

Maybe Apple is starting off by emphasizing the power of just eyes and hands here, similar to how Steve Jobs initially refused to give the iPad a stylus. But the iPad has a Pencil now, and it’s an essential art tool for many people. Dedicated physical peripherals are helpful, and Apple has none with its Vision Pro headset (yet). I do like VR controllers, and Meta’s clever transforming Quest Pro controllers can be flipped around to become writing tools with an added stylus tip. As a flood of creative apps arrive on the Vision Pro in 2024, will Apple address possibilities for dedicated accessories? Will the Vision Pro allow for easy pairing of them? Hopefully, yes.

The Apple Vision Pro is a long way from arriving, and there’s still so much we don’t know. As Apple’s first AR/VR headset evolves, however, these key aspects should be kept in mind, because they’ll be incredibly important ways to expand how the headset feels useful and flexible for everyone.

Technologies

TMR vs. Hall Effect Controllers: Battle of the Magnetic Sensing Tech

The magic of magnets tucked into your joysticks can put an end to drift. But which technology is superior?

Competitive gamers look for every advantage they can get, and that drive has spawned some of the zaniest gaming peripherals under the sun. There are plenty of hardware components that actually offer meaningful edges when implemented properly. Hall effect and TMR (tunnel magnetoresistance or tunneling magnetoresistance) sensors are two such technologies. Hall effect sensors have found their way into a wide variety of devices, including keyboards and gaming controllers, including some of our favorites like the GameSir Super Nova.

More recently, TMR sensors have started to appear in these devices as well. Is it a better technology for gaming? With multiple options vying for your lunch money, it’s worth understanding the differences to decide which is more worthy of living inside your next game controller or keyboard.

How Hall effect joysticks work

We’ve previously broken down the difference between Hall effect tech and traditional potentiometers in controller joysticks, but here’s a quick rundown on how Hall effect sensors work. A Hall effect joystick moves a magnet over a sensor circuit, and the magnetic field affects the circuit’s voltage. The sensor in the circuit measures these voltage shifts and maps them to controller inputs. Element14 has a lovely visual explanation of this effect here.

The advantage this tech has over potentiometer-based joysticks used in controllers for decades is that the magnet and sensor don’t need to make physical contact. There’s no rubbing action to slowly wear away and degrade the sensor. So, in theory, Hall effect joysticks should remain accurate for the long haul.

How TMR joysticks work

While TMR works differently, it’s a similar concept to Hall effect devices. When you move a TMR joystick, it moves a magnet in the vicinity of the sensor. So far, it’s the same, right? Except with TMR, this shifting magnetic field changes the resistance in the sensor instead of the voltage.

There’s a useful demonstration of a sensor in action here. Just like Hall effect joysticks, TMR joysticks don’t rely on physical contact to register inputs and therefore won’t suffer the wear and drift that affects potentiometer-based joysticks.

Which is better, Hall effect or TMR?

There’s no hard and fast answer to which technology is better. After all, the actual implementation of the technology and the hardware it’s built into can be just as important, if not more so. Both technologies can provide accurate sensing, and neither requires physical contact with the sensing chip, so both can be used for precise controls that won’t encounter stick drift. That said, there are some potential advantages to TMR.

According to Coto Technology, who, in fairness, make TMR sensors, they can be more sensitive, allowing for either greater precision or the use of smaller magnets. Since the Hall effect is subtler, it relies on amplification and ultimately requires extra power. While power requirements vary from sensor to sensor, GameSir claims its TMR joysticks use about one-tenth the power of mainstream Hall effect joysticks. Cherry is another brand highlighting the lower power consumption of TMR sensors, albeit in the brand’s keyboard switches.

The greater precision is an opportunity for TMR joysticks to come out ahead, but that will depend more on the controller itself than the technology. Strange response curves, a big dead zone (which shouldn’t be needed), or low polling rates could prevent a perfectly good TMR sensor from beating a comparable Hall effect sensor in a better optimized controller.

The power savings will likely be the advantage most of us really feel. While it won’t matter for wired controllers, power savings can go a long way for wireless ones. Take the Razer Wolverine V3 Pro, for instance, a Hall effect controller offering 20 hours of battery life from a 4.5-watt-hour battery with support for a 1,000Hz polling rate on a wireless connection. Razer also offers the Wolverine V3 Pro 8K PC, a near-identical controller with the same battery offering TMR sensors. They claim the TMR version can go for 36 hours on a charge, though that’s presumably before cranking it up to an 8,000Hz polling rate — something Razer possibly left off the Hall effect model because of power usage.

The disadvantage of the TMR sensor would be its cost, but it appears that it’s negligible when factored into the entire price of a controller. Both versions of the aforementioned Razer controller are $199. Both 8BitDo and GameSir have managed to stick them into reasonably priced controllers like the 8BitDo Ultimate 2, GameSir G7 Pro and GameSir Cyclone 2.

So which wins?

It seems TMR joysticks have all the advantages of Hall effect joysticks and then some, bringing better power efficiency that can help in wireless applications. The one big downside might be price, but from what we’ve seen right now, that doesn’t seem to be much of an issue. You can even find both technologies in controllers that cost less than some potentiometer models, like the Xbox Elite Series 2 controller.

Caveats to consider

For all the hype, neither Hall effect nor TMR joysticks are perfect. One of their key selling points is that they won’t experience stick drift, but there are still elements of the joystick that can wear down. The ring around the joystick can lose its smoothness. The stick material can wear down (ever tried to use a controller with the rubber worn off its joystick? It’s not pleasant). The linkages that hold the joystick upright and the springs that keep it stiff can loosen, degrade and fill with dust. All of these can impact the continued use of the joystick, even if the Hall effect or TMR sensor itself is in perfect operating order.

So you might not get stick drift from a bad sensor, but you could get stick drift from a stick that simply doesn’t return to its original resting position. That’s when having a controller that’s serviceable or has swappable parts, like the PDP Victrix Pro BFG, could matter just as much as having one with Hall effect or TMR joysticks.

Technologies

Today’s NYT Connections: Sports Edition Hints and Answers for Feb. 18, #513

Here are hints and the answers for the NYT Connections: Sports Edition puzzle for Feb. 18, No. 513.

Looking for the most recent regular Connections answers? Click here for today’s Connections hints, as well as our daily answers and hints for The New York Times Mini Crossword, Wordle and Strands puzzles.

Today’s Connections: Sports Edition has a fun yellow category that might just start you singing. If you’re struggling with today’s puzzle but still want to solve it, read on for hints and the answers.

Connections: Sports Edition is published by The Athletic, the subscription-based sports journalism site owned by The Times. It doesn’t appear in the NYT Games app, but it does in The Athletic’s own app. Or you can play it for free online.

Read more: NYT Connections: Sports Edition Puzzle Comes Out of Beta

Hints for today’s Connections: Sports Edition groups

Here are four hints for the groupings in today’s Connections: Sports Edition puzzle, ranked from the easiest yellow group to the tough (and sometimes bizarre) purple group.

Yellow group hint: I don’t care if I never get back.

Green group hint: Get that gold medal.

Blue group hint: Hoops superstar.

Purple group hint: Not front, but…

Answers for today’s Connections: Sports Edition groups

Yellow group: Heard in «Take Me Out to the Ball Game.»

Green group: Olympic snowboarding events.

Blue group: Vince Carter, informally.

Purple group: ____ back.

Read more: Wordle Cheat Sheet: Here Are the Most Popular Letters Used in English Words

What are today’s Connections: Sports Edition answers?

The yellow words in today’s Connections

The theme is heard in «Take Me Out to the Ball Game.» The four answers are Cracker Jack, home team, old ball game and peanuts.

The green words in today’s Connections

The theme is Olympic snowboarding events. The four answers are big air, giant slalom, halfpipe and slopestyle.

The blue words in today’s Connections

The theme is Vince Carter, informally. The four answers are Air Canada, Half-Man, Half-Amazing, VC and Vinsanity.

The purple words in today’s Connections

The theme is ____ back. The four answers are diamond, drop, quarter and razor.

Technologies

Today’s NYT Mini Crossword Answers for Wednesday, Feb. 18

Here are the answers for The New York Times Mini Crossword for Feb. 18.

Looking for the most recent Mini Crossword answer? Click here for today’s Mini Crossword hints, as well as our daily answers and hints for The New York Times Wordle, Strands, Connections and Connections: Sports Edition puzzles.

Today’s Mini Crossword is a fun one, and it’s not terribly tough. It helps if you know a certain Olympian. Read on for all the answers. And if you could use some hints and guidance for daily solving, check out our Mini Crossword tips.

If you’re looking for today’s Wordle, Connections, Connections: Sports Edition and Strands answers, you can visit CNET’s NYT puzzle hints page.

Read more: Tips and Tricks for Solving The New York Times Mini Crossword

Let’s get to those Mini Crossword clues and answers.