Technologies

Apple Should Build Its AR on the Meta Quest

Commentary: Its glasses may be on hold, but the immediate future is already in front of us.

Apple’s first-ever VR-AR or «mixed reality» device is expected this year, and already its follow-up product seems to be on hold. According to Bloomberg’s reliable Mark Gurman, a planned set of AR glasses isn’t coming from Apple anytime soon. That’s OK: AR glasses don’t seem like they’re imminent from anyone.

After a trip to Las Vegas to try out some of the emerging technologies around future VR and AR headsets, one thing is increasingly clear: Everyone’s trying to figure out AR glasses, but everyone’s trying to perfect VR headsets. The difference between those two scenarios is bigger than you think.

All-day AR glasses that are useful, work convincingly, have long enough battery life, work with your phone and function as actually legit prescription glasses haven’t materialized, although the pieces are coming into place. Companies like Meta have promised a decade-long path to these glasses. It turns out, maybe, that’s going to be the case for everyone else, too. I’ve seen smart glasses that look real but don’t do much, or glasses with AR that feel chunky and do some things, but don’t quite work with my vision and can’t figure out how to work with my phone yet. Chipmaker Qualcomm is working on this; Google, Apple and Samsung need to solve it someday, too.

In the meantime, VR already has a very real and reasonably popular product that most families I know in the everyday world are familiar with: the Meta (formerly Oculus) Quest 2.

That recognition is no small thing. I think of the Quest’s place in everyday life like the Amazon Echo was years ago: something odd that over time became familiar, normalized. Something reasonably priced, and good enough to do a few things actually well. The Quest 2 is basically a game console. Where Meta has struggled is figuring out how to expand that base beyond gamers.

Following Meta’s playbook is something I expected Apple would do. Heck, I expect most companies are going to do it. The Quest 2 works just like most people imagine it will, or better. It’s a bit of instant magic that’s totally wire-free.

The Quest 2 does have downsides. In fact, those problems emerge the more you use it. I find connecting with friends and social spaces gets weird and buggy, prone to lag, disconnects and way-too-basic avatars. The battery life is bad. For fitness apps, which the Quest 2 can do surprisingly well, it’s still not good at really managing sweat or keeping my glasses from fogging.

Even though Meta wants the Quest 2 and higher-end, work-focused Quest Pro to open up new ways to work by creating virtual monitors around my laptop, the connections and display quality aren’t good enough to be more than a clever experiment most of the time. I can see my laptop keyboard with the Quest Pro’s passthrough cameras, but typing feels awkward and nowhere near as good as when I’m just on my laptop… and I can’t see my phone screen to check messages, either. The Quest can show me phone notifications like a basic smartwatch from a decade ago, but I can’t interact with them.

These VR headsets can even do some basic AR, using passthrough cameras that «mix reality» to show the real world in fuzzier video, with VR layered on top. The effect is sometimes pretty amazing, and could even approach feelings I’ve had with early AR headsets like the Microsoft HoloLens 2.

So where does that put Apple? Clearly, there’s a headset coming soon. And according to Bloomberg’s Gurman, the next goal after this first expensive headset is to work on a more affordable model. It’s like Meta’s approach to the Quest and Quest Pro, in reverse. And there are plenty of things Apple could focus on to make its entry into VR (and AR) worth the effort.

Better comfort, better fitness

The Quest 2 is already an affordable fitness device, and pairs with watches to show heart rate and fitness stats. Apple clearly has an advantage on time spent developing the Apple Watch, fitness and health tracking, and its Fitness Plus subscription video workouts, which also have overlaid fitness stats.

Apple could emphasize workouts and fitness on its headset, with comfortable, breathable straps and face pieces that could feel better for exercise. Meta is starting to realize it needs to improve comfort for VR: A recent Razer partnership using head straps made by CPAP-maker ResMed shows a need for better materials. I’d expect Apple to make this aspect a key part of the headset’s advantages. There are other advantages, too. Apps like Beat Saber and Supernatural use music for fitness, and Apple already has all of Apple Music at its disposal.

Connect better with laptops, iPads, phones, watches

VR headsets right now have an extremely hard time working well with all the other things we have lying around us. I can’t get a Quest to connect nicely with my phone all the time. To work with my laptop, I need a specialized third-party app with its own thing I have to install on my laptop and turn on.

Meanwhile, Apple has been focusing on handoffs and continuity across AirPods, HomePods, iPhones, Apple Watches, MacBooks, Apple TVs… all over the place. That’s what’s needed to make a VR headset seem seamless and integrated into other stuff. I want to check my watch in VR, or use it to control apps. Or use my phone, and also see the phone. Suddenly grab my laptop, and the headset connects. Incoming calls? No problem. Send myself things back and forth from my phone or laptop and get all the files and things I want, and not feel like I’m on a vacation from them. That’s what Apple’s headset could set out to achieve.

That’s a best-case scenario. Much like the first Apple Watch and iPhone, the actual Day 1 functions of this headset might end up disappointing.

Better social

Even though the metaverse is on everyone’s minds, there aren’t many big social spaces in VR that work well. Microsoft’s Altspace is nice, but often feels empty. VRChat is wild, experimental, full of big features and ideas, and feels like a messy explosion that’s hard to jump into. Meta can’t get enough people into Horizon Worlds. Even when these platforms do work, for concerts or events, the limits on people who can attend at once, the lag and drop-off, not to mention the avatar limits, make it a trade-off versus any other way you could connect on a phone or laptop.

Apple may not be able to solve this any better for larger-scale experiences, but for more intimate and several-person FaceTime-like moments, Apple could make shared experiences in VR work a lot better. Meta hasn’t perfected social VR yet, and someone needs to.

Can Apple make a better controller? (Or none?)

The Quest 2 controllers are fine, but all of VR leans on the same game controller-like inputs for headsets. Apple’s headset could lean more on hand tracking, or wearable inputs like the Apple Watch. I’m curious if a more work-oriented controller or accessory can be created that makes the headset feel better for taking on apps beyond games. Meta’s working on a long-term, game-changing shift to neural input wristbands eventually, but it’s unclear whether this approach will end up succeeding.

The Quest platform has continually improved its hand tracking over the years. However, hand tracking’s reliance on particular gestures without any physical feedback is an imperfect solution right now. Maybe Apple tries hand tracking along with using an Apple Watch or the iPhone for tactile haptic feedback, or finds a smaller go-between accessory.

I’ve been trying out experimental haptic technology recently, trying to imagine how VR could think its way to new inputs. This headset feels like the biggest opportunity Apple’s ever faced to create a brand-new type of input device that could make a big impact on the landscape. If it’s done right, maybe it’ll be the input accessory that makes its future AR glasses, whenever they arrive, seem feasible.

Build out more interesting mixed reality

For all the Meta Quest Pro promises to blend AR and VR with its mixed-reality capabilities, not many apps tap into its extras yet. I’ve seen some mind-blowing demos of mixed reality in VR with the ultra-high-end Varjo XR-3 connected to a PC, which at least showed me ways that a VR headset could begin to feel like a portal interconnected to my own home reality. Apple could start experimenting with more engaging AR moments in a high-end VR headset, and at least get the ball rolling on things that work in advance of whenever its AR glasses are ready, years from now.

Smaller sessions in VR may make more sense right now

VR is a thing I don’t use all the time, and that’s true for most people. Maybe that’s exactly where Apple should start. It’s not a given we’ll want to wear AR glasses everywhere, or even what those glasses would be good for. In the meantime, a VR headset at home that’s meant to be worn sometimes, but not all the time, is the place most of us feel safest to start. It’s why the Quest is something people actually use.

It’s also a way to avoid dealing with questions of accommodating true prescription vision needs in everyday glasses, something no one’s succeeded in tackling, either. VR headsets sometimes need prescription inserts, but many just fit right over the glasses we already have. I prefer the easy-fit solution: I don’t need to make VR a thing I spend a whole day in. I’ll settle for a truly useful hour or two, and if Apple can make that hour or two even better than what we have now, that’s a big enough step forward for me.

Editor’s note, Jan. 20: Adds mention of Meta’s hand tracking for the Quest.

Technologies

Here’s How a Former Overwatch Pro Made the Support Hero He Always Wanted

Scott «Custa» Kennedy used his experience as a former Overwatch League pro to design one of the game’s most popular heroes in Reign of Talon season 1.

Overwatch’s Reign of Talon season 1 is starting to wind down, and the biggest story has been the five new heroes who joined the roster. A lot of attention has understandably gone to Jetpack Cat, a hero once scrapped in the game’s early design, but resurrected on the cusp of the game’s 10th anniversary. She’s been the subject of bans and memery due to her unique kit that features permanent flight and the ability to fly any other hero through the air with her Lifeline ability.

But another support hero has quietly gone under the radar as one of the most-played characters in the new season: Mizuki.

Mizuki is a complex hero, similar on paper to support heroes Brigitte and Lucio, who mix damage with healing in the radius around them, but with his own unique mechanics. He has a constant healing aura around him, which grows more powerful as he deals damage with his weapon or uses other healing abilities. His main weapon is a projectile that bounces off surfaces. One of his abilities, Katashiro Return, offers a burst of movement, but also the ability to teleport back to your starting point within a few seconds.

That all adds up to a hero design that gives players lots of options but also requires you to carefully strategize to turn the tide of battle. Do you stay with your team to maximize the value of your healing aura? Or do you split from them for a higher-risk, higher-reward play? Do you use your Katashiro Return ability to flank behind an enemy team, or save it to disengage from an unexpected attack?

Despite spending most of my time in Overwatch playing support heroes, including Ana and Kiriko, I found Mizuki challenging early in the season, even as I watched enemy Mizukis pump out damage and secure clutch kills while constantly healing their teams.

This «unlockable challenge» element was an intentional part of Mizuki’s design, as I was soon to learn from chatting with the hero’s creator, a former Overwatch eSports pro.

By and large, support players have embraced this challenge. An Overwatch spokesperson told me via email that «Mizuki is consistently in the top four for all support picks in Season 1, across every region.» He’s one of several elements powering a revival of the game, along with a new ongoing story, weekly faction missions and the promise of more new heroes every season. People have flocked back into the game since the start of Season 1, with its average player count on Steam more than doubling over the past month.

Mizuki’s design was led by Scott «Custa» Kennedy, a longtime presence in Overwatch’s professional scene as both a player and match analyst, and now an associate hero designer. I spoke with Scott at Blizzard’s spotlight event and also spoke with him and Mizuki’s character artist, Melissa Kelly, in early March to discuss how they created one of the game’s most popular heroes.

From professional player to associate designer

After a few years as a professional player and several more as an analyst and caster for the Overwatch League, Kennedy was looking for the next step in his career.

«Overwatch [had] been my life for, like, the last 10 years in many different facets,» he said, but as he reached retirement age in the esports realm, he wanted a change. He spoke with some of the Overwatch developers, including associate game director Alec Dawson, about what it would take to get into game development.

After doing some QA work and hands-on game development («I made the world’s hardest 2D cat platformer in three days,» he said), Kennedy secured an associate hero designer opening for Overwatch, which was a perfect fit with his experience.

When given the task of envisioning the game’s next healer, Kennedy said he didn’t want to make another support designed around «point-and-shoot» mechanics that healed teammates and hurt enemies, like Ana or Juno.

«I wanted [Mizuki] to be more of an AoE healing aura-type hero because I think that’s something that’s been underrepresented in our heroes,» Kennedy said. Instead, he came up with the area-of-effect healing that’s similar to how Lucio and Brigitte heal, but with the added layer of that healing becoming more powerful the better you play in combat.

Managing that nuance was a learning experience for Kennedy.

«One of the biggest things that I learned is how complexity can be really cool on paper, but when you’re making a hero how quickly that snowballs into making a player overwhelmed,» Kennedy said. But he feels the team ultimately found a good balance, where inexperienced players can still contribute with him, while more experienced and skilled players can benefit even more.

Kelly added that Mizuki was a complicated hero on the design side, too.

«One of the issues is that he was looking kind of like a [damage hero],» she said. «He looked very aggressive for a healer. So we were just trying to soften him up.» Kelly pointed out that Mizuki’s weapon is a mix of a priest’s staff and a sickle, which also blurs the lines a bit between support and damage heroes.

That nuance seems to be a big part of Mizuki’s appeal. Even though I generally prefer the kind of «point-and-shoot» healing hero Kennedy said he wanted to avoid, I’ve found Mizuki to be one of the most interesting additions to the roster, especially among support heroes. His Binding Chain ability, which roots an enemy hit by the chain into place, rewards good aim and timely use, while his Healing Kasa and Katashiro Return abilities let my brain ponder over creative escapes and ambushes.

When I play Mizuki, I’m always thinking while I fight, and I enjoy feeling that kind of active engagement with the game.

Mizuki’s reception and prospects for pro play

Kennedy worried that players would be turned off by how complex the hero is — wondering, «Are players going to try him, not understand him and then be like… ‘I’m just gonna play the cat?'» (The cat, of course, is Jetpack Cat, who was released alongside Mizuki in season 1 and immediately became one of the most popular and most-banned heroes. She has a more intuitive, point-and-heal design, although her launch state also allows for particularly aggressive gameplay.)

Instead, Kennedy has enjoyed watching players stick with Mizuki and later post about how they’ve «unlocked» the hero by figuring out the formula to succeed with him. Kennedy said it’s rewarding to see players grasp his original concept for the hero as it plays out in-game. After that initial, somewhat disastrous first game I played, I started clicking with Mizuki, too.

Players still struggled with parts of Mizuki’s kit, and Kennedy noted some initial frustrations with «intentional design limitations» he and the team placed on the hero. Players seemed to want to use his Katashiro Return ability to go on aggressive flanks, but found it didn’t last long enough to successfully move behind enemy teams. That kind of larger repositioning would go against the design team’s vision for the hero, who is meant to stay near his team and use the ability to return to them quickly.

Now, Kennedy said, «players seem to understand the limitations of the hero, and that’s been cool to see.»

Mizuki has had a strong launch, and has been sitting around a 54% win rate in competitive modes since the start of the season. That’s quite high, ranking just behind last season’s top performer: the damage hero Vendetta. I asked Kennedy how he reads that data — whether Mizuki is overtuned or just a good fit among this season’s most-played heroes.

Kennedy said Mizuki was in a «pretty healthy» spot, but could get pulled down a bit in future seasons. «The numbers that he can put out in terms of healing and damage output are things that really put him above everyone else at this point. So it’s definitely something we’re keeping an eye on.»

But that power won’t necessarily translate to Mizuki being picked up in professional play, at least based on last month’s Overwatch Championship Series Bootcamp. Kennedy said the hero’s kit isn’t as good for staying alive and executing plays as heroes such as Lucio and Kiriko, who have long been must-picks in pro play.

«I could see Mizuki getting more playtime in a world in which we start playing more rush metas [centered around tanks like Ramattra or Orisa],» he said, «but with how fast the game is being played at the highest level, it can be difficult for Mizuki to keep up.»

Kennedy brought up one of Overwatch’s biggest and most inevitable challenges over its decade-long tenure: balancing heroes for both the pro level and the rest of the game, and how the difficulty lies in the fact that certain resources — such as speed boosts, mobility and burst damage — are more valuable at the highest levels of coordinated play. The design team is always working to make sure heroes are never totally out of balance at either skill level, he said.

That work has been on display since the launch of Season 1, with balance patches coming out virtually every week up through the midseason patch on March 10. Those updates mostly focused on the five new heroes but also included some changes to Vendetta, who continues to terrorize the game with a very strong win rate and the ability to cut someone down almost out of nowhere, leaving opponents very little time to react.

Still, the season overall has been a win for the game, thanks largely to the influx of new heroes and the different playstyles they add to the game.

«[I’m] definitely a little overwhelmed with how positive everyone has been with Mizuki — and honestly, the five heroes in general,» Kennedy said. «I think the reception’s been awesome. We couldn’t have asked for anything better.»

Technologies

Amazon’s Spring Sale Just Added a Ton of Gaming Deals. Here Are Our Favorites, Expiring Soon

Technologies

Today’s NYT Connections: Sports Edition Hints and Answers for March 30, #553

Here are hints and the answers for the NYT Connections: Sports Edition puzzle for March 30 No. 553.

Looking for the most recent regular Connections answers? Click here for today’s Connections hints, as well as our daily answers and hints for The New York Times Mini Crossword, Wordle and Strands puzzles.

Today’s Connections: Sports Edition is a tough one. You’ll need to know a little about four very different sports in order to solve it. If you’re struggling with today’s puzzle, read on for hints and the answers.

Connections: Sports Edition is published by The Athletic, the subscription-based sports journalism site owned by The Times. It doesn’t appear in the NYT Games app, but it does in The Athletic’s own app. Or you can play it for free online.

Read more: NYT Connections: Sports Edition Puzzle Comes Out of Beta

Hints for today’s Connections: Sports Edition groups

Here are four hints for the groupings in today’s Connections: Sports Edition puzzle, ranked from the easiest yellow group to the tough (and sometimes bizarre) purple group.

Yellow group hint: Make a racket.

Green group hint: Goooooal!

Blue group hint: Baseball stars.

Purple group hint: Toss the pigskin.