Quest Augments

Disclaimer: details on this page are restricted to publicly known information. See the list of mentions for sources.

Announced at Meta Connect 2023, Augments were applications you could place around your physical space that stayed put between sessions on Meta Quest VR devices. Using the mixed reality capabilities of Meta Quest 3, an Augment - a clock, a photo frame, a virtual pet, a game board - could be anchored to a wall, a piece of furniture, or anywhere in your room, and it would remain there until the next time you put the headset on.

This was a new way to interact with Quest devices. Originally purely Virtual Reality devices - where the world around you is blocked out - they were now being used for Augmented Reality, through the device’s enhanced camera passthrough, for the first time.

The result was Augments. In practice, they worked like persistent mini-apps sitting around you: they could be moved and resized, they were interactive, they didn’t take over your whole view, and they could often be used to launch full experiences, such as VR games. In a way, they were a lot like the “widgets” on Android or some desktop operating systems, but in 3D.

Augments could be built using several internal platforms, including Spark AR/AR Studio and a heavily expanded internal version of ReactVR called “ReactAR”.

I spent my time working on a number of different Augments that we were going to launch for Quest 3: virtual picture frames for Asgard’s Wrath 2, a POPULATION: ONE virtual cockpit, virtual “destination” posters and cute instructor Bobbleheads for Supernatural, decorative shortcut items for Beat Saber and SUPERHOT VR, a launcher for new user onboarding, and a number of social widgets - leaderboards, group rankings - that any VR game could use.

In total, 46 different “augments” were created as part of this project. For most of these, the original work (assets and core 3D functionality) was created by separate partner teams, and our team’s work was to bring them into the “Augmented” space. The work was distributed among a large team of developers, without clear ongoing ownership, so it’s hard to put into words everything that I did for this project. Mostly, my job was to make them work within the system’s Augmented environment, to advise studios on how to produce the best content, and to optimize the performance of each augment. In addition, I led the effort to make all augments work within our automated CI/CD environment, so each could be independently built and released without the need to manage each of the 46 augments individually.

Unfortunately, none of this ever launched. The work was deprioritized while we were still testing it internally, and Quest 3 was never strongly positioned as an AR device. AR Studio itself was shut down shortly after.

Still, this was a project I was very happy to be a part of. It let me (and my team) explore a novel way to interact with applications - whether VR or AR - and work closely with platform teams as we grew and enhanced their capabilities. Another one I’m proud of, even if it never shipped.

Previews

Media

Connect 2023 Keynote Announcement