How to Build a Mixed Reality Experience in 6 Weeks

When our product strategist Kasia got an email from Wendy Posner, little did she know how the following 6 weeks would shake up her day-to-day activities. An invitation for Nomtek to collaborate on the 2022 Infinity Festival in Los Angeles meant recognition for our team’s experience in augmented reality, but then something dawned on Kasia: nothing short of a “wow effect” would do the job, and with only weeks to prepare the experience that meant only one thing. A grind.

What Is Infinity Festival?

Infinity Festival Hollywood is one of the world's most renowned events where Hollywood collaborates with Silicon Valley giants to explore stories with nuanced interaction between tech, art, and life.

Among the 2022 speaker lineup were VR storyteller Elijah Allan-Blitz, the creator of “Remembering,” starring Brie Larson; Head of LA Labs at Epic Games Connie Kennedy; and VP of Lucasfilm Vicki Dobbs Beck, who spearheads immersive storytelling.

Being invited to an event this size and industry impact is an honor and incredible recognition — one that can only be reciprocated by a presentation of similar magnitude.

Creating the Vision and Concept

The seemingly omnipotent ChatGPT, eerily agile Boston Dynamics robots, or practically self-driving Teslas are incredibly inspiring, but can also be worrying. With so much intelligent technology, we start to question our place in the world and whether the skills we’ve honed over decades of hard work aren’t irreplicable. By machines, at that.

Yond is a careful study of these new technologies wherein we shift the mindset of losing control to retaining your agency. Claiming your agency elicits an empowering feeling of certainty and a positive outlook on the future of humanity. Yond is also a creative exploration of what’s possible with AR and VR in an experience that starts in an artfully crafted physical space that then transforms into mixed reality.

Framing the experience

Yond is meant to be a soul-enriching experience held in a meditative headspace. This world is created from a beautiful Art Deco-inspired pattern Cha Cha that embodies harmonic co-existence.

The Cha Cha pattern served as the main building block for Yond — everything in the 3D world is made of the pattern’s variations.

The pattern was created by Los Angeles-based surface artists collective, Londubh Studio, who pursue their artistic endeavors by writing a never-ending love letter to maximalism and glam. Londubh’s Brynn Gelbard and Lisa Donohoe use surface art to transform generic spaces into thought-provoking experiences on a visceral level.

Setting Up the Team

The goal was to create a "wow effect" and make the experience not only technically and visually stunning but also entertaining and soothing. The team was a cross-functional combo that evaluated the creative and delivery sides of the project. Having a highly collaborative team served three purposes:

• The creative team dove deeper into the storytelling part, the look of the virtual world, and the physical space where the experience would start.

• Other duties on the creative side: dialogues, lighting, colors, proportions, and shades.

• The development team took care of the feasibility aspect of interaction methods.

• Other tasks on the development side: finding the right ratio of the 3D graphics quality to retain performance and consulting with the creative team on the placement and look of visuals for the best experience.

• Because the team worked on multiple elements at the same time, the work would be finished on time, minimizing uncertainties.

Kick-Off

The first brainstorming session helped us narrow down the main requirements for the experience:

1. Smooth onboarding and transition from physical, to augmented, to virtual through a 3D portal.
2. Interaction with the VR experience and the ability to control the elements to further underline human agency.
3. Hand-based interaction method to make the experience more familiar for the user.

Smooth onboarding

We knew that VR sickness can significantly spoil a VR experience. By opting for a gradual introduction of the user into the experience through a 3D portal with a smooth progression, motion sickness can be minimized.

The experience itself was designed in a way that didn’t require users to move a lot (moving the whole body in VR can induce motion sickness). Most of the interactions were limited to hand gestures, which decreased the likelihood of motion sickness and boosted the “claim your agency” aspect.

Possible interactions with 3D objects

Setting up interaction methods in extended reality development is complex and time-consuming. There are many elements and objects to consider for testing and then implementing into the experience. The development team started this part of the work right after kick-off.

Hand-based interaction 

Yond doesn’t require any external controllers for the user to enjoy the experience. Dropping controllers eliminated the need to onboard users and explain how to use them. Instead, each user had a brief onboarding session with the narrator — none other than Jason O’Mara — who explained how to use hands to navigate the virtual world. This also helped increase the sense of having control over technology.

Preparing the Venue for the Event

Since we knew the physical space where Yond would take place had to be in tune with the experience itself, we put in a lot of work in arranging the venue.

Our team arrived a few days before Infinity Festival to put on wallpapers and decorate the room for an optimal viewing experience.

Yond at Infinity Festival 2022

Reception

Over 400 people participated in the experience, most with a “wow” reaction. Of that number, only one person experienced slight discomfort, which was indirectly caused by the VR experience. In fact, Yond was one of the best received experiences during Infinity Festival 2022. And we’re honored to have been nominated for Monolith Awards.

Self-Organizing Teams vs. Corporate Reality in Rapid Execution

The team delivered the experience in six weeks — a feat that can take anywhere from six to twelve months in a traditional corporate setting. Getting various approvals and buy-in can significantly hinder the progress of any creative undertaking. Self-organizing teams, however, act fast and efficiently. They cut the time necessary for multi-stakeholder consultations and instead focus hard on rapid but thoughtful execution.