Touching the music

_

Sector_
Entertainment
Services_
XR Development
Scope_
Product Design & Development
Technology_
Lumin SDK
,
Unity
Facts_

What was the problem?_

There was no real problem, just a curiosity about how Mixed Reality can impact Entertainment and how it will shape the way people have fun and how we can explore different senses together. Since we are a part of Factory Berlin Community and friends with their Creators Lab and Artists in Residence, we were asked to join the beautiful collaboration in Berlin’s pop-up club Night Embassy by Jagermeister. We had a lot of fun with some great Artists like Rania Kim, known as PortraitXO while designing and developing the Mixed Reality audiovisual experience based on the Magic Leap, that allows users to manipulate AI-generated audio samples with hand gestures. We call this experience “touching the music” as you can simply mix and scratch the sounds simply by touching the 3D objects



What we have done_

We were asked to join this new project to connect all the dots, including designing and developing the experience that was Proof of Concept. We had very limited time, only 2 weeks, to make it all happen. The Artists had given us the 3D objects, AI-generated sounds and voices, and we wrapped it up in Magic Leap, to allow people to explore how the senses can mix, and perceive the music in a Mixed Reality. We have used some of our Unity capabilities, along with expertise on Lumin SDK, and together we have launched a beautiful experience and brought it for the people. For all of us, it was proof that the power of Mixed Reality and its future potential can bring technology closer to the people and show how it can re-shape entertainment and art.


What we have learned_

The whole project was one big learning experience, something we have never done before. It was a priceless experience putting together 3D objects, and AI-generated sounds into Mixed Reality, and most of all, observing the people's reaction to it. Of course, once again, we have learned about the current limitations of Magic Leap Headset. One of the problematic areas is QA testing, especially in dark rooms and space without enough light. But, more importantly, the collaboration with the artists and bringing the experience to the party people, made us aware of the true power and potential behind the Mixed Reality, in everyday life and entertainment. It was a great learning experience and a journey for us.



Achievements

The whole project was one big learning experience, something we have never done before. It was a priceless experience putting together 3D objects, and AI-generated sounds into Mixed Reality, and most of all, observing the people's reaction to it. Of course, once again, we have learned about the current limitations of Magic Leap Headset. One of the problematic areas is QA testing, especially in dark rooms and space without enough light. But, more importantly, the collaboration with the artists and bringing the experience to the party people, made us aware of the true power and potential behind the Mixed Reality, in everyday life and entertainment. It was a great learning experience and a journey for us.



Heading
Heading
Heading

It's been a dream to work with such a talented team of people at nomtek

Rania Kim (PortraitXO)
Music Artist & Creative Director