Designing the Mind Gamers experience
Technology was always viewed as intrinsic to storytelling. It’s time for another leap.
What does immersion mean to you?
We’ve grappled with this question (and those in its orbit) for some time now; how one can use art and design to completely and definitively suspend reality. Questioning the line between belief and surety; augmentation and manipulation; transportation and tranformation. Now peel back a layer: what role does technology play: Accelerant? Crutch? A means by which to design new dreams, or an excuse to stop dreaming altogether? Earlier this year we collaborated with music supergroup Mind Gamers, new-media artist Vincenzi and the teams at Facebook to find an answer, bringing our collaborative ethos to life in search of the question.
Daniel Stricker was 12 years old when he experienced virtual reality for the first time; watching an experimental boxing film in single-digit polygons at the Sydney Convention Centre. “All I could think at the time was ‘this is going to be amazing’ he says. “It became this mythological, magical piece of tech with the deepest ramifications because you could be transported to another world.” Already keenly interested in music as its own form of world-building, the early-era VR experience opened up a visual world of possibility, though it wasn’t until decades later he could act on it.
When Stricker found himself at the heart of a new musical supergroup ‘Mind Gamers’ the stage was immediately set for collaboration on both a musical and visual plane. He describes the group, featuring himself, John Carroll-Kirby (Blood Orange, Solange) and iconic french artist Sebastien Tellier, as an always-on, free collaboration with limitless capability, capable of collaborating on a song with Karl Largerfeld as much as creating a film in new-media formats. "It doesn’t feel constrained by anything, it’s just this free, liquid thing” Stricker explains, describing Tellier as “French in the classic sense”, open to any and all crazy suggestions they had.
Mind Gamers have always been interested in pushing boundary's visually, so when film and new-media artist Vincenzi (of creative studio Sensory Systems) approached the trio about filming a new video in 360-degrees, the timing was only right. “Vincenzi wanted to try some stuff” Stricker recalls. “He’d been a big inspiration in terms of how important the visual side of music is, so as we were rehearsing one day he came down with a set of green screens and filmed us. Then he spent about a year developing the piece. I'd been working on immersive experiences with Indigenous Australia, so when Semi Permanent approached me I developed an Augmented Reality experience consisting of the VR film and live dance and music.”
“What’s most alluring to me is music’s transformative ability to shift you from reality into new extended realms” says Vincenzi, who also worked with Simon Ward to bring this VR project to life. “We knew we wanted to start from a world based in reality, and as the song began they would be shifted into a heightened dream realm. It made sense to place this piece in outerspace, where the participants can end the experience literally lost amongst the stars.” As these themes of transportation and transition morphed into the concept of the video, Semi Permanent had been waiting in the wings with Facebook in search of the ideal creative idea to show off their VR capability. After speaking to Vincenzi and Stricker, this project finally felt like a worthy creative opponent; they just needed the right hardware for the job.
6 Jason Juma-Ross is Facebook’s head of technology strategy, a role that brings together media, entertainment and creativity through the lens of technology and hardware — though he sees them as intrinsic to each others success: “All those words just mean technology” he contemplates from Facebook’s Sydney offices. “Media is a technology and a communication channel. Entertainment and creativity are enabled by technology. I’m really interested in those areas and their impact on how we communicate, share, exchange ideas and connect.” With the impending release of the 'Oculus Go', a powerful, lightweight VR headset at a low price point, Ross knew it was the final piece in a constantly expanding puzzle.
By the time Semi Permanent had connected the three, the project took on its own life as a quietly radical experiment in immersive design. At the Semi Permanent Sydney event, viewers would wear a neon-pink poncho (why not) and sit in a cavernous space before putting on the Oculus Go headset and holding on for dear life. As the video nears its conclusion, the viewer would be instructed to remove the headset where a cast of dancers would create the virtual world in front of them (with the help of some cosmic costuming and a lot of rubber tubing). "“[It] was interesting to explore was the transition from the virtual world back into reality once again" says Vincenzi. "Some might liken a VR headset to a glorified blindfold. When the participants finally remove [their headsets], they see the set around them has been transformed and the surreal figures from the virtual world are still dancing around them. It was a fun way to play with that disconcerting transition period back to reality."
“We talk about the paradigm changing from ‘storytelling’ to ‘storyliving’, where you’re a part of the art rather than a passive observer. It’s a fascinating dynamic."
Jason Juma-Ross, Facebook
“The next challenge was in ensuring everyone in the room had the same experience because typically these are standalone, independent devices that aren’t experienced collectively” Ross explains. “We had to find a technical and creative solution that coordinated the experience.” Like any new technology, the role VR played in this film presented its own challenges to the story (and by extension, the paradigm of immersion itself). “It isn’t cinematography in the traditional sense” Stricker says. “We can’t say ‘OK, we’re going to show this frame and tell that story’. You’re dropping someone in a completely new environment then guiding them along a path. VR takes it to another level because you’re injecting someone into a place, like a straight-into-the-veins, all-the-senses-at-once experience.” Vincenzi agreed that to push the sense of immersion and transportation, the viewer had to be in a constant state of motion without moving themselves: “We had to start this experience off slowly before we added in the more excessive visual stimuli. It’s easy to make participants feel ill if you wish to just speed them through a slap-dash rollercoaster.”
The Semi Permanent experience arrived as the technology, art and collaborators were at the top of their collective game. “We’ve been enthusiastic partners with Semi Permanent for a few years” Ross says, highlighting the growing excitement at Facebookaround VR as a creative expression as well as faster, more lightweight technology like Oculus Go becoming available at lower cost. “It’s important to work confidently with the technology available to you at a particular period of time” adds Vincenzi, while Stricker briefly mentions that they’re working again with Facebook around new and improved prototypes for deeper experiences.
The delineated path of technology as a force for storytelling is not a new concept, but one we continue to improve over time. “Virtual reality is a technological evolution of immersion that includes everything from text messages to photography, video, and what will now be known as augmented/mixed reality” Ross concludes. “These are richer and more immediate ways for us to share our experiences, so I think about this continuum of human psychology in an evolutionary way as opposed to just a series of step-changes. We’re also currently creating various 6DOF (six degrees of freedom) experiences using games engines, photogrammetry, LIDAR sensors and other toys, but ultimately they’re just a progression of tools to further storytelling and communication.”
Another aspect which was interesting to explore was the transition from the virtual world back into reality once again.
Vincenzi
But what is the next leap on the question for true immersion? The soft-centre of permeance is what has kept Stricker’s attention on technology all the way since that chance encounter at the Sydney Convention Centre: “It’s about working in emotion and empathy” he adds, before hopping on a flight back to Paris after a rapid 24-hour stay in Sydney. “If you can make experiences that move people — whether through music, art, VR or live performance — that’s one of the last forms of magic we have.” Vincenzi is equally excited about the far-future, though cautiously optimistic about the methods by which we get there. “Ultimately, the endgame is quite frightening” he says, “if it means future societies decide virtual worlds are more stimulating and worthwhile than the real world. But until we get there, it’s exciting to explore the potential of these realms in a creative sense. I still love seeing people use programs [like] Tilt-Brush for the first time. It really opens ones mind to creativity in new dimensions. I’m sure I wouldn’t be able to comprehend the virtual worlds created in 100-years. People think we’re living in a simulation anyway, so we might as well dive on in.”
All photos by Toby Peet
"MindGamers — Come to America" VR/360 Stereoscopic Production
Production Company — Sensory Systems
Director/Producer: Vincenzi
CGI/Co-Producer: Simon Ward
VR Camera: Brandon D’Silva
Camera/Lighting: Ryan Alexander Lloyd
Gaffer: Ben Morgan
Assistant: Anthony Rilocapro
www.sensory.systems
Production Company — Sensory Systems
Director/Producer: Vincenzi
CGI/Co-Producer: Simon Ward
VR Camera: Brandon D’Silva
Camera/Lighting: Ryan Alexander Lloyd
Gaffer: Ben Morgan
Assistant: Anthony Rilocapro
www.sensory.systems
“Mind Gamers x Oculus” Semi Permanent BTS video.
Production Company — Sensory Systems
Director/Producer: Vincenzi
Co-Producer: Anthony Miller
Camera: Jordan Maddocks
Camera Assist: Charlie Slessar
VFX: Studio Pancho
Co-Producer: Anthony Miller
Camera: Jordan Maddocks
Camera Assist: Charlie Slessar
VFX: Studio Pancho