Documentary Virtual Reality (VR) has finally arrived. Among the 24 projects being exhibited at the 2018 Sundance Film Festival's New Frontier program of experimental media, three exhilarating documentary VR experiences stood out above the rest and showed how, at last, VR's promise is now being fulfilled. The three pieces are: Awavena, Lynette Walllworth's dazzling follow-up to her landmark VR piece Collisions; Zikr: A Sufi Revival, from VR pioneer Gabo Arora and his co-creators, who have demonstrated the first significant social VR documentary experience; and Hero, from the team behind 1979 Revolution, which marks a milestone in the dynamic relationship between audiences, VR and real-world environments.
In multiple ways, these VR works, as well as a number of other New Frontier projects, demonstrate how this young medium is expanding the limits of the documentary form: by incorporating deep collaborations into their work practices; redefining filmmaker/subject relationships; utilizing technology not simply as a means to an end but as an equal partner in the creative storytelling process; and by replacing the passive role of the viewer with a more engaged and active participant in the experience. One can argue that not since the cinema vérité movement of the 1960s has there been a period of such radical transformation in how documentaries are made and the ways that artists are defining the art form.
Awavena
Directed by Lynette Wallworth and produced by Nicole Newnham, Awavena is the second chapter in their VR series (Collisions, the first chapter, won an Emmy). It tells the story of how a Yawanawán woman named Hushahu became the first woman among these indigenous peoples of the Brazilian Amazon to be trained and accepted as a shaman. In Awavena, through an increasingly layered and intimate journey, the viewer enters a series of portals that open onto different story worlds. Beginning with passage into the Amazon, we meet the Yawanawá and learn the details of how Hushahu, as a young girl, first sought to become a spiritual leader but was initially rejected because women, up until that time, were not permitted to take on that role. Later, once he is convinced of her seriousness and senses her special skills, the Yawanawá chief Tata changes his mind and permits her to undergo the arduous spiritual training required of its shamen. We are then invited to experience Hushahu's visioning experience. And finally, we are returned back home.
To great effect, Awavena was able to rely on and take full advantage of cutting-edge technology to create a visual experience that mirrors that of an actual visioning. As amazing as that effect is, more far-reaching consequences may result from how the project has flipped the script on filmmaker/subject relationships. While it is not that unusual for those who have stories to tell to reach out to filmmakers whom they admire and think will do justice artistically to their stories, the situation with Awavena is particularly striking.
The process began as soon as Tashka Yawanawá, chief of the Yawanawán people, finished watching Collisions in 2016. According to Wallworth, "He took the headset off, and he was immediately absolutely clear about the way this tech could be used by them: 'OK, my friend, I can see how this works. It works like a visioning. It opens a portal. It carries you without your body to another place. Colors and sounds are intensified. You meet the elders. You're given a message and then you are returned.'" In telling Wallworth, "We have a message we can send out," Tashka shaped the narrative course for Awavena, established that the tech would serve the story and not the other way around, and provided a roadmap for how the relationship between filmmaker and subject would proceed.
The collaborative relationship continued in October 2016 when the Sundance Stories of Change Lab brought Wallworth and Tashka together to imagine the work and shape the fundamental vision. When she asked, "What does the vision need to say?" he replied, "That everything is alive."
At the Lab, the story emerged of how Tata, the tribe's elder shaman, had trained Hushahu as the first woman shaman. At that point, the intent was for Wallworth and Newnham to come down and film both Tata and Hushahu and to have the 90-year-old Tata send out a vision. However, by the end of the year, Tata became very ill and the Yawanawá had an immediate request that Wallworth come down with recording equipment. Wasting no time, she found two Brazilian VR filmmakers who knew the tribe and asked them to travel right away and put a camera in the room where Tata lay dying.
Wallworth continued, "That's a moment, right? When a community's calling you because someone's dying and says, 'Bring cameras in. This man and this story is important'… the Yawanawá wanted the tech there in the most intimate place." This first scene that was shot, of Hushahu and others tending to Tata in his dying days, turns out to be one of the most powerful in Awavena.
When it came time for Wallworth, Newnham, DP Greg Downing and the rest of the crew to shoot the bulk of the piece, three canoes full of cameras, scanners and other gear were required to capture the Yawanawán's message and re-create it using the latest tech tools. Among the equipment used were special Canon night cameras and a prototype PX 80 LiDAR portable scanner provided by Occipital. The LiDAR, one of only three available in the US at the time, is capable of capturing 300,000 points of data per second in order to render a perfectly accurate version of the Yawanawán forest.
At the time Wallworth sat down with Hushahu, she was familiar with just the bare bones of her story. She knew that Hushahu had been trained as the first woman shaman, but had no idea that no woman had ever been allowed to participate in the visioning ceremony, that the gender division in the Yawanawán culture was so extreme, or that a complete cultural revolution resulted once she was allowed to be a spiritual leader. She also didn't realize the full extent of Tata's role in the story: that he risked his entire spiritual authority by breaking down these gender barriers and that he was derided by his people, especially the men, because it was considered taboo. In the face of these obstacles, he was able to draw from a source of phenomenal inner strength, and, over time, won over the tribe. The ramifications of his decision were profound. Within 12 years, the structure of Yawanawán society had been changed forever.
"This transformation that occurred had happened only 12 years ago," Wallworth explains. "That was stunning to me. We were staying in the house of a woman chief. We were watching the gatherings of men and women chiefs coming together to discuss things. We were experiencing the kind of transformational shift that this breakthrough moment with Hushahu had set into that community. And so, suddenly, I understood: this is what they're sending us."
Once it was established that they would tell the story of Hushahu's visioning, technical challenges still remained about how to best replicate that. One of the most intense sequences in the film is when the viewer is taken, virtually, inside a tree in the Yawanawán forest. In addition, Wallworth wanted to capture the fluorescents that her friend and collaborator Dr. Anya Salih, lead scientist on the project, told her were "absolutely real but not necessarily visible to us." Wallworth continues: "So, Anya came and we purchased blue torches and yellow filters that would reveal the fluorescent colors and looked for a DP who could shoot these. The science should be real; that what we're seeing is correct, but we're seeing it in a way that is not normally visible to us. That was my way of interpreting the Yawanawá visioning."
The tech is a powerful tool, but it is there to translate the vision rather than be a message unto itself. According to Tashka, "Most people in the Amazon come from an oral tradition. When an elder dies, it's like burning a library. The knowledge goes. Now, through technology, we can record and show to the next generation how the Yawanawá is organizing today and connected to our spirituality… Lynette translated the vision through technology rather than through the cup, just as Hushahu translated the vision to Tata through a drawing." Wallworth reiterates that, "The tools aren't what matters. They are transient. What matters is the power of story to connect us to one another, which is one of the messages of this work."
Wallworth's exemplary collaborative relationship with Hushahu, Tashka and the Yawanawá is evident in the clarity of its story and point of view, its vivid and purposeful use of technology, and also in their mutual desires to have this story sent out and received. This relationship also serves as something of corrective to the false idea that VR is some kind of magical machine that imbues its users with an instantaneous and overwhelming sense of empathy for the subjects who are portrayed. Wallworth rejects this impulse: "The work aims not to provoke empathy for the Yawanawá people but is rather a gift from them, to those who will virtually visit their forest and receive this transmission — a gift that they hope can shift our consciousness."
Zikr: A Sufi Revival
While Awavena fully engages the user with its dazzling visuals and the power of its almost mythical story, it is, for the most part, a relatively passive viewing experience. By contrast, Zikr: A Sufi Revival largely depends on the social interactions, movements and voices of its participants. The piece itself is not complete until you and three other people join in. With each person given a headset and controller, you are not just a participant but a performer.
While Chris Milk and Aaron Koblin's Life of Us at the 2017 New Frontier broke new ground in the social VR sphere, what makes Zikr path-breaking is that it is a fully realized social VR documentary. Even as you move your controllers to play one of four instruments and control a string of prayer beads, you remain within a carefully constructed environment that does not stray from its documentary reference points.
Zikr's director, Gabo Arora (who, as the first Creative Director at the UN, made such seminal VR works as Clouds Over Sidra and Waves of Grace), was inspired to take on his first social VR piece: "I knew that if I didn't do something with social VR that was meaningful, it probably wouldn't happen because people would use it to shoot players and superheroes and jump around—which is fine, but how do we combine that so that we can do something that is playful yet profound? Very hard to do, but that's something we're going to try to do." He found the perfect storytelling vehicle when he began to explore the idea of focusing on Sufism—specifically, the music and movement that are at the heart of Sufi practices and rituals.
He then partnered with John Fitzgerald and Matthew Niederhauser of the creative studio Sensorium, whose experience spans documentary, art, new media and technology. Following the group's first 360 video shoot at a Sufi shrine in Tunis, they decided to cross the social VR threshold. New partners Superbright were engaged to bring in their expertise with social VR. Another key member of the team, editor and producer Jennifer Tiexiera, helped secure, on a very tight timeline, the necessary remaining funding for the piece, and to shape the piece in the edit room.
The Sufi groups were fully on board with the idea of audience participation. According to Arora, "They only agreed to do this because they knew people were going to participate. It can give you the space to go in deeper. If we took that out, you'd be much more of a voyeur. You wouldn't make yourself vulnerable. With vulnerability comes the ability to empathize and love and open yourself. You feel like at least you have a purpose. You don't have to be a Muslim to be a part of these rituals."
As Arora points out, the story structure ended up being quite simple: "What is Sufism? What are these rituals? Why are they important? Why should they be saved?" To that last point, Sufis and Sufism have often been under threat by extremists in the Middle East and North Africa. In a horrific attack in Egypt this past November, militants bombed a Sufi mosque in Egypt, killing over 300 worshippers.
As far as the affect Zikr has on its participants, one may feel, at first, slightly inhibited, but that's not surprising given that this is a social situation involving strangers. Over time, however, the music and the movement—and yes, the beauty and the grace—take over and one's inhibitions begin to dissolve. I found one long scene shot on the roof of a Sufi shrine with a group of male worshippers to be particularly hypnotic. Because the VR camera is placed in the middle of the group, the fourth wall comes down; you are no longer just watching the group but are a part of it. The men's voices and movements become melded with your own. (Enhancing the experience is a nuanced and very affecting sound design from the creative audio studio Antfood.) It is a startling and, perhaps, even transcendent experience, combining the best of what VR and documentary can do when in perfect sync with each other.
Note: On the final day of Sundance, Dogwoof, the UK-based distributor, international sales company and production fund, secured worldwide distribution rights to Zikr with the goal of exhibiting it at festivals, museums and art venues.
Hero
While Awavena expands our minds by re-creating in vivid detail a deeply moving visioning experience, and Zikr activates the joyful journey from merely observing an entrancing ritual to participating in one, Hero goes one step further: it puts the user at the center of an event that bends to one's own choices and actions.
While the setup for Hero is the same for everyone, the second half of the experience and its dramatic conclusion are in no way predetermined. The story, including the ending, is what you make it, and the story doesn’t even end when you take off the headset. You will no doubt reflect back on the choices and actions that you have just made and contemplate what they say about you and about your connection to your fellow human beings. It's a daring and profound piece, one that has consequences for the VR field, for documentary storytelling and for those who experience it.
Hero's lead artists Navid Khonsari—an established game developer who has worked on such popular franchises as Grand Theft Auto and Resident Evil—and Vassiliki Khonsari—an accomplished documentary filmmaker and visual anthropologist—created the genre-bending video game 1979 Revolution, which they were invited to develop as Fellows at the 2014 New Frontier Story Lab. In that game, users take on the role of a young Iranian photojournalist who is forced to make a series of ethically fraught life-and-death choices following the 1979 Iranian revolution. With its real-world backdrop set against unfolding political events and one that involves making impossible choices in a heightened dramatic situation, Revolution laid the path that Hero would take in new directions.
Created to counteract what the artists perceive to be a distant and "disembodied" collective response to war in general and airstrikes in Syria in particular, Hero is an "effort to resuscitate the humanity of these profound moments and re-awaken them through the use of new technologies and art."
The Khonsaris, co-founders of the creative studio iNKStories, pitched the idea of Hero to Starbreeze Studios, an international gaming and VR company looking to develop more projects in the immersive VR space. Starbreeze's Brooks Brown had been championing the idea of large-scale multi-sensory VR projects akin to Alejandro González Iñárritu's high-profile VR experience, Carne y Arena, when he heard the Khonsaris' pitch. He had an immediate and strongly positive reaction. With Starbreeze on board as partners, the funding was secured as well as the tech, including the StarVR headset.
In developing the story, the goal was to veer away from the idea of telling a linear story, or even much of a story at all. According to Brown, "We had this idea that we wanted to create an event. We didn't want to create a story or a linear narrative, but it's not easy because we have all of these little parts of our brain that have been trained over all these games and films and things we've made that [tell us], 'This is how it’s supposed to be done.' But we're not making a VR game."
Navid, too, had to resist the urge to have everyone experience the story all the way to the end. Eventually, he was able to make peace with that: "It is their journey that is going to take place and however that journey plays out, you've done what you needed to do as a creative to get them there."
While events in Syria were clearly an inspiration for the piece, in the end, the filmmakers chose not to depict actual events or places in favor of creating an environment that felt truthful and real, but could be a stand-in for any number of war-torn communities in which people find the courage to live out their daily lives in the face of calamity and chaos. Even though the setting is not tied to any specific place, Navid and Vassiliki are clearly influenced by the documentary tradition, even as they seek to adapt it to the kind of filmmaking they are doing and to a native VR environment. In fact, they have co-opted the term "cinema vérité" within the full title of the piece, Hero: A Vérité VR Experience, which makes clear their desire to see a new documentary grammar take hold.
According to Navid, "In vérité cinema, the role of the creatives is to capture moments that are true to life. In vérité VR, we are capturing the moments that are true to life, but not allowing you to be a viewer at a distance. We are actually putting you in that world as who you are and allowing you to observe that vérité world, allowing you to walk away with your own personal experience because you are an individual and you bring something to that vérité experience."
There may be a whiff of irony in the fact that these filmmakers are relying so heavily on cutting-edge technology to create the kind of vérité experience they are describing. Ironic or not, however, tech is the key to ensuring that their creative goals can be achieved. According to Vassiliki, "More and more, in fact, the world around us is not experiential, and so reprising experience through technology sounds like a long way around, but when we're interacting with our computer it's not experiential, when we're interacting with a book or a pen and paper, it's not experiential. But this is."
In VR, there is an art to the tech. It involves performing a delicate sleight-of-hand in which the tech, even as it does the heavy lifting, makes itself disappear. As Navid states, "We used haptics and the headset and that world that we created to strip you of all the layers that you put on top of yourself, so that we can get to your core and then put you in that position where you as you go through that — no role-playing, no nothing else. It's you."
Once those layers are stripped away, and it's just "you" left to go through the experience, where does that put you in relation to "hero"? What does it mean to act like a hero? Is there some standard of behavior that we use to define what qualifies as heroic behavior and what does not? Typically, in the media landscape that we consume every day, the hero is someone outside ourselves, someone we are counting on to save the day. But when we are left alone to grapple with that role, it can become a frightening prospect.
According to Vassiliki, "I think our world needs us to move beyond these unattainable ideas of who we need to be and who we should be and who is going to come and save us. This is a refocusing and a reframing to bring it back to the individual. We want to give the individual both the responsibility and accountability but also the inspiration to connect with other human beings and recognize the power we have in that."
Other Notable New Frontier Exhibition Experiences
In addition to Awavena, Zikr and Hero, there was a plethora of impressive projects at New Frontier, spanning documentary and fiction genres and often defying easy classification by skirting the slippery edges in between. I wasn't able to try everything, but here are highlights of the other documentaries I experienced.
Elastic Time, Mark Boulos' mixed-reality interactive documentary about space-time, combines narration from astronomer Tony Stark with a thoroughly mind-bending experience in which you are able to see a hologram of yourself as you walk around (and outside of) an observatory. There was a lot of buzz around the fact that, rather than inhabit an avatar of some other character, the digital avatar in this case is you. Seeing your POV shift between first person and third person as time goes backward and forward was quite a trip.
Space Explorers: A New Dawn—Veteran VR artists Félix Lajeunesse and Paul Raphaël take us through the rigorous training and still awe-inspiring aspirations of NASA astronauts as they train for the International Space Station and possible future exploration of Mars. Adding considerable value to this experience were the very cozy Voyager chairs from Positron, which not only created a more immersive environment through its gentle rotation and spatial audio but also incorporated a cool 2001 vibe into the journey.
SPHERES: Songs of Spacetime—Sharing thematic turf with both Elastic Time and Space Explorers, SPHERES allows you to make a virtual dive directly into a black hole. The first of a projected three-part series, the VR film is directed by Eliza McNitt and features Jessica Chastain as narrator, Darren Aronofsky as EP, and a soundtrack by the Stranger Things musicians from Survive. SPHERES made big news at the festival, with the trades reporting that the VR film had been acquired by CityLights for a seven-figure deal, the largest purchase price to date for any VR film at Sundance.
New Frontier Films and Performances
Deep Astronomy and the Romantic Sciences—This enthralling and delightful performance piece explores, through director Cory McAbee's uniquely imaginative lens, a wide range of science and astronomy-related topics. Combining mini PowerPoint slide lectures with heartfelt songs from his Small Star Seminar, this high-wire performance act was pitched with perfect timing that felt simultaneously polished and raw, spontaneous and meticulously constructed. According to McAbee, "I have no idea how I did that. I just put these ideas together and tried to create a sense of timing that gave a feeling of excitement to these ideas. Once it was over, it felt like something I had never done before."
Part of the 2012 New Frontier Story Lab, Deep Astronomy has gone through an extensive evolutionary process, beginning first as a global arts collaborative called "Captain Ahab's Motorcycle Club" that was initially working on a film about Abraham Lincoln's embalmer before gravitating towards the Small Star Seminar canon. While performing these songs, McAbee would often give short lectures on science in between his musical sets. The music plus the lectures then formed the basis for Deep Astronomy. The planned final phase of the project is a feature film based on these new performances.
McAbee's lectures and slides are the most fascinating lectures you never had in college physics, and while they may not contain a whole lot of provable hypotheses, no matter; it's how they unleash the human imagination that counts. Besides, he has even had a quantum theoretical scientist come up to him after one of his lectures and tell him, "I know you’re joking, but you're right."
One of the true delights of Sundance, Narcissister Organ Player, a hybrid personal documentary/performance film by the Brooklyn-based performance artist Narcissister, was nothing short of a revelation. The film masterfully fits together multiple story shards from Narcissister's experiences growing up, the emotional terrain of her biracial identity, a fraught relationship with her dying mother, and provocative performance pieces that comment on these events even as they create new narrative landscapes. Narcissister's film is, like the ever-present iconic mask she wears over the top half of her face, both a reminder of the loose-fitting cover we put on to hide the roiling emotions that define us as human beings and a marker of that same emotional and psychological terrain.
A Thousand Thoughts, a new performance gem from the always invigorating Sam Green and the brilliant editor Joe Bini, shined bright from the stage at the Egyptian Theater. Standing in his usual place behind a podium, Green held forth on the history of the Kronos Quartet, as the musicians sat, instruments at the ready, on the other side of the stage. For someone who has never heard Kronos live, this was a unique opportunity to hear this path-breaking group of flawless and inventive performers bring to life a collage of selections from throughout its 40+ year career. Green's brilliant mix of archival treasure-trove hunting, illuminating commentary and refreshing humor was immaculately timed to the group's playing and to the montage of eye-popping visuals playing on the screen behind them. One can only hope that future collaborations between Green and Bini are on the horizon.
(Star), by Johann Lurf, provided yet another example of how au courant exploration of outer space was at the 2018 New Frontier. Drawing on the rich vein of visual representations of stars and space in movies, Lurf has created a sprawling but also minutely fascinating montage that engages the audience on multiple levels. Lurf's film allows our imaginations to roam free over this vast landscape of celestial-related imagery and to ponder the unknowable void of space that seems to be always slightly beyond our intellectual, artistic and emotional reach. Bolstering the hundreds of clips is a fractured sound design that helps give shape to the narrative space even as it leaves us asking more questions.
Ken Jacobson, a contributing editor at Documentary magazine, is VR and documentary programmer at the Minneapolis St. Paul International Film Festival and VR curator and Forum programmer at AFI DOCS Film Festival.