Under the leadership of Chief Curator Shari Frilot, this year's New Frontier program at the recently concluded Sundance Film Festival continued to push the boundaries of film, art, media, technology, music and performance and reaffirmed its stature as one of the world's premier showcases of virtual and augmented reality programming. Having only been a part of Sundance since 2007, New Frontier is still a relative newcomer to the creative goings-on in Park City, but this recent edition leaves no doubt that the program is on par with Sundance's celebrated film programming. With its expansion to new venues in 2017, one might expect NF to be bursting at the seams, but careful planning, skilled management and savvy programming have allowed it to scale up without seeming overwhelming, chaotic or diluted.
What was new in 2017? The most conspicuous change this year was the unveiling of a new venue one block off Main Street called the VR Palace, which was home to 16 virtual reality "experiences." Access to the VR Palace was granted to ticket holders only, who used Sundance's regular ticketing system to purchase one-hour time slots for viewing the projects. Slots sold out quickly, proving that New Frontier tickets are already just about as coveted as Sundance screenings.
For those without tickets, there still remained the option of checking into the New Frontier site on Main Street, which hosted 11 projects of its own (Chris Milk and Aaron Koblin's astonishing Life of Us was available at both sites). In addition, on five nights, a selection of VR projects and performances was available at the VR Bar at the Music Café on Main Street. Supplementing all of these offerings, a number of key players in the VR space, including Oculus VR, Jaunt VR and Condition One, set up shop on Main Street to show off a range of their latest projects, including some that were in the New Frontier lineup.
All told, while the challenges in exhibiting VR and augmented reality projects still remain, New Frontier 2017 represented a major step forward in allowing masses of people the opportunity to experience an incredible array of cutting-edge work in an orderly fashion and share impressions with each other and, in many cases, with the creators themselves.
Creative Breakthroughs: Augmented Reality, Social AR/VR and Longer Pieces
Two noteworthy creative breakthroughs at New Frontier were the blossoming of several astounding AR pieces and the introduction of social VR and AR. While until now, VR - due to easier access to tools by creatives and the relatively cheap cost of headsets - has been the predominant exhibition medium, AR is now emerging as its equally impressive twin. If you are wondering about the difference between AR and VR, a reductive way to consider the difference is to compare the headgear: AR is most familiar to us through the model laid out by the nearly forgotten Google Glass of a few years ago, while VR headsets are those oversized goggles made by companies such as Oculus Rift and Samsung Gear VR. Generally speaking, AR is a medium in which objects appear before you and you are able to interact with them, and VR is one in which you enter a rendered environment that allows varying degrees of self-exploration. But this is a somewhat artificial distinction; the differences between AR and VR are best understood by trying them out yourself.
Another related technical and creative breakthrough at NF was the introduction of a social interactive component in several VR and AR projects. Since VR's inception, questions have arisen about its solitary aspect. Unlike film, with its communal experience of gathering strangers together in a dark room to watch images projected on a screen, VR is almost always experienced one at a time, with the user strapping on goggles and cutting oneself off from social interaction. While the merits of one versus the other experience can be debated, the question remains open about how effectively and to what extent VR/AR experiences can be socialized. In fact, a panel at NF entitled "Step Right Up: Live VR and Multi-user VR" explored this very question.
The socially interactive projects such as Life of Us, The Journey to the Center of the Natural Machine (see below) and NeuroSpeculative AfroFeminism (Lead Artists: Ashley Baccus-Clark, Carmen Aguilar y Wedge, Ece Tankal, Nitzan Bartov) all involved a level of interaction with other users in dynamic creative environments, in which the social interactions of those simultaneously experiencing the projects helped determine the unique ways that the projects can unfold in real time.
A third trend would also be the inclusion of longer VR pieces, including four 15-minute projects and another, Miyubi, by VR pioneers Félix Lajeunesse and Paul Raphaël, clocking in at 40 minutes, prompting New Frontier Co-Director Kamal Sinclair to tweet, "Miyubi is the first VR movie to feel like a real film." In the coming months, we can expect to see more VR pieces creeping closer to feature-length films or replicating the series model.
Screenings/Live Performances
Besides exhibiting over 20 projects and presenting four panels, New Frontier built on the tremendous success of its world premiere screening last year of Kirsten Johnson's Cameraperson with three screenings in 2017: 18 Black Girls / Boys Ages 1-18 Who Have Arrived at the Singularity and Are Thus Spiritual Machines: $X in an Edition of $97 Quadrillion, by Terence Nance; Did You Wonder Who Fired the Gun?, by Travis Wilkerson; and World Without End (No Reported Incidents), by Jem Cohen.
I was able to attend Wilkerson's performance documentary, a riveting and deeply disturbing personal reflection about a deadly encounter in the rural South that sliced through both his own family history and that of an entire region and epoch of post-World War II America. What could have been a soulful, but singular elegy for the lost ideals of America's civil rights movement expanded, through the sheer force and concentration of Wilkerson's narration, as well as through his meticulous historical research, beyond traditional storytelling boundaries to become a multilayered, transcendent experience. This reckoning of the dead morphed into an urgent call to action for the living. Surprised to hear that another film festival invited him to re-mount his performance, an equally drained and exhilarated Wilkerson seemed to come to his own reckoning: People are calling out for this kind of innovative, urgent, in-person storytelling, and not just in Park City.
While I was not able to experience all 26 New Frontier projects, I did manage to see a great range of truly impressive work. An overview of some of the projects shown at New Frontier, as well as several projects exhibited at the Oculus House and Jaunt VR Lounge, follows:
New Frontier AR and VR Experiences
The Journey to the Center of the Natural Machine
Lead Artists: Daniella Segal, Daniel Lazo, Eran May-Raz, Charles Niu
This astonishing AR experience begins when you and another person, seated on opposite sides of a semi-circular bench, are outfitted with the Meta 2, a pair of highly sophisticated AR glasses. Between you sits a small, round stump of a table. Soon, the two of you see, floating above the table, the first of a series of strikingly detailed, dazzlingly colorful images of a human brain. This is the brain as you've never seen it - from every angle, all sides exposed. More amazingly, as a narrator describes the evolution and function of the brain, you and your partner take turns sweeping your hands toward the center space and "pushing" different spheres of the brain up into the model that you now see being constructed in front of you. It's a breathtaking, collaborative experience that not only feels like a sci-fi movie come to life, but serves as an invitation to unleash your own imagination about the educational and interactive potential of the AR medium.
Heroes
Lead Artist: Melissa Painter
Heroes deploys both AR and VR, thereby simultaneously highlighting the distinction between the two media and their potential complementarity. Described as a "duet in mixed reality," Heroes succeeds by featuring multiple literal and metaphorical duets: in part one, an enthralling and sprawling dance piece, choreographed by Helios Dance Theater founder Laura Gorenstein Miller, unfolds through the close up intensity of VR; and, in part two, through the magic of AR and a pair of HoloLens glasses, a kind of meta duet ensues between you and the dancers that takes you from passive observer to active participant. In part two, the designed room serves as a backdrop for your own performance as you use a series of verbal commands, hand gestures and body movements to control the relative size of the dancers, your own positioning in relation to them, and the pathway that you and the dancers take together. At the command of "And then…," the narrative moves boldly forward to a seemingly limitless future. Once again, as with Journey, a new medium's potential seems to be realizing itself not only in front of your eyes but in partnership with your own dreams and desires.
Tree
Lead Artists: Milica Zec and Winslow Porter
Featured in Sundance 2017's "The New Climate" strand, Tree stands on its own as a towering, inspiring work but also serves as the second in a planned trilogy that began with last year's Giant. Tree represents both a continuation of that level of intense personal experience and a departure in technological innovation and creative storytelling terms. Ingeniously composed of several parts, Tree begins when you are given an actual seed to plant in a planter box and then are outfitted with a vibrating backpack and VR headset. Inside the headset, you now take on the virtual role of the seed itself, quickly becoming a sapling in the rainforest. With your arms as branches and body as the trunk (and with the aid of controllers), you make yourself grow taller and taller, striving upwards, among the canopy, toward the sunlight. At this point, events greater than you take over. The use of a fan and shaking floor further enhance the multi-sensory experience. Once you remove the headset and face the wall, you see a brilliant projection of the tree that you "created" in virtual reality. When you step off the platform, you are handed another real seed. As you walk away with the seed in your pocket, you literally take the experience home with you and are reminded of the transformative journey that you have taken part in.
Out of Exile: Daniel's Story
Lead Artist: Nonny de la Peña
Nonny de la Peña, one of the true pioneers of VR, has gone even deeper at this year's New Frontier with her latest film, the moving and innovative Out of Exile: Daniel's Story. As she has in the past with such groundbreaking projects as Hunger in Los Angeles, Project Syria, Kiya and Across the Line, de la Peña constructs the first part of Out of Exile around an actual audio recording, made by Daniel Ashley Pierce, a young man whose family verbally and physically accosted him about being gay before kicking him out of the house.
The inspiration for the film came when de la Peña was informed by the nonprofit True Colors Fund that 40 percent of homeless youth in America identify as LGBTQ and pointed her in the direction of Daniel's story. Part one of the piece involves a meticulously re-created VR rendering of the family living room, which was the setting for the violent encounter between Daniel and his family. In this gripping scene, there is no escape from being implicated in the events themselves.
Upon multiple viewings, taking full advantage of the film's volumetric "walk-around" capabilities, I made a point of moving close to different characters in the piece, not restricting myself to simply looking at the different people but trying to see things more literally from their perspective. While there was never any doubt where my sympathies lay in the events taking place, I did find it striking that once I moved closer to different people, I was able to see things more from their perspective than if I had simply stood off to the side and looked at them as mere objects. This experience highlights what is perhaps the most fundamental and profound breakthrough of the VR medium itself.
In part two, de la Peña incorporates new hologram technology from 8i to record real people and put them in virtual environments. In this section, Daniel's story expands and becomes more universal as several other LGBTQ homeless youth are recorded telling their stories.
With Out of Exile, de la Pena has once again proven that no other filmmaker can match her ability to combine cutting-edge technology with immersive, rigorous journalism.
Chasing Coral: The VR Experience
Lead Artist: Jeff Orlowski
Melting Ice
Lead Artist: Danfung Dennis
Zero Days VR
Lead Artists: Scatter, Yasmin Elayat, Elie Zananiri
Three of the dynamic new VR projects at New Frontier were tied to feature documentaries, two of which, Chasing Coral: The VR Experience and Melting Ice, premiered at the festival. The Chasing Coral film and VR projects were both directed by Jeff Orlowski, whose groundbreaking 2012 film, Chasing Ice, earned an Excellence in Cinematography Award at Sundance, as well as an Emmy Award for Outstanding Nature Programming. Melting Ice, the VR companion to this year's Sundance opening night film An Inconvenient Sequel: Truth to Power, was directed not by the makers of the documentary but by Danfung Dennis, whose searing Academy Award-nominated feature doc Hell and Back Again premiered at Sundance in 2011, the same year he founded Condition One, one of the leading producers of VR content.
Chasing Coral takes samples from the feature's visual treasure trove of images and uses them as the basis for creating an original VR experience. Orlowski uses 3D time-lapse photography and underwater moving images to mesmerizing effect, but, perhaps just as impressively, exhibits supremely economical filmmaking skills by giving one of his main subjects, reef aquarist and researcher Zack Rago, the role of guide in the VR piece. Through Rago's eyes, we see the horrors of coral bleaching that are devastating the world's reefs.
Dennis capitalizes on access to both Al Gore and Greenland's diminishing glacial ice sheets to build a deeply moving dramatic narrative arc. Few VR experiences at New Frontier 2017 could match the quiet power of sitting in the middle of a glacial ice sheet and hearing the "snap, crackle and pop" of the ice sheet melting beneath you. Dennis has exploited VR's full potential as an aural and visual medium to infuse the viewer with a powerful sense of calm self-reflection and deep contemplation about what is happening to our natural world as a result of the devastating effects of climate change.
Zero Days VR has yet a different relationship to the feature documentary that shares its name. The VR project, created by Yasmin Elayat and Elie Zananiri and the Brooklyn-based team Scatter, features the same reality capture and graphics work that Scatter created for the 2016 documentary by Alex Gibney, but picks up where its cinematic representation of the Stuxnet virus left off by translating it into a stunningly visceral, immersive environment. And, for its dramatic, eerie conclusion, things get personal: The viewer finds oneself face-to-face in the virtual world with the film's whistleblower, both of whom are now represented in 3D, thanks to a live-streamed scan powered by an innovative software program called DepthKit and the Intel RealSense R200 Camera. Once again, as with Chasing Coral and Melting Ice, Zero Days VR both draws upon and refers to its companion feature, but also stands alone as a unique immersive experience.
Oculus VR for Good: Notes to My Father and Use Your Imagination
A jam-packed premiere event at Oculus House featured eight films and one trailer from the inaugural Oculus VR for Good Creators Lab, a commendable new initiative that matches 10 up-and-coming filmmakers with 10 nonprofits and 10 A-list mentors and experts. In August, the teams attended a boot camp where they received intensive hands-on instruction and began to develop their projects; given access to Nokia cameras and post-production services, as well as a modest budget, the filmmakers then dispersed around the world with the intent of producing VR projects that highlighted issues related to the work of the nonprofits.
In Notes to My Father, Lead Artist Jayisha Patel, working with the nonprofit My Choices Foundation and mentor Gabo Arora, a well-known VR artist and Creative Director at the United Nations, adroitly and sensitively balances the perspectives of a father and daughter coming to grips with the fact that the father played a direct role in his daughter's sex trafficking. Notes to My Father could have easily taken the route of showing how widespread the problem is or what steps can be taken to combat it. Instead, the film examines, through these two perspectives, the huge toll that sex trafficking takes on one family and illustrates how forgiveness is possible and lives can be repaired. Patel fully explores VR's deeply immersive aspects and gives a fresh perspective on the cinematic language of point of view.
Described as "the world's first Virtual Reality music video featuring artists with autism and special needs," Use Your Imagination is completely unlike any of the other VR for Good projects. A creative, joyful celebration filled with vivid costumes, elaborately constructed sets (including a totally far-out elevator made specifically for the video), and toe-tapping original music written and performed by young artists on the autism spectrum, Imagination was created by Lead Artist Roberto Drilea, working in collaboration with Spectrum Laboratory and mentor Opeyemi Olekumi of the Tribeca Film Institute. Given that Spectrum Laboratory's mission is to empower autistic artists to create original works of film and music and collaborate with professionals, the VR for Good initiative is a match made in psychedelically rendered heaven.
Jaunt VR: Under the Canopy
Not to be outdone, Jaunt VR opened up its Lounge on Main Street to exhibit 20 VR experiences, some of which were featured in New Frontier. One of the more striking pieces I saw was Under the Canopy, which offered a privileged glimpse of the Amazon rainforest as told from the perspective of indigenous peoples. Developed with Conservation International, Canopy employed old-school filmmaking techniques along with stunning 360 VR video to create a journey that could not have been replicated in any other way - not even in person. Beginning with a shot that starts above the trees and then slowly descends beneath the canopy and ending with the same shot in the opposite direction, Creative Director Patrick Meegan has created a piece of almost perfect balance and harmony; nature is not overly romanticized but on full display in its mesmerizing raw state; indigenous subjects are not merely guides, but real collaborators and storytellers.
This piece, along with many of the works at New Frontier, may create real impact once more people have the opportunity to slip on a VR headset and see that film can be more than a passive viewing activity. VR is a series of multi-varied, vibrant, interactive and meaningful experiences that are destined to evolve in unexpected and spectacular ways; thanks to New Frontier, the medium is no doubt here to stay.
Ken Jacobson, IDA's former Director of Educational Programs and Strategic Partnerships, is a documentary programmer for the Palm Springs International Film Festival and VR/AR Programmer for the Minneapolis St. Paul International Film Festival.