I’m sitting in my office with Toshi Anders Hoo, a mixed reality designer with a background in film, chatting about 360 videos (also referred to as spherical videos). 360 videos are often the gateway drug that gets people hooked on VR. However, while the uninitiated are getting their first taste of 360, early adopters are already beginning to develop a more demanding palette, and content creators are scrambling to discover the new vocabulary of 360 storytelling. This rush to discover the “language” of 360 storytelling belies the truth that 360 must evolve to stay compelling beyond the initial wow factor.
One of the issues facing 360 is that it became synonymous with VR media from the outset. Before the Gear VR was even called Gear VR, John Camac of Oculus came to the Samsung Research offices in Dallas for a little show and tell. Both teams were independently working on mobile VR prototypes and oddly enough, both brought some version of a 360 viewer. Was this a coincidence or something more?
As it turns out, 360 content is great for showing off the technical capabilities of a VR device. You can use it to demonstrate the fidelity of sensors and algorithms, device performance, screen refresh rates and more elements essential to the viability of mobile VR. Moreover, there was already a plethora of 360 content from niche photographers and 3D artists. This meant that most mobile VR tech demos used 360 videos, and almost imperceptibly this became synonymous with VR content.
Unfortunately, this made the thought of using anything but 360 content anathema to anyone hoping to show off VR. I say unfortunately because this threw out a huge body of work and established techniques for storytelling in rectangular formats. Instead of determining the role of rectangular content in VR, creators and developers alike chose to concurrently innovate both VR tech and 360 capture, storytelling and delivery. The combination of the two provided a green field for innovation, but may have also conflated the success of one with the success of the other.
Spherical formats are exciting because they give the user agency — the power to experience a story becomes a collaboration between the director and the viewer. While this new ability leaves the user feeling empowered, it also requires a greater expenditure of energy and may create a fear of missing out. If I get too absorbed looking at the apatosaurus in front of me, I’ll probably miss the pterodactyl flying overhead. The downside of this thrill is that the chemical surge of excitement can only last a few minutes before the viewer needs to relax. This is one of the reasons that the 4-5 minute threshold is so hard to break.
This time barrier used to be a storage and bandwidth limitation, but technical achievements in streaming from players like Facebook, Samsung and Jaunt have removed many of these technical hurdles. Granted, the cost and effort of producing this content is still higher than traditional formats, yet advances in capture technology like all-in-one 360 stereoscopic cameras with built-in ambisonic mics are removing some of the production pain points. This leaves us to wonder: Is the time barrier related to a physical limitation of the viewer’s attention?
In our conversation, Toshi compared 360 to music: “It can’t all be crescendo. An audience needs the low points to both rest and to realize that the high points are high.”
So, how can content creators improve the mechanics of 360 video? One option is to move beyond the idea that VR content must commit exclusively to a 360 format. Instead, creators can see 360 as just one of many lenses used in VR storytelling.
Experience designers Katy Newton and Karin Soukup, in association with Stanford’s d.school, the National Film Board of Canada and filmmaker Paisley Smith, conducted some amazing research on audiences’ experience and understanding of VR content which has been published in The Storyteller’s Guide to the Virtual Reality Audience. In one experiment, Newton and Soukup restricted the view of users watching a story to 90, 180, and 360 degrees. The results were profound, revealing that each format had a different impact on the user’s understanding of the story, ranging from different experiences of audio to recalling a character’s name. While the authors don’t address it directly, the ramifications of this study support the idea that different VR formats could be combined to provide more targeted VR storytelling.
I recently saw a film where the director used different content formats as storytelling mechanisms. In this story, the director switches back and forth between rectangular and 180 formats to reinforce the different spaces, both physical and emotional, as they relate to the story’s plot. The effect was so subtle and profound, that I actually thought I came up with the idea, until I saw it used again throughout the story. Unfortunately, the film has not yet been made public, so I can’t refer to it by name, but the lesson I took away from it was that 360 is an option in cinematic VR and not a prerequisite.
Lastly we come to Vicarious, a storytelling toolset and platform which our team has been working on for the last year. Vicarious is best described as a storytelling engine. We provide the tools, technology and mechanics for storytellers to express themselves using both 360 and legacy media, encouraging the combination of formats in new and unforeseen ways. Our paginated system reads like a multimedia magazine where each page could be 360, rectangular or a combination of both. User control over progression provides agency without the fear of missing out. Furthermore, custom geometry for each format ensures that empty pixels aren’t wasted, if not required. The best part is that embracing, rather than rejecting, legacy media formats opens the door for anyone to create content for VR.
In closing, I’d like to reiterate that this is not a condemnation of 360 content. Spherical video is both compelling and necessary. Advancements in 360 quality and storytelling are improving every day due to the hard work of individuals, teams and companies committed to elevating the medium. As VR pioneers, it’s our responsibility to contribute through our conversations, research and actions.
For more on the future of VR and the latest tech, join us at SDC 2016.