Inside COVID19 follows emergency department director and physician Dr. Josiah Child as he readies staff in five different hospitals to respond to the pandemic. In the midst of his preparations, he is infected with the novel coronavirus, not in a hospital but in his home. Despite all of his measures to ensure the safety of healthcare workers, Dr. Child wasn’t prepared for how sick he and his family were about to become.
The stereoscopic 360-degree video is nominated for an Emmy in the “Outstanding Interactive Program” category and is meant to be viewed using an Oculus VR headset. Produced by WisdomVR Project (www.wisdomvr.org), the piece was co-directed by Gary Yost and Adam Loften, with Yost also serving as producer and Loften acting as editor. Here, the team reflects on their Emmy nomination and the challenges faced in producing this virtual reality project.
Photo: Yost
What’s it like being nominated for your work on Inside COVID19?
Yost and Loften: “It feels great! Anytime that you’re recognized by a group of peers for your creative effort it’s an affirmation that there’s meaning in your work outside of the personal satisfaction it provides.
“Our film Inside COVID19 uniquely chronicles both the global and personal perspective of the pandemic in virtual reality stereoscopic 360 video. The nomination is both an honor and also a platform to more widely share the experience of Dr. Josiah Child, an emergency physician who battled COVID on the front lines and within his body. Understanding how sick he became while building resilience in his five hospitals’ emergency departments is a story that deserves to be remembered. As we face another wave of COVID-19 it’s essential that we deepen our understanding of how SARS-CoV-2 spreads and how we can come together, both nationally and globally, to keep each other safe.”
How did you ultimately get involved in this project?
Yost: “In 2019 we founded the WisdomVR Project and just before the pandemic hit we had completed nine immersive documentary experiences funded by Oculus. We were riding a creative high, having spent the previous year honing our VR360 documentary skills. My background as the inventor of Autodesk 3ds Max and my grasp of the technical aspects of 3D and stereoscopic imaging perfectly complement Adam’s documentary filmmaking background and his ability to deeply listen to a subject’s story and channel it into a film.”
Photo: Loften
Loften: “After sheltering in place for two months and only seeing each other on Zoom, we set up a face-to-face meeting in Gary’s garden with Dr. Child. As we sat six feet apart with masks on, he shared with us the story of how he nearly died from COVID-19 the previous month. He wanted to share his story with the world, but we were terrified — of going into hospitals, traveling by air to remote locations, and generally, just exposing ourselves. This was the first time we had gathered with anyone outside of our households, and the opportunity to ask a front-line physician all of our virus questions brought the pandemic into focus.
“That meeting was a creative jolt and we became excited about documenting his story in VR360, but starting a production safely during the pandemic felt impossible at the time. This project required filming in hospitals and traveling to New Mexico, and we were still disinfecting our groceries! When we reflected on our garden meeting the following day, we both were in awe of the way Dr. Child not only received our concerns and curiosity, but helped clarify much of the news and contradictions swirling around us. He has a background in philosophy and the way that he shared his personal experience of the virus and his perspective on the pandemic actually made us feel safer.”
Yost: “We knew we needed to share this feeling of orientation and safety with audiences. The last year of honing our immersive documentary filmmaking made us the perfect team to share this story and we felt a responsibility to do it.”
What gear do you typically use to create VR projects, and specifically Inside COVID19?
Yost and Loften: “Our goal with documentary VR360 filmmaking is to have the smallest footprint possible. Our production package consists of one camera, one tripod, one iPad, one sound recorder, two microphones and two people. Our entire kit fits into two backpacks.
“Our video capture device is a Zcam V1, which integrates 10 cameras, each with a 190-degree FOV lens. This array captures the entirety of the space, allowing enough overlap between lenses so we can stitch the images into a highly naturalistic stereoscopic 360-degree 7K-by-7K video file. We are always trying to get as close as possible, and unlike other capture devices, this array allows us to film within 12 inches of our subject, while other cameras limit you to three feet.”
Yost: “It’s always our goal to get out of the way of the action and because the camera captures everything in the scene, we literally have to leave the room when we’re filming or find a place out of sight of the camera.
“Although we can use clean plates and paint ourselves out of the frame in post, we have learned by not being in the room that the subject can make a deeper direct connection to the camera (which is a proxy for the viewer-in-headset)
“We use wireless lavs and an iPad to monitor the takes, and because the 10 cameras create a mountain of data that has to be processed later and we don’t have playback, in some ways it’s like we’re back to shooting on film. Inside COVID19 uses a mix of observational and verite filmmaking, and as we are removed from the subjects space monitoring the moments on the iPad, we are actually crafting and editing the story in our heads, thinking of the next question to ask or an alternate perspective to film.
“We capture all of our sound with the Sound Devices MixPre-3ii 32-bit float recorder because the ‘raw audio’ it records with infinite headroom takes the stress out of having to watch levels during a take, freeing both of us to focus on the performance. Having to intensely focus is critical because with immersive cinema there are no opportunities to edit within takes later in post.”
Photo: Loften and Yost monitoring an interview
Where do you start on a project like this?
Yost and Loften: “We get to know every subject and the story way before we start filming these documentaries. This allows us to spend time choosing the locations and the best times to shoot, just like we would in all of our traditional documentary work. But unlike traditional documentaries, in VR360 every shot is typically its own scene, so it’s critically important that each location motivates the moment that the subject is sharing there.”
Yost: “I knew that we’d be filming in hospitals, but when Dr. Child suggested that we film up in the high desert near Los Alamos, I didn’t instantly feel how that location was motivated — plus I was afraid of traveling. But as he described his philosophies of the pandemic, what a virus is and how humanity was reacting to this moment of isolation and confusion, I realized what an amazing visual metaphor the vast empty desert would be for this conversation.
“In the case of the observational scenes in the hospitals, we were always set up to quickly capture our shots during important moments. VR360 cameras capture the entire scene around them, so no repositioning of the camera is necessary once it’s in the optimal spot. For the verité moments, we had more time to prepare the cinematography, which has special issues in VR360, particularly with regards to the exact distance from subject to camera and the emotional impact of that distance. It’s a game of inches and we fine tune that to connect the viewer-in-headset with the subject making direct eye contact. As the story evolves, we’re constantly processing how it all fits together and ensuring we’re asking the right question or capturing the needed context to serve the story.
“While there are still opportunities for discovery within the edit, when it comes to choosing takes and structuring the story, we essentially lock ourselves into most of the decisions we’re making in production, so we strive to capture as much as possible within the entirety of each take.”
How does shooting in VR affect the editing process?
Yost: “In VR360 documentary production, every shot is its own scene; we never cut to a closer angle or a wider perspective because those kinds of cuts would pull you out of the immersiveness. There are no cuts in reality!”
How did you adapt to the problems presented by filming with a doctor during the pandemic?
Yost and Loften: “Dr. Josiah Child opened his life to us completely; he invited us into his hospitals and his home. Because we work with a crew of just one or two people, we were already accustomed to efficient shoots and in many ways we didn’t have to significantly adapt our creative process to conform with COVID protocols.”
Yost: “Of course it was stressful because like the hospital workers that we were filming, we had to come home to our families after each shoot and we considered it a matter of life or death that we learn how to use the PPE safely.
“One of the positives of using a VR camera that captures everything without needing to be operated is that there were times when we put the camera in harm's way much more than we were, alone in the hospital room with the doctor and a COVID patient. That gave us more leeway to film without actually exposing ourselves in those small spaces.”
What are some of the takeaway from working on Inside COVID19 and will it affect how you approach future projects?
Yost: “I was excited about the explorations of this with our animator Andy Murdock during the summer of 2020. One of the big takeaways from the project is how much we learned while creating 3D/360-degree visualizations about how SARS-CoV-2 virions get into our cells. As miraculous as our immune systems are, nature is incredibly powerful and what we learned gave us massive respect for the diabolic nature of this virus. We had to do a tremendous amount of research to visualize the way the infection happens on the molecular level so we could represent it accurately inside the headset experience.
“Inside COVID19 is just the latest immersive film from the WisdomVR Project. Our mission is to preserve the wisdom of our time for future generations using the most immersive medium possible. We see our role as documentarians to be conduits for the most pressing social, environmental and spiritual stories of our time and we learned that even in the face of a global pandemic you need to say yes to important and even potentially dangerous stories that deserve to be told.
“The success of Inside COVID19 galvanizes our commitment and dedication to the craft of VR360 documentary filmmaking and is an affirmation that this work matters.”
Photo: Loften directing Dr. Child
What are you working on next?
Yost: “VR headsets are finally affordable, mobile and becoming ubiquitous – two huge players are pushing hard on it. Facebook’s Reality Labs/Oculus division now has 10,000 people with an $18B/year budget, and Apple is also spending billions per year on their own upcoming VR ecosystem. While there’s a ton of focus on games and 6DoF synthetic experiences, in the immersive documentary film space, there are only a handful of people focusing on making high-quality VR360 content.”
Loften: “Gary and I believe in this new art form because it allows unmediated experiences of people and places, preserving them in situ for generations to come. Understanding the highest potential of this storytelling medium is WisdomVR’s mission and something we’re devoting the rest of our lives to.
“As millions of new headsets are entering people's lives around the world, a mass environmental and cultural extinction is simultaneously taking place. VR360 is an opportunity to preserve and share precious people, places and cultures, especially those that are most vulnerable. WisdomVR is dedicated to two things: 1) refining the tools to create the highest-quality impactful VR360 documentaries and 2) sharing everything we learn along the way so that others can preserve their own precious heritage and cultures. There’s a huge gap in the VR space for these types of humanities experiences and we want to change that. The audience of viewers has a hunger for meaning and even if early adopters aren’t conscious of it now, this immersive medium will drive a new wave of humanities and social impact.
“Our long-term goal is to create a living library of immersive cinematic documentary experiences where people can sit with amazing people like Dr. Child and have a 1-to-1 communication of wisdom. In addition to creating discussion guides for each experience, we are also developing an open source VR360 heritage documentation framework for the public good.”
Yost: “Precious stories are being lost and we can help to make sure that they’re not forgotten. We hope to have many collaborators – our nonprofit’s mission is to share everything we know and we’re openly looking for partners to make all of this possible.
“We are all born storytellers, dependent upon our cultures to carry our evolving wisdom to future generations. New tools for the preservation of the wisdom of the world are emerging in a library that anyone can experience and add to, as well as a curriculum where people can use the benefit of our experience to preserve their own heritage. Inside COVID19 is one part of the bigger library that we've been creating since 2018.”
Where can readers learn more about your work?
Yost: “You can follow us on twitter @WisdomVR, on FB WisdomVR, www.insidecovid19vr.com and www.wisdomvr.org.”