Poignant, chilling and more relevant than ever today, the award-winning documentary The Social Dilemma turned a global spotlight on the addictiveness of social media and its role in radicalizing users. More than 38 million households in 190 countries streamed the film within its first month making it both the first documentary to reach the number one spot for a month on Netflix and the first documentary to achieve any top 10 spot on the platform.
The format of The Social Dilemma – fusing investigative documentary and scripted narrative – allows viewers to experience the dangerous human impact of social networking in a unique, compelling way. But the film’s use of visual effects are what heightened its cautionary message, taking them inside the personified algorithms of a social platform.
How did it all come together? The answer is a feat of motion design that involved a functioning faux social platform, in-camera device interaction and dynamically-built rigs that allowed for maximum flexibility on a tight deadline.
Filmmakers Jeff Orlowski and Larissa Rhodes tapped Denver-based Mass FX Media for the job, knowing the team was capable of using motion graphics to humanize and bring emotion to complex, nonfiction content.
Mass FX’s founders Matt Schultz and Shawna Schultz, who had worked on Orlowski’s previous film, Chasing Coral, mobilized quickly. Initially, the goal was to create a faux social media network with all of the functions of today’s major existing platforms, merged into one, and then bring that interface into the narrative sequences.
“We spoke with the director about how we wanted our fictional platform to look, behave and function throughout the film,” explains Matt Schultz. “As Skyler Gissando’s character, Ben, becomes more and more absorbed in the digital world of his phone, we would illustrate this by gradually filling any available space in the frame with an endless social feed of increasingly-radicalized content as the film progresses.”
The phone elements and UI seen floating next to the characters here and there at first would ultimately line the rooms and overwhelm scenes, focusing viewers as much on the social media platform as the characters staring at their phones.
With the script calling for numerous shots of Ben on his phone, Mass FX had two options for creating the mobile platform: shoot green-screen inserts of the phone and replace the screen images following the talent’s mimed interaction, or build a prototype platform they could film practically, with full interaction. Given the aggressive post schedule, shooting practically offered the best performances on-set and gave the editors a clear picture of how the phone cutaways would play in the scenes.
The AI and integrated feeds in Ben’s world relied heavily on presenting his social feed to the audience. And since a social media platform has an infinite scroll, Mass FX needed a way to flexibly build, place and animate hundreds of social media posts throughout the film in a way that appeared as natural as possible. To do so, Mass FX generated more than 350 faux posts in After Effects.
Mass FX founders Shawna and Matt Schultz
“It would have looked unrealistic to have all our most important posts one after the other, so we needed to generate several posts to fill the space between scrolls… things like ‘here’s my dog’ or ‘we won the game,’” notes Shawna Schultz. “We also had to create filler posts that other users and friends would have created, as well as advertisements for random shoes, toothpaste and hair gel that our character might be served up. And, importantly, we needed to create posts for the Extreme Center group that was radicalizing Ben, and those needed to feel like they were coming from a single group that might be paying to get followers.”
To assemble the feed posts and enable the screen to interact with the talent’s gestures, Mass FX built a prototype mobile feed using a mobile web-developing tool called Figma. Depending on the requirements of each shot, they would embed a transition or button state for the talent to scroll to and activate. Matt, as on-set VFX supervisor, would cue the prototype output preview to the talent’s phone and trigger screen events in realtime. The director could then call for notifications, phone calls, or social posts to arrive on the prop phones during the shoot.
Another advantage to Figma is that it’s a cloud-based collaborative tool, which enabled the design team at Mass FX’s office to update screen content for the shoot and have their changes immediately reflected on-set.
For a couple scenes that were adjusted during rehearsals, Mass FX was able to have changes committed as the director called “action” and the camera rolled. The updated content was piped to the talent’s phone in the nick of time. Even on-set, notes could be adjusted there and ready for the next take.
“Knowing that our post schedule was so tight, this ended up saving the visual effects team at Mass FX weeks of work replacing screens and animating screens in post production. Shooting it practically was truly the best way to go,” Shawna notes.
One sequence Mass FX didn’t have time to fully design and rig in Figma prior to the shoot was the beautification app Isla (played by Sophia Hammons) uses in her bedroom.
“We needed to replace her phone screen with her selfie photos and the final design of the app, but also be able to direct her gestures to the areas of the screen that would become UI buttons, so our solution was to build a mockup of the app UI in subtly different shades of green,” explains Matt. “Sophia was able to discern the different target areas of the UI on-set, but it was still green enough to pull a decent key off the screen. We helped coach her and the director through how we imagined the final app would behave and direct her finger taps and gestures around the green UI.”
With a tight compositing schedule, Mass FX needed to come up with an efficient way to create and animate the floating UI graphics in the narrative sequences, so they put together a handful of expression rigs that would adjust container sizing, view counts and post spacing in After Effects. This allowed for quick changes if the director wanted to swap out posts in any of the scenes or change the scroll animation timing. Rigging the feed itself was a complicated step. Because each post was a unique height, any rearrangement of the feed stack would throw off the placement of the posts. Mass FX built a JavaScript post stacker that would bring in all of the post pre-comps and dynamically arrange them based on their index in After Effects’ layer stack.
The team also added some responsive elasticity to the feeds, so the posts would breathe a bit and space apart during the scroll, then gently come back together. They were able to rearrange the feed and scroll through any number of posts based on their layer index, so each target post would land exactly where expected, regardless of how that post was sized. The feed rig proved to be a lifesaver as changes were addressed. Mass FX pre-rendered all of the feed sequences and then passed them through its Nuke compositing pipeline to get them integrated into the scenes.
In addition to creating the social media platform, Mass FX also created a matching control board representing the platform’s behind-the-scenes operation in the world of Vincent Kartheiser’s AI characters.
Finally, for the documentary portion of the film, Mass FX was challenged with designing infographics and visuals that could help support the data from interviews and information the director was trying to make clear to the audience. This included an animated backstory of the main documentary subject, Tristan.
In all, Mass FX delivered over 100 shots for the film, including the title package, character animation, AI screen content, and data visualizations used in the documentary.
“We loved being able to solve the creative problems of how to bring this content to life in both the narrative and documentary worlds,” recalls Shawna. “In this film, the graphics and animation played a vital role in bringing humanness to the numbers and helping viewers visualize difficult-to-understand concepts like algorithms. It was a fun challenge for which we were proud to create a solution."
Mass FX Media credits included post production coordinators Carolyn Bush and Ryan Scobey; motion designers & animators Breece Kelsey, Joel MacCluskey, Aaron Lenius, LC Miranda, Jon Rom and Ryan Walker; VFX compositors Julie Henninger and Joel MacCluskey; VFX Editor Tatiana Jones; illustration producer Lidia Scarlat and illustrator Maxim Maevschi.