LOS ANGELES — To bring Brian Selznick’s unique illustrated novel “The Invention of Hugo Cabret” to the screen, director Martin Scorsese teamed with top names in VFX to ensure that stereo 3D was used to maximum effect to create the world of Hugo, an orphan who lives behind the walls of a Paris train station in the 1930s.
The child seeks to uncover the mystery of his legacy — a damaged automaton crafted by his clockmaker father — while eluding a drunken uncle, an authoritarian station master and learning the secrets of the station’s embittered old toy store owner who turns out to be pioneering filmmaker George Melies.
Many see Hugo as a game changer in the deployment of stereo 3D as a storytelling device. “I think Hugo will alter the way people view 3D in general,” says Oscar- and Emmy Award-winning VFX supervisor Rob Legato, who served as the movie’s VFX supervisor, 2nd unit director/DP. He also conformed the film. James Cameron called Hugo a masterpiece” at a DGA screening prior to its release.
“3D was not a gag added on top of the movie,” says Legato. “It was never meant to be a gimmick for extra sales value. It was part of the dramatic storytelling and gave a greater emotional impact than 2D. We designed and shot everything as a three-dimensional movie, so it was never distracting. Marty is known for eliciting great, truthful performances, and 3D in the hands of a master — and all the other Academy Award winners who collaborated on this movie — is really fulfilling.”
Hugo marked Legato’s first “direct experience” with stereo 3D. Although he had worked on a portion of Avatar, creating the virtual camera system for Cameron, Legato shot some 60 days worth of material for this new Scorsese film. He, and cinematographer Bob Richardson, used several Vince Pace-designed prototype Fusion rigs and a Steadicam rig built especially to house the smaller, lighter Arri Alexa cameras in a stacked configuration.
Pixomondo, with offices worldwide, was the lead VFX house for Hugo. Santa Monica’s Lola VFX, a sibling of Hydraulx, handled beauty work for Ben Kingsley’s George Melies character, performing “probably the best de-aging I’ve ever seen; it’s so subtle and well done,” says Legato. Novato, CA’s Matte World Digital provided matte paintings for about a dozen shots and helped to art direct other paintings; LA’s Uncharted Territory built and phototextured in 3D a River Seine scene based on photos taken during a location scout in Paris. London-based Nvizage crafted the previz package.
Hugo is filled with homages to classic motion pictures and to various film techniques, especially those invented by Melies. “It’s a love letter to watching movies,” says Legato. “Marty [Scorsese] says that as an asthmatic child he spent more time in movie theaters than on the playground, and that permeates Hugo: You get a real sense of what movies have meant to him. Not a movie history lesson but the appreciation and escapist joy that audiences get watching movies.”
THE VFX
Hugo was the biggest VFX job to date for Pixomondo (www.pixomondo.com) and for Ben Grossmann, who acted as the company’s VFX supervisor and spent the production on set with Legato. Pixomondo’s offices around the world devoted 441 days to the film creating close to 1,000 VFX shots, over 800 of which made it into the final cut.
Any one of a number of VFX sequences — from Hugo’s nightmare filled with fantastical animation to a persistence of vision effect featuring Hugo and his friend Isabelle to flashbacks of Melies in his heyday as a magician and cinema magician — was “a pretty big task,” says Grossmann. “Add them up and you have a hefty accomplishment.”
He echoes Legato’s observation that it was Scorsese’s goal to “push the value of stereo 3D,” and notes that 3D worked especially well “in long shots where the camera travels,” including the movie’s opening and closing shots, which were reminiscent of the director’s famous Steadicam shot in Goodfellas.
Legato says that when Cameron shot Avatar he remarked that no one in recent memory had watched three hours of sustained 3D, so they only went to about 75 percent 3D not knowing if audiences could take more. “Now it’s been proven that if 3D is done well people are comfortable with it,” he says. “So we pushed Hugo’s 3D further than Avatar on purpose.”
Much of that pushing is so integral to the storytelling that audiences may take it for granted. “Subtly converging on a person so they move from the back of the screen to the front of the screen over a couple hundred frames is almost like a dolly shot without the dolly,” Legato explains. “A lot of 3D gives the effect of moving within the same composition. It also blends shots like an invisible cut. It’s not jarring; you massage the convergence so the eye doesn’t leap as the edit changes.”
Grossmann, who had worked on the 3D Alice in Wonderland, was one of the few people on set with extensive stereo 3D experience. “When people were backing off [how far to push 3D] I told them with Alice we figured out we could have gone bigger. I kept telling stereographer Demetri Portelli, ‘It looks fine, but I’m pretty sure we can go bigger — we’re live, so let’s see what it looks like.’ Over the course of the show, as we did test stereo balances and edits, and people got more comfortable with 3D, we kept making it bigger.” Scorsese himself told Portelli, “Don’t skimp on the pate!” as he urged him to “crank up the stereo,” Grossmann reports.
Nevertheless, Grossmann admits to being nervous when he discovered that the stereo values of the biggest shot in Tron: Legacy were equivalent to the smallest shot in Hugo. “We checked and checked: We were four to six times bigger than any other 3D movie. But everything looked amazing. It could have been courage or just dumb luck, but we had the ambition to go big and feel immersive. We needed somebody who’d say let’s do this all the way, and Marty was that person.”
AN IMMERSIVE EXPERIENCE
He notes that on set Scorsese “never watched a flat monitor — he always watched stereo so he could make 3D adjustments, camera move changes, actor changes. As more time went by, he was able to judge 3D in the same way he’d judge the lighting or an actor’s performance.” Grossmann points out that even temp VFX shots early in the edit were done in 3D. “It’s been standard procedure that you’d only do a 3D version of the shot after the 2D version had been finaled,” he says. “But when Hugo artists submitted 2D shots I told them we couldn’t show them to Marty unless they were 3D.”
The automaton that forms the heart of the mystery in Hugo was both a prop mechanical man and a CG character. “Several versions of the automaton were built and programmed to function and draw,” says Grossmann (pictured above). “There were different stages of the prop as Hugo assembles it, and there was a stunt version, too.”
A CG automaton was required for animated sequences, when the mechanical man was thrown into the air and when a facial expression or some body language needed to be expressed. “In the last shot we couldn’t get the camera close enough to the automaton to get the framing we needed so we built a CG version to match the real specs of the prop so they could interchange it as necessary,” he adds.
The boy Hugo inhabits three worlds: his own world inside the walls of the train station, the station he cautiously ventures into, and all of Paris outside. To go behind the walls into Hugo’s hidden world “we built five different partial sets for the camera rig to go through, then filled in with CG as if it were one long Steadicam shot,” says Legato. A set that revolved around the camera was fabricated for a shot that takes audiences down a coal chute with Hugo. “To create the illusion that [the camera] followed Hugo down the chute 25 feet to the ground we did a previz and knew where and how to make the break to CG and stitch those environments together,” he reports.
Production designer Dante Ferretti built an enormous train concourse set at Shepperton Studios, creating the heart of the depot but ending each direction with a greenscreen to facilitate set extension. “The trains were CG, the ceiling was CG, the station façade was CG,” says Grossmann. “When you went down the hallways and corridors you went off into greenscreen.”
“Pixomondo’s work on the train station was so well done that you don’t know where the real set begins or ends,” notes Legato. “You get a fully realized sense of place.”
In designing the exterior of one portion of the train station Grossmann used as a foundation a photo he took of some buildings near the Opera, the same façades that Melies shot when he accidentally discovered stop-motion photography. “You can spend months analyzing all these nuances throughout the film,” he laughs.
LA’s New Deal Studios engineered the mechanics of the train crash, which culminates in a shot that mimics an actual photo of a train, which crashed and ended up dangling out the second-story window of a Paris station. “It’s one of the few miniature shots in the film,” says Grossmann. “The scene was a perfect candidate for a miniature — it would have been hard to shoot in stereo and added a huge level of complexity.” The crash was supplemented with CG set extensions.
Grossmann spent about two weeks in Paris during which interiors were mostly shot; four shots feature actual city exteriors. “One of Marty’s signatures in this film, since it was an homage to early cinema, was to have Paris look [like] the Paris of a 1930’s movie set, as if everything had been built and designed by set designers of the ’30s,” he explains. “Some details replicate French movies of the period. “Hugo was a swirling nest of homages to classic cinema from the early documentaries of Edison and the Lumieres to the slapstick comedians to early French dramas.”
For example, the view of Paris outside Hugo’s window, composed of matte paintings by Pixomondo, recreates the rooftops of the opening shot of director Rene Clair’s 1930 Sous les toits de Paris, he says. A nightmare train crash sequence has camera framings, characters and performances reminiscent of director Jean Renoir’s 1938 La Bete Humaine. “We even recreated the original screening of the Lumiere Brothers short of a train arriving at a station, which caused people to jump out of their seats,” he reports. “Other scenes pay homage to Buster Keaton, and Hugo hanging from the hands of the station clock is pure Harold Lloyd.”
When Hugo and Isabelle are on the verge of discovering Melies true identity, they sneak into his boudoir and find a box of his papers and drawings, which fly out and swirl around the room. “Marty wanted this to be a magical moment as the papers explode in an artistic flood of imagery yet keep it somehow plausible,” says Grossmann. “So we paid homage to the Muybridge photos and created a persistence of vision effect throughout the tumbling papers.
“We imagined that each paper was a hand-animated progression, so as they flutter away you get the illusion of a flipbook. The swirls of paper also have different images on each side, so animations come to life before the kids’ eyes. It took a lot to coordinate these shots, to create a beautiful swirl of papers and choreograph how to move them to give the sense of a flipbook.” Earlier in the film Pixomondo also crafted a CG flipbook as Melies scans the notebook he’s confiscated from Hugo and its drawings start to lift off the page in 3D.
In a direct reference to Melies’ discovery of the technique, hand-animated stop motion was used in the scene where Hugo repairs a mechanical mouse he’s damaged in the toy shop. “The mouse could have been a wind-up toy or CG, but we decided to pay homage to Melies and use stop-motion animation composited and blended together with live action,” says Grossmann.
Some vintage tricks proved hard to execute in stereo 3D or couldn’t be exactly recreated, such as iris wipes, he notes. Adding grain was also a challenge. Pixomondo ended up developing a technique to wrap grain around objects “so the objects themselves feel grainy and you don’t get the sense that the footage looks like a shower curtain of grain.”
THE TOOLS
Pixomondo began working early on with The Foundry to gain functionality in their Ocula family of plug-ins for stereoscopic imagery. “We told them what we needed to do, what we needed help on, and they came up with tools that we beta tested and which have now been rolled into the new release of Ocula,” says Grossmann.
For matchmoving — “the most difficult thing to do in stereo because there are two cameras mounted to each other but moving independently of each other” — Pixomondo tapped Andersson Technologies’ SynthEyes camera tracking software and added custom tools on top to better accommodate stereo 3D. Modeling and animation was done in Autodesk’s 3DS Max and Maya.
Legato is highly skilled at using the versatile tools within the Adobe Production Premium CS5.5, and on each movie he makes he uses the suite as sort of a “Swiss Army knife.” He deploys Photoshop and After Effects for VFX compositing, creating guide frames for VFX vendors, previz compositing, color look development, shot stabilization of footage using the new Warp Stabilizer and stereo compositing alignment checks. He also uses Premiere Pro, Adobe’s NLE, for digital I/O, quick conforms, sequence review and playback, and tape layoff, and Adobe Media Encoder for transcoding and multi-format deliverables, including Adobe Encore for creating Blu-ray DVDs for review by the production.
All of Pixomondo’s offices around the world took part in Hugo in order to meet the demands of the schedule. “We needed a 24-hour workday,” says Grossmann. “As one office went offline, two more came online. If Marty or Rob gave us a note in the afternoon, we had a new version of the shot for him when he woke up.”
It was no mean feat to choreograph the workflow and “move sequences around the world to stay ahead of the time zones and take advantage of certain facilities’ strengths,” he notes. “It was a huge engineering accomplishment to get everything working quickly and seamlessly.”
Pixomondo’s team of pipeline engineers and programmers customized Shotgun’s database management tool for worldwide deployment. “I could take dailies from Stuttgart and send a certain playlist of shots for review in LA, and Shotgun would initiate the file transfer to the screening room in LA,” he explains. “Then I’d click and get a CineSync review with Stuttgart. Or I could get Shotgun to make Avid MXF media files with metadata and Avid bin transfers for Marty and [editor] Thelma [Schoonmaker] in New York.”
The sheer amount of stereo 3D added to the complexity of the workflow. “When you’re creating left and right eye versions of every shot you’re doubling the amount of render time and doing something like four times the amount of work to produce a seamless composite or VFX shot,” says Legato. “It’s hard to be as nimble as a 2D movie because 3D affects your throughput: You have to devote more people and horsepower to it.”
THE COLOR
He notes that using the same viewing monitors on set as in the edit rooms and deploying the same FilmLight Baselight color correction system throughout helped make the process of 3D judgment calls seamless. “We color corrected in log space and delivered Avid and Lightworks dailies (play outs were screened on Avid; Schoonmaker did the creative cut on Lightworks) with external LUTs for the monitors. All the VFX shots used the same color correction system, so when Marty and Thelma were cutting, [the VFX shots] would look identical to the dailies they were used to seeing.
“Every time we screened something it got closer and closer to what it was really going to look like, and we could make very good tangible decisions. It wasn’t a matter of saying, ‘After we do the color correction it will look right.’ We saw exactly what it was going to look like.”
When Legato moved to New York to work with Scorsese and Schoonmaker, Laser Pacific (since purchased by Technicolor) built out a temporary DI theater in midtown Manhattan for him and colorist Greg Fisher and equipped it with Avids and a Baselight system for creative color correction and VFX cutting, compositing and conforming. The purpose-built DI theater ultimately rendered all the files for versioning and sent them to LA where the deliverables were produced.
“It was such a nimble working environment,” Legato says. “It was easy to hop on a machine and send files directly to the Baselight, then walk over to the DI theater to check it. It made it so much easier to see things live on the big screen in 3D while we were working.”
Thanks to this set up he was able to react rapidly to requests for changes and fixes. He recalls how quickly he was able to get the specific look that Scorsese wanted for a double-exposure sequence. “Marty needed it printed just so — not too bright or dim. It took minutes to see multiple iterations and get the right formula.”
Ditto for the last shot Legato cut in, a shot that Scorsese wanted have a heavier snowfall than was in the shot originally. “I had asked for separate snow elements so that I could have the opportunity to double or triple them in DI if need be, so when I showed Marty the shot with a new, increased amount of snow he loved it and finaled the shot. Instead of going back to the VFX house for a 24-hour turnaround I was able to deliver more snow in under 20 minutes. They were waiting desperately for this last shot to cut it into the film and finally print out the movie.”
If there were risks to pushing stereo 3D in Hugo the rewards have been many. “People who have pooh-poohed the technology and said it’s only for gimmicky films have now seen a director as credible as Scorsese use 3D to tell and enhance a story, not gimmick it up. They’re starting to see the possibilities of 3D for a regular dramatic picture,” says Legato.
“The question isn’t, ‘Why do you need 3D?’ but more, ‘Why don’t you want the added depth of 3D?’” he notes. “It’s like when color or sound first came on the scene. This added dimension, now in the hands of a talented filmmaker, becomes another tool, another vital element in telling a dramatic story. 3D can make an excellent film better and much more involving. It’s exciting to be on the forefront of 3D as it moves from gimmick to art form.”
For more on Hugo,
check out Michael Goldman's ARTICLE.