HOLLYWOOD — How do you top the surprise 2003 blockbuster, Disney’s Pirates of the Caribbean: The Curse of the Black Pearl? If you’re director Gore Verbinski, you assemble the same crew — including stars Johnny Depp, Orlando Bloom and Keira Knightley, as well as the below-the-deck visual effects talent — and quickly set sail for more exotic ports of call. And taking a cue from another monster production, The Lord of the Rings, you grit your teeth and take a daring walk on the plank by shooting two sequels — this summer’s Dead Man’s Chest and next year’s as yet-untitled-film — back to back.
“The biggest challenges were the sheer physical demands and exhaustion from shooting for 200 days,” reports Verbinski, whose credits include The Ring and The Weather Man. “To be honest, it’s just mad making two films back to back. As my wife said, ‘You’re no longer the architect, you’re the contractor.’ And you’re also facing all the audience expectations and trying to design the film to have surprises and twists and even better visual effects than the first one. There’s so much pressure involved, but you just have to go for it.”
To help him try and get a handle on the two-headed beast, the director called up Lord of the Rings and King Kong director Peter Jackson. “We talked about everything from the problems of scheduling effects shots to maintaining crew morale and the need to take your breaks whenever you get them,” he reveals. As it turned out, Verbinski and his crew got more breaks than they expected when production was shut down twice because of hurricanes in the Caribbean, forcing shooting of the third film to be split up. Final filming is set to be completed this month in the Caribbean and on soundstages in Los Angeles.
THE VFX CHALLENGE
The pair also discussed the huge challenges posed by the sequel’s demanding post schedule, which included the completion of three times as many visual effects shots as in the first film. In addition, many of these shots were “far more complex” than anything in the first film, largely because both Verbinski and producer Jerry Bruckheimer were determined to raise the bar.
“For instance, in the last one we had the device of moonlight, to step in and out of shadows when we wanted to show the effect [of the ghost pirates],” explains Verbinski. “From the start we decided not to have that place to hide this time. We had to be bold and play the complete CG characters as if they existed in the real world, and the first big challenge was designing Davy Jones [captain of the ghost pirates].”
The director began by doing a lot of “really bad” pencil drawings before collaborating with artist Crash McCreery. “We talked a lot about the origin of these characters, coming from the bottom of the sea. So I didn’t want them to be ‘ghost pirates,’ which is how they were originally conceived by the writers. I wanted them to be physical and tactile, which led to them having all this rotting sea life on them. And from that, we came up with characters formed and mutated out of shells and barnacles, to the point where one character has become part of the ship.”
To deal with all these creatures from the depths, as well as preserve actor Bill Nighy’s mocaped performance as Davy Jones, and create a mythic sea monster called the Kraken, the director once again enlisted the help of Industrial Light & Magic (www. ilm.com). “We worked extensively with ILM’s John Knoll [visual effects supervisor on the film], and right from the start we had these very in-depth discussions about how we were going to approach all this and make it happen on time and on schedule,” he says.
Indeed, post was ultimately completed in little over a year, “not very long for a project this big and complicated,” reports Knoll, who definitely knows of what he speaks. Over the past two decades, Knoll has worked as the VFX supervisor on some of ILM’s biggest jobs, including Mission Impossible, Star Wars: Episode III Revenge of the Sith and the first Pirates movie.
“The Star Wars films were all done over a two-year period,” he notes, “so this was a lot tighter in terms of post, especially when you consider that we didn’t even get all our sequences turned over until January this year.” Dead Man’s Chest ultimately needed some 1,400 visual effects shots, ranging from the very complex to more normal stuff, like wire removal and clean up. And of that huge number, close to 1,000 shots were done by ILM. “That’s more shots per week than we were doing on Star Wars, and it’s the most intense schedule I’ve ever worked on. And most of these shots were extremely complex.”
Knoll goes on to note that, “All the easy stuff got farmed out to other companies, including Asylum, The Orphanage, Method, CIS Hollywood and Pacific Title pretty early on. We were left with all the hard stuff.”
DAVY JONES
In terms of “the hard stuff,” Knoll breaks down the challenges involved into three main areas: “First, creating Davy Jones and his whole crew, who are all entirely CG; then creating two very complex action sequences where this huge tentacled monster, the Kraken, attacks two ships; and finally everything else.”
For scenes with Davy Jones and his undead shipmates, the main difficulty was “having to provide a good, realistic performance in a variety of sizes — wide, close-up — and with a whole range of emotions, from anger and rage down to very subtle gestures and looks,” explains Knoll. “So they had to perform very realistically next to the live actors, and that was the first thing we began working on, even before the shoot started.”
To achieve the look they wanted, ILM first designed and developed a brand new, specialized motion capture suit to be worn by actor Bill Nighy. “The idea was to come up with a system that would deliver the quality of motion capture, but which would have a low footprint on set, so we could take it on location without slowing down the shoot and putting unreasonable constraints on stylish photography,” he adds. “So this special suit let us recover his skeletal motion, and he acted on set the same way as the other live actors, giving us all the benefits that come from that. Gore could direct him, and the operator could frame up just the same way, and then the editor had a performance he could cut into the film. So stylistically, there was no break between the live action and the CG scenes.”
The recovered skeletal motion then went to the animators who worked on the CG Jones and crew. “We split-screened the performance,” says Verbinski, “so consequently there was a lot of hand-tracking going on and a lot of physical animation, but they were also constantly referring back to Bill’s performance, so we’d have a little eye compression, a nuance. And it was a big problem losing his jaw-line because of all the tentacles. So we had to figure out ways to translate all that and then capture it in animation, especially his eyes.”
The director says that he originally planned to keep the actor’s own eyes throughout all the animation, “but we ended up using completely CG eyes in the end,” he reports. “And if you split-screen the two performances, it’s amazing how close they are, and all the animators were constantly saying that Bill’s performance was both very inspiring and really challenging, because he’s so inventive. So the animators couldn’t create their usual library of facial movements and looks. Every shot had to be done from scratch.”
“We had up to 60 animators on this show, and the big challenge with Davy and the rest is that, though they’re mutated and distorted, they’re still human characters,” notes Hal Hickel, animation supervisor at ILM. “That always makes it far more difficult as the eye is so familiar with human motion and movement. We didn’t want them to end up just looking like very good CG characters.”
To avoid that, the team first tracked the actors’ movements with mannequins that conformed to their proportions, using in-house tracking software. “That was then exported to [Autodesk] Maya, so the animator gets a scene that has a CG camera matching the one in the scene,” explains Hickel. “The mannequin matches the actor’s moves on set…to a degree. It isn’t perfect, but a pretty good facsimile, which needs a little tweaking. We wrote some scripts that’d bring in the appropriate character for that actor, so if it’s Bill Nighy, we bring in the Davy Jones creature rig and layer it on top of the mannequin.”
Because of the slight differences between the actors’ and creatures’ proportions, the animators then did a “re-targeting” or fitting procedure. “It’s kind of like fitting a suit, and making sure it all looks right,” he adds. “And from there, the animator finesses and tweaks it to make it all fit within the scene. Gore might ask for changes from what the actor did, or he may like what he did but then it’s not interacting quite right with the set. So all those adjustments are done in Maya.”
Once the team was happy with that process, “we brought it back into Zeno, ILM’s platform for all our proprietary tools, and did all the facial animation,” Hickel explains. “We’d study what the actor did in the plate, assuming that Gore wanted that. And in the case of Nighy, it was almost completely faithful to his performance. Very little was changed. But sometimes with background characters we had to do something very different, usually for editorial reasons.” With facial animation complete, “the next step was all the rendering,” he says.
KRAKEN
For the sequences with the Kraken — a sort of monster hybrid of an octopus and squid — the team faced other challenges. “Gore felt that it’d be more scary and effective if we never quite see its body, so we focused on its tentacles,” says Knoll. “Those are everywhere, reaching up out of the sea and snatching sailors and smashing things, so it made sense for us to model a whole bunch of individual tentacles, and we did a lot of previs for all that.” Starting with a design of the whole creature, the team then realized that most shots could be done by using, “as many disembodied tentacles as needed” placed throughout the shot, he adds.
“It was all keyframe animation done on Maya,” reports Hickel, “and we spent a lot of time and effort getting our controls right, as things like tentacles are very hard to animate. Once we’d done the movement with Maya, we also did a flesh simulation pass on some shots, where it’d be more blubbery, and that was all done on Zeno.”
Although the team initially modeled the tentacles with regular rows of suckers, “Gore wanted them to be more scarred and messed up, so we came up with a system that let us procedurally replace suckers with a random assortment to get the needed look,” he adds. “That was also done with Zeno, after the animation was done.”
Such scenes were then enhanced with various atmospheric elements, including smoke and spray. “My general philosophy for all that kind of stuff is to do as much as possible live on set,” states Knoll. “And it makes a difference to the photography, as smoke acts as a bounce card which changes the overall contrast. So I prefer to shoot the plates with all those elements present, even though it makes the compositing much harder.”
Ultimately, the team “did create a very hard compositing situation for ourselves,” he admits. “We had to put tentacles behind one piece of smoke but in front of that water spray, and so on. It helped that we have a big library of smoke, spray and wood splinter elements we’d shot, so we’d comp bits and pieces and build it up that way.”
TENTACLES AND MORE
For Hickel, whose credits include A.I. and the first Pirates film, even more challenging than creating Jones and his crew or the Kraken’s tentacles was the creation of Davy Jones’ tentacled beard. “That was a huge technical hurdle, as it had 46 tentacles all moving and colliding with each other,” he notes. “There was no way you could do it with traditional animation.”
Instead, the R&D team wrote new tools for their simulation engine. “It basically divided all the tentacles up into separate little joints, with a little motor at each joint telling it how much to bend and so on,” he adds. “And we created these coiling behavior patterns and added that.”
The last piece of the puzzle? A new program that the team called “Stiction” — “a mix of stickiness and friction. We didn’t want all the tentacles writhing around too much as Davy Jones walked, and that program let us keep some of the tentacles pressed against his chest more while the ends moved around.”
Other major VFX included various ships at sea, including The Flying Dutchman, which breaches from the sea at one point like a surfacing whale. “So we had to create a lot of CG water effects for that,” adds Hickel.
Knoll reports that while many of the visual effects represent “incremental advances” from previous films, this sequel also showcases some new software and technology. “RenderMan now supports a feature called Arbitrary Output Variables,” he says. “And as you render, instead of just rendering to a single RGB image, you can separate out as many ‘products’ as you like, such as reflection from ambient from diffuse, and so on. It gives a lot more flexibility in the composite.”
This proved invaluable, especially with Jones and his crew. “It really helped make them all look as good as they do, as you can make little tweaks as you go.”
Knoll reports that, “This is the first show to go through ILM’s new creature pipeline. That system was first developed for Jurassic Park, and had needed a complete overhaul, which we began two years ago. War of the Worlds and The Island went through it, but Pirates 2 is the first ‘creature-heavy’ show to use it, and we’re pretty pleased with the results.”
“It’s definitely the biggest, most challenging production I’ve ever been involved in, but I’m very happy at the way it turned out,” sums up Verbinski. “Everyone from the ILM team and [editors] Craig Wood and Stephen Rivkin, to [supervising sound editors] Chris Boyes and George Waters did an amazing job of bringing this film to life.”