You just can’t keep a good pirate down, at least not if it’s Jack Sparrow, or, for that matter, any number of undead pirate hunters. For Pirates of the Caribbean: Dead Men Tell No Tales, Disney’s fifth installment of films based on a theme park ride, Johnny Depp returns in his iconic role as the swashbuckling Captain Jack Sparrow.
Also rejoining the series after missing the fourth film are Orlando Bloom as Will Turner and Keira Knightley as Elizabeth Swann. Geoffrey Rush returns as Hector Barbossa, the one-legged former captain of Sparrows’ Black Pearl. New to the franchise is Javier Bardem, Sparrow’s nemesis Captain Salazar, an undead pirate hunter.
The film has garnered a reputation as the most expensive of 2017 and second of all time. All told, it has 2,000 visual effects shots, with 150 of those shots full CG. The Moving Picture Company’s Gary Brozenich was overall visual effects supervisor, and MPC was the primary vendor.
“Half the film takes place at sea, but it was mostly shot on bluescreen,” Brozenich says.
With 11 ships in the film, the task of bringing them in, redressing them in dry docks and then sending them back out to sea would have extended the shooting schedule. Instead, although foredecks and rear decks for some ships were practical, CG ships often sailed the digital waters and pirates walked digital decks.
“The biggest tasks were enhancing Salazar and his ghost crew, building the ship they were on and creating the water that surrounded all the boats,” Brozenich says. “Also, the ocean divides and splits in two. Obviously that’s a supernatural effect. It sounds good on paper. But, it took a lot of head scratching.”
Leading the postproduction effort at MPC were Sheldon Stopsack in London and Patrick Ledda in Montreal. In addition to MPC, several other vendors worked on the film.
“Atomic Fiction was our next big creative vendor,” Brozenich says. “They did the opening sequences, the big town environment. There was a beautiful set, but anytime you can see beyond a four- or five-square-block area, you see their extensions. They also did some digital Jack Sparrow work when he’s on the way to be executed.”
Lola Visual Effects took Jack Sparrow [Johnny Depp] back to age 18. Rodeo FX composited around 400 shots, bringing them to life with added textures. CoSA VFX provided matte paintings. A small in-house team worked on the film, as well.
“And ILM [Industrial Light & Magic] contributed some bonus work on the film,” Brozenich teased. “You’ll see.”
Brozenich cites the water as the biggest technical challenge. “The volume of water work we needed to do, the challenge of ensuring that it blended well with live-action plates and real water, and having it perform as needed for the film,” he points out. “When you see ships in the film, they’re typically surrounded by CG water and an environment created for them.”
LANDLUBBERS AT SEA
In Helensvale (Queensland), Australia, a massive 150-foot-long (on each side) and 70-foot-tall bluescreen surrounded the practical ship sets. The extension was so big it would block the sun and shorten the shooting day. So, the crew devised a system that would let the sun shine in.
“We had containers with inflatable bluescreens mounted on top,” Brozenich explains. “We mounted these air walls in 20- and 40-foot sections that we lowered and raised proportionally to allow the sun to creep in. By deflating them, we extended the shooting day by three hours.”
Because the on-set ships sailed on turntables, and because the weather was unpredictable, the crew decided it would have been too difficult to do data capture of actors playing the ghostly pirate hunters during filming.
“It was a huge disappointment,” Brozenich says. “Giant [Studios] had come up with some amazing ideas on ways to create a capture volume, but the logistics became too much. It just didn’t make financial sense.”
Instead, the crew installed an extensive network of witness cameras: eight Sony a7s because of those cameras’ low light capability, and two Canon 5Ds that moved around the principal camera.
“We had four quadrants on each ship and moved the cameras to the quadrant where the action was taking place,” Brozenich says. “Later, we did hand tracking [of the actors].”
To have live-action plates of the water for reference and to use in the film, Brozenich shot footage in Australia and Key West, FL.
“The benefit of Key West is that it’s at the tip of Florida, so you can get a clear horizon at sunrise and sunset,” Brozenich says. “We used some of the water plates throughout the film with live-action skies comp’d behind, but directly cut those plates with CG water and skies. We hoped that if we kept mixing between them, no one would know.”
Creating environments for the ships on the bluescreen stage comprised the nuts and bolts of the water work for the film, but for some shots, the visual effects team needed to be able to direct the water. The director might want a big wave to hit a particular point, or want a certain quality of water, for example.
“We tried to base everything on the Beaufort scale to keep things as real as we could,” Brozenich says, referring to a measurement system that relates wind speed to ocean or land conditions. Rather than saying, for example, “strong wind and big seas,” the filmmakers and visual effects artists could be specific by using the Beaufort scale.
To create the digital water, the MPC artists used a combination of techniques and software packages, including Side Effects’ Houdini, Scanline’s FlowLine, Auto-desk’s Bifrost within Maya, and Dr. Jerry Tessendorf’s algorithms.
“Water simulation software typically produces a bit of a look and feel that audiences have become familiar with,” Brozenich says. “So, we took an approach to break that curse and went into it with more of an open mind.”
UNDEAD PIRATES
Sailing Salazar’s ship is a crew of 50 ghost pirate hunters, some of whom are live--action actors augmented with digital and prosthetic makeup, and some, CG characters.
“Salazar and his key officers all have damage that looks like they were hit with one cannonball,” Brozenich says. “One has his right side damaged, Salazar’s left side is damaged, one guy has the bottom half and front ripped out. When you look at the ghosts on the ship, there’s a narrative structure — one that we just made up.”
Other ghosts were missing so much of their bodies that they became full-CG characters.
One is only a cap and a hand. Salazar himself — that is, actor Javier Bardem — wears prosthetic makeup that’s enhanced digitally.
“We also replaced his eyes to give Salazar a shadowed retina look, and took off the back of his head,” Brozenich says. “The biggest challenge on the film was trying to work with Javier’s performance in a supportive, non-disruptive way. We had to add so much onto him, we wanted to make sure we were supporting everything he was trying to do.”
But that wasn’t the only challenge. This film’s conceit is that all the ghosts are eternally trapped in an underwater environment.
“We had to make actors shot at a normal 24-frames-per-second speed look as if they were floating in water,” Brozenich says. “Every-thing that can float does. Medals on their chest. Their hair, clothing, rags, costumes.”
For reference, the crew spent a day filming a stunt actor wearing a period costume while underwater in a pool.
“It’s one thing to believe you understand what that would look like, and another to see it,” Brozenich says.
Once they knew what the characters would look like underwater in full costume, they had to figure out how to replicate the look on dry land.
“Sheldon, Patrick and I spent a lot of time going through footage, trying to find the magic formula,” Brozenich says.
On set, all the performers wore partial costumes to make it easier to add the digital cloth and run simulations later. Those who would appear in close-ups in long coats wore chopped coats that stopped at their rib cage.
“All the ghosts also have CG hair and digital prosthetics on their bodies,” Brozenich says. “Their legs, arms, hands might be digital. They are all a conglomeration of digital prosthetics and practical makeup and costumes. So, we made amazingly detailed and naturalized digi-doubles of all of them. In a lot of cases, it made sense to rotomate and add the necessary [digital] parts back in. It was really a harsh pipeline of people working at MPC in Bangalore getting in there and just doing it.”
Three of the four previous Pirates of the Caribbean films received Oscar nominations for best visual effects, with the 2006 film Dead Man’s Chest winning the Oscar. And, all four films were box-office successes. The first, The Curse of the Black Pearl, earned $653 million; the second, Dead Man’s Chest, and the fourth, On Stranger Tides, topped $1 billion worldwide; and the third, At World’s End, brought in $958 million. This fifth film has a lot to live up to. But, like the ones before, it has unique, state-of-the-art visual effects powering the story and, of course, Jack Sparrow. As the irreverent trickster puts it, “No matter how difficult, I will always prevail.”
GHOSTLY COSTUMES
To give the impression that Salazar and his crew of ghosts are trapped under the sea even though they move around above water, the artists at MPC devised ways to flow the costumes of 15 hero characters as if they are underwater. Two of the characters are always CG. Actors wearing partial costumes played the others.
“That was a decision we made early on,” says MPC Visual Effects Supervisor Patrick Ledda. Ledda managed the group working in Montreal on the characters, while Visual Effects Supervisor Sheldon Stopsack supervised a London-based team working on water technology.
“We could have had the actors in blue or gray suits,” Ledda says, “but we decided to keep the practical [partial] costumes. They provided good reference for our look-development artists and the directors, and we could see how the material looks under lighting. The downside was that sometimes the costume was in the way. When that happened, we had to paint it out and replace it. With a blue suit, we could have put something on top. So, there were advantages and disadvantages, but overall, I think it was a good decision.”
The team created digital versions of each character, costumed the digital models, and received approval from the director. Then to create the shot, they tracked and fitted the digital cloth onto the partial costumes in the roto-animated characters frame by frame. A team in Bangalore did the tracking and roto-animation.
“It had to be perfect,” Ledda says. “Otherwise, you can get a strange jittering that you only notice later, and you have to go back to square one. We spent a lot of time initially to create rigs and controls to allow animators to really match the costumes with high precision. It was a huge amount of work: There are many, many shots with ghosts, and lots of ghosts in those shots.”
The roto-animation drove the cloth simulation for the characters played by real actors. Underlying animation drove the simulation for the digital characters. As with the hair, the cloth had to act and look like it was underwater.
“Imagine a character is wearing a long coat,” Ledda says. “As he walks, the tail of the coat lifts up and follows behind. Then, to give it an underwater feel, we add water currents. We placed invisible force fields in certain areas that would guide the simulation to give the cloth a little bit of nice, gentle movement from left to right. It’s like on a real set when you set fans in certain areas.”
The challenge was in creating a simulation that felt slow even though the characters moved quickly – more quickly than they could have underwater.
“Traditionally in animation you work in beats,” Ledda explains. “You make sure a character moves from one shot to another in a coherent way. But, when it comes to more technical disciplines like simulation, they aren’t used to working that. We had to learn to work in sequences. It took a different mindset.”
STATE OF THE ART
We asked MPC visual effects supervisor Patrick Ledda in what ways this film pushed the state of the art of visual effects. Globally, approximately 800 artists at MPC’s studios worked on the film. In broad terms, artists in Montreal worked on the characters, while those in London developed tools for water simulation, parted the seas and created the third act.
“For us at MPC in Montreal, it was the hair work, the technical animation and the simulation of cloth and hair,” he says. “The amount, the accuracy, the art direction needed was a first. We hadn’t pushed that kind of work to that level. The simulation needed to be the main actor in a way. In London, the work they did on water was on the next level. It’s really amazing.”
SHARK ATTACK
The crew on Salazar’s ship aren’t the only ghosts. MPC artists created ghostly birds, and for a long, complicated sequence, digital sharks.
Here, Salazar’s crew throw three dead sharks from the ship into the sea. As soon as they touch water, they start swimming and chasing Jack Sparrow (Johnny Depp), who is in a rowboat. The sharks jump the boat, attack it and bite chunks out of it — out of, in fact, a digital boat.
Although the sequence has a few shots of the boat filmed in a tank, for most shots, Johnny Depp rowed a gimbaled boat on dry land. MPC artists surrounded the boat with water and sky, and put an island in the background.
“There are very few pixels on the screen that are practical,” says Patrick Ledda, MPC visual effects supervisor. “There were wide shots with water, and water in any shape and form is complicated. And it was a long sequence, with approximately 150 shots.”
To have the dry-land rowboat interact properly with the digital water, the team built a placed digital rowboat in the CG sea. During the shark attack, they replaced the practical rowboat with a digital double and, occasionally, Johnny Depp, too. Then, toward the end of the sequence, the ghosts jump off the ship to join the chase.
“There’s a beat where you see the whole crew running on water,” Ledda points out. “Those are fully computer-generated shots.”