VFX FOR FILMS

Visual effects people's job is to trick you. And today some of the best effects are the subtlest. Add to that sleek new workflows and capturing lots of stills for plates and you're looking at the current state of VFX…If you can actually see them.
Even if you've only seen the Inglourious Basterds trailer, you are unlikely to forget the :01 or :02 showing an actor swinging a baseball bat — hard — at a kneeling Nazi officer's head. The batter, in Quentin Tarantino's story, is one of a group of US soldiers of Jewish background who've been dropped into occupied France. Their job is to kill Nazis.
So with that mission, and with that director, you're in for some justifiable homicide. But Tarantino's vision was to create a film that had the look and feel of its milieu. So the visual trickery — Oscar-winner John Dykstra is the VFX designer — is made to look real and tangible. Even when it's not practical — CIS Hollywood worked on 90 effects for the film, which Tarantino shot in Cinemascope with DP Robert Richardson.

CIS

Gregory Liegey has a long list of credits at CIS (www.cishollywood.com) for big-budget effects-driven films, many as a VFX supervisor or compositor. As the baseball-bat scene was originally shot, Liegey says, they only used a bat handle. The rest of the bat was CG and the actor playing the Nazi officer mimicked the impact of a bat. "The main issue we dealt with on these shots was giving the CG bat realistic heft," he says. "We had to experiment with shifting the fulcrum of our CG bat to find a balance between weight of the bat and the position of the [Basterd] actor's hands."
One important sequence takes place in a dairy farm. Most of the farmhouse sequence was captured on location with real exteriors, but certain shots were done on a studio replica of the farmhouse interior with greenscreen backing. "Working with Efilm to constantly reference the DP's [Richardson's] most current looks for the sequence, we had to match the background plates' color range and feel to the very specific lighting and color palette of the location shoot," Liegey says. One complicating factor was the location farmhouse windows' wavy "old glass," which CIS had to emulate with texture maps that distorted what you see through the glass.
The script — and the dialogue — called for cows to be visible grazing outside. But real cows at the location were impractical. "We were called upon to add stock footage of cows for various views out of and around the farmhouse," Liegey says. They used stock cows from a cheese commercial and "the challenge was to choreograph natural cow movement and then sculpt the lighting of the stock cows to the specific lighting of the farmhouse sequence. John [Dykstra] had very good instincts as to when a cow should move and part of his genius was choreographing the background action." Dykstra adds that he really appreciated Liegey and CIS's attention to detail, right down to emulating film grain. [Visit www.postmagazine.com for more from Dykstra.]
Additional CIS shots included adding a stylized iris wipe to reveal hidden bombs strapped to the Basterd's legs; removing breath vapor from actors on location for continuity purposes; and various wire removals, muzzle flash additions and other technical fixes.
The climactic sequence is set in a Paris theater where the Nazi high command attends the debut of a propaganda film. There's additional action in the projection booth where a local girl — the film's protagonist — works. "She helps execute the Basterds' plan for maximum Nazi destruction," says Liegey. "We created the movie-projector beam [as seen in the booth] and reflections, in addition to compositing in views of the movie theater and audience outside the projection booth. We developed a method of using the actual projection footage to create the natural striated flickering nature of the light beams, which also felt in sync with the movie image on the screen." 
Pac Title, which went into receivership in June, was the other main effects house. They created the effect of projecting footage onto the theater screen, including projection beams over the audience.
Inglourious Basterd's post deadline was "incredibly tight" due to the Cannes premiere in mid-May. Liegey adds that, despite his team's work on some violent shots, the film as a whole maintains a real sensibility. "The effects certainly don't control the movie; the movie controls the effects." Tarantino "loves everything about film," Liegey says. "It was a pleasure to work with him because he's so dedicated to the craft. It was inspiring."

CINESITE

Cinesite London's Michele Sciolette, head of VFX technology, and Holger Voss, CG supervisor, have seen lots of action on Harry Potter films and other fantasy-driven movies such as Charlie and the Chocolate Factory, Fred Claus, Hellboy II, The Golden Compass, Underdog and Beverly Hills Chihuahua. Sciolette and Voss are both working on next year's Clash of the Titans.
"Our visual effects technology is constantly evolving, depending on the shows that we are running at the moment, to meet the demands of our clients," says Sciolette.
"Recently, we have developed a realtime tracking system that works off the live video feed of the main-unit camera, which allows the director to preview our CG content when he is framing the shots on set. We have been using that on Clash of the Titans." The remake of the Ray Harryhausen fantasy flick is helmed by Louis Leterrier with Nick Davis (Potter; Troy; Chocolate Factory; Dark Knight) as VFX supervisor.
"There is a need to preview any big CG creatures or big CG content — to get a feeling of what is going to be added later on." Cinesite's technical assistance aided the production's recent location shoot in Tenerife and Sciolette says the proprietary system is still evolving "to be even more flexible." He adds, "There's a lot of interest in this kind of technology as you can apply it to a very broad range of shots with completely free camera moves."
Holger Voss has been involved in developing Cinesite's creature pipeline in recent years. One focus of the pipeline has been fur, which is understandable, given Cinesite's visual effects film credits such as The Golden Compass. In addition, the focus has been on the CG muscle pipeline and other tools while developing "a very robust workflow that can do a lot of shots and a lot of animations [without] mixing up versions between lighters and effects people and animation people," Voss says. "All the 3D creature work got connected into our production database and it's fully transparent for each artist."
CG artists start with Maya and then proprietary tools manage all the data.
So hasn't CG fur gone about as far as it can go?
"The big issue on fur is its interaction with actors." Voss says of a demand that he expects will come — fur dynamics also include fur-to-fur collisions. "There was nothing on the market to do complex fur work. Another big thing for everybody is furry creatures interacting with fire or water or other CG effects."
Sciolette says that on another film in the works, Prince of Persia, director Mike Newell wanted to have the freedom to zoom a lot during VFX shots, something that traditionally has been tricky. "So we have established a new pipeline to deal with lens distortion," he says. This involves attaching an encoder to each lens so that it records the position of the zoom ring for each frame. "We have built a system that takes that information and retrieves all the lens properties for that particular position of the zoom ring and calibrates lens distortions."
Cinesite has set up its own photography department, headed by Aviv Yaron. One big duty for Yaron's department is to travel to location shoots and collect thousands of high-dynamic-range stills that capture subtleties of the environment, lighting, textures and more "so that we have all the details of how the lighting was on the day. Recently we've improved our system to make it more efficient and accurate for cataloguing stills," Sciolette says, "so that every artist in the facility can go through these many hundreds of thousands of stills and find that particular frame that he needs to light our CG creatures."
Voss adds that HDRI (high dynamic range imaging) is also helpful for interior work, especially where image-based lighting can add realism to a CG creature moving through various rooms with different lighting.
Cinesite did the VFX for the recent film Moon and, Sciolette says, "our photographic team was working to capture all the information we needed to do the lighting of the CG elements in the spaceship and on the moon base."
Sciolette has been working extensively on motion-analysis techniques for visual effects. "Motion analysis allows you to track every single pixel along a sequence. For instance, it allows you to do extremely high-quality re-timing of sequences," he says.
recreating Hawaii
Hawaii. Wish you were there? At Comen Visual Effects (www.comenvfx.com), based in Santa Monica, they'll take you there, even if a film's stars were actually traipsing around the wilds of Puerto Rico.
For the new release A Perfect Getaway, VFX producer Josh Comen and VFX supervisor Tim Carras were the ones who trekked the rugged cliffs of Kauai, HI, to photograph stunning vistas, sheer drops and dramatic weather, all of which would be transparently comped into location shoots in Puerto Rico creating about 100 shots that looked like Hawaii. "We ultimately were the solution because we made Puerto Rico look like Hawaii," says Comen. Before founding his VFX company, Josh Comen worked as VFX producer on such films as Napoleon Dynamite; Saw and Little Miss Sunshine.
Just a few years ago, Carras adds, films produced at A Perfect Getaway's budgetary level would not have had the option to change location backgrounds on 100 shots without the benefits of today's DI techniques. The total shot count for the film, directed by David Twohy, was 183.
Cinematographer Mark Plummer shot most of the film on 35mm and most of it is shot outdoors. Flashback sequences, from the perspective of stars Steve Zahn and Milla Jovovich, were shot on a Red camera customized to shoot infrared images. "Vegetation goes pure white," Comen says, "clear sky turns black and everything has this shiny, high-contrast look to it." Waves are white but the water itself is black, adding to the unsettling feel. Some shots acquired on 35 had to be repurposed and desaturated for use in those flashback sequences, he adds, "so we ended up building some algorithms that would 'fake' film for digital to match those sequences."
Giving A Perfect Getaway a realistic Hawaiian setting was of prime importance to director Twohy, who is known to be "precise and accurate." Hawaiian settings include backgrounds of cathedral mountains rising from the center of the island; Kauai-specific atmosphere and rain patterns; and cloud patterns over the Pacific. "There's no other place on earth that looks the same way," Comen says, adding that the duo's trip to Kauai "was easily the most important time we spent in the entire project."
Comen and Carras shot about 400 stills of the Kauai environment on their Canon SLR.
Back in Santa Monica the job involved keying against the sky and "rotoing a whole lot of palm fronds." But the aim was to impart a highly dynamic, dimensional feel to the shots. Comen says, "A big component of what we did was compositing multiple plates together in layers of depth. We'd make a conscious choice to have three or four different photographic elements comprising the background. We'd track them all together in a 3D projection." Comen adds, "all the compositing in the show was done on Shake." Maya served as the package for 3D geometry; Mental Ray did rendering and Trapcode's Particular was the particles plug-in.
In addition to the actors' footage from Puerto Rico, Comen and Carras typically added a distant sky/clouds background; a layer of distant, misty mountains and atmosphere; the tall "Jurassic" mountains; and, between those, some additional layers of foliage to help blend the foreground with the background. Comen's lead compositor, Brandon Criswell, built a custom Shake macro that procedurally generates a specific horizon mist to help blend everything in. About nine compositing artists worked on A Perfect Getaway.
Comen VFX's sister company, Picturelock Post, is headed by Carras and provides DI services for many VFX clients, many of which are indie films often aimed at Sundance and other festivals. Color work is performed on a MacPro workstation running Apple Color with Tangent control panels with about 12TB of FibreChannel storage attached. Picturelock clients view a 50-inch calibrated broadcast monitor. 
"Over the last 30 years we've been on a wave of technological innovation where all these breathtaking new methods of using digital imaging have just exploded," Carras says. "Now it's time for everybody else to kind of catch up to that and see what we can do from a storytelling perspective with all those shiny toys."
Comen and Carras hope that A Perfect Getaway's audiences will ask, "Why are all these effects people credited?!"

IMAGEWORKS

Rob Bredow recently transitioned from VFX supervisor at Sony Pictures Imageworks (www.imageworks.com) to chief technical officer. He worked on the upcoming all-CG Cloudy With a Chance of Meatballs as both VFX supervisor and creative technology supervisor, and Sony Pictures Entertainment EVP George Joblove officially handed Bredow the CTO reins on July 1st.
Bredow feels that, with his live-action production and CG background, SPI can improve its technology/production relationship even more.
But aren't VFX for live action and VFX for animation essentially all a form of CG?
"There is an awful lot in common between our fully animated feature shows and our live-action effects shows," says Bredow. "On G-Force, we have a huge number of completely synthetic environments even though it's a live-action film." There was extensive manipulation of background plates that allowed for some late-in-the-game story changes. "Shots were changed in post production; we could build shots that we just didn't have the ability to shoot on-set. G-Force, while it's a live-action hybrid movie, had substantially more all-CG work in it than a film like Stuart Little did a few years ago." Experience on animated films like Surf's Up enhanced SPI's ability to handle huge amounts of data efficiently in their production pipeline.
Cloudy offers a good example of effects that can go both ways — live action and CG. "The system that we used to generate the stylized clouds in the movie that bring the food was the same rendering system —and some of the same artists — who worked on our [proprietary] SVEA system to create the Sandman character in Spider-Man 3," Bredow says. SVEA, Imageworks' specialized volumetric renderer which enabled artists to render enormous volumes of smoke and sand, was essentially re-used for Cloudy "with very few technological changes — just a lot of artistic direction."
He adds that "SVEA's entire job is to take a volume that has been calculated in Houdini or Maya and export it in various file formats and turn that volume into an image. With SVEA we can control the shading — the noise patterns on the clouds, the density of the way it calculates the light and shadows. We can render extremely detailed, high-density clouds relatively efficiently." 
Another valuable proprietary development is SPI's fire and smoke technology. "Our movie, Cloudy, actually used some of the same explosion simulations [from Ghost Rider] to give some really great, fun explosions," Bredow says. "Five or 10 years ago, when computers weren't powerful enough to give you the right answer, you'd have to do some very rough approximation to get the shot done and there was very little re-use because there were so many cheats we had to use. Today, they hold up in different uses. If you have a 50-gallon tank of fuel exploding, if you need that in multiple movies you can cut-and-paste that between them. We've spent a lot of our time recently making that very efficient so the set-ups can be minutes instead of days for some of these very sophisticated effects."
Artists can click on a desired SPI effect and basically drop it into their Houdini file and customize it from there.
Bredow is proud of Stuart Little's titular star. "He's one of the first furry, clothed CG characters to star in a [live-action] movie. We've been cutting our teeth on the leading edge of both fur and cloth ever since then. At this point it's not, 'Can you do a furry character?' Now it's how efficiently can we 'groom' the most complicated hair styles and put them in 500 shots. We actually grow the hair on characters and then groom the hair with digital tools."
Cloudy With a Chance of Meatballs boasts 70 or 80 characters all in need of "grooming" and they appear in about 1,400 shots. The creatures in G-Force had even more follicles to be looked after.
Surf's Up and now Cloudy benefit from a proprietary moving-camera system that lends the animation an almost hand-held documentary feel. And, thanks to workflow efficiencies, it's not that much extra work.
Challenges? Besides doing cutting-edge work, such as a fully CG character performing throughout the entire length of Watchmen (the blue guy), Bredow sees a challenge in the sheer amount of shots that scripts and directors call for today. Bredow is keen on being able to supply both: "We have a pipeline where you can do 1,700 gerbil shots if you need to!"

PRIME FOCUS

Chris Bond is a co-founder of 12-year-old Frantic Films, which has locations in Winnipeg, Vancouver, and LA, and recently changed its name to Prime Focus VFX. Prime Focus (www.primefocusworld.com) offers high-end visual effects services and also markets its own branded VFX software, such as Krakatoa. Bond's main focus remains the supervision and production of visual effects, as well as providing creative direction for all VFX projects completed at Prime Focus VFX.
Prime Focus provided previs for a majority of shots in G.I. Joe and created VFX for 124 shots, including about 70 that comprise the film's finale.
"In the finale there's a number of sequences taking place at the same time." Bond says. In a key sequence, the Ripcord character steals a super-jet called the Night Raven in order to catch up with deadly missiles endangering both Washington, DC, and Moscow. "We did this entire aerial sequence completely synthetically," Bond says, "there's no aerial photography or anything." The only organic images are close-ups of the pilot, Marlon Wayans, in the cockpit. "We built the entire plane around him. We did a CG version of the actor for some shots. The biggest thing that's innovative for us was to build this pipeline and processing tools to do clouds and big, big aerial vistas."
The purloined jet is attacked in flight by a green cloud of microscopic "Nanomite" technology — the same metal-devouring stuff that the film's villainous genius uses to topple the Eiffel Tower. "We wrote this whole Nanomite pipeline using one of our tools called Krakatoa," Bond says. "We wrote some new tools for Krakatoa [www.franticfilms.com/software] to be able to control and manipulate the particles." Prime Focus displayed its new particle pipeline at SIGGRAPH.
Krakatoa started out as a particle-rendering tool but has grown substantially. "Our particle renderer will render millions of particles — or billions of particles — in minutes. It's specifically designed to do high-density particle rendering," Bond says. "We've added a tool called Magma so you can generate your particles and then post-manipulate them." The Krakatoa/ Magma combo worked well for G.I. Joe to represent a deadly threat and also for next year's more benign Tooth Fairy, starring Dwayne The Rock Johnson (think in terms of lots of particles).
Prime Focus also created the clouds and distant vistas through which G.I. Joe's hero must pilot the jet. The matte-painting department first started with a 360-degree low-res cloud environment. Bond's people would then return a new version of the 360-degree work delineating where the jet's path — and, therefore, the camera's — would go so they knew which areas they'd have to make higher resolution. To this background the team combined near-range CG clouds with proper density values and proper parallax.
For the Washington, DC, scenes, Bond himself shot about 13,000 stills up the Potomac (they had to get clearance). "We captured the environment and built that inside the computer so we had this path that we flew up to the White House — and built a synthetic White House."
Describing the genesis of Krakatoa, which anyone can purchase, Bond says, "This all came from an idea a long time ago, where the best smoke rendering would be one where you actually generate sub-pixel-size particles and generate a lot of them and give them density value — then they'll operate and behave like real dust. Most people have done a volumetric approach — and our particle renderer will do volumetric as a fallback — but the approach is really brut-forcing as many particles as we can. It became a stronger and stronger tool in our facility and it pretty much gets used on every show that we have now to render fire, smoke, explosions."
Eyeon Fusion, which has historically been Frantic/Prime Focus's compositing tool of choice, served G.I. Joe as well. "I think everybody throughout the company has a seat of Fusion open on their desktop," Bond says. Prime Focus has written 12 or 13 plug-in tools for Fusion, which they market as Awake and which includes tools for matching film grain.
"The biggest thing we're working on now is a stereo-conversion process from mono images," Bond says, especially for repairing shots in stereo films with a computer-generated "second eye."