THE ART OF COMPOSITING
By: Christine Bunish
Issue: October 1, 2005

THE ART OF COMPOSITING

The art and science of compositing is part and parcel of just about every product these days: TV series and movies, commercials, feature films, games and music videos. Post asked a panel of compositors for their views on the status of compositing today, the challenges they confront and why they love what they do.

How has compositing changed/evolved during your career?

LLOYD BARNETT: "It has changed dramatically. I got in at the tail end of optical effects when I worked at CFI, when they used Oxberry cameras and printers. That was a strong base [of skills to have]; you were running film through your fingers doing amazing things before the digital revolution with 'trick' photography. Now compositing tools are unreal. Every year they come out with more."




PETER AMANTE: "In my eight years as an effects artist, the biggest change is that HD is no longer an option, it's a way of life. It's not uncommon to start a job in SD only to have the client decide to go to HD. Though this slows the process a bit, it's not the burden it was several years ago."

DAVID EBNER: "When I first started out there really was no digital compositing. The early compositing tools were slower than the 3D tools for rendering, so I often used the 3D applications and layered things that way. Green- and bluescreen extraction was done in a separate tool and it just rendered the matte and suppressed foreground images. We even programmed our own tool for this, as the ones available were not very good. When digital compositing evolved into a dedicated discipline, the advantages of having quickly-adjustable tools that [could] alter your elements and composite became very apparent, and a new workflow flourished. The next evolution in compositing has been the addition of a 3D environment... and perhaps the best innovation in the last five years, related to working in film, has been the jump from 16-bit color space to floating point."

RICARDO TORRES: "I was lucky enough to start 15 years ago in the early days of digital compositing. Today you have much finer control over composites. More traditional 2D compositing applications are using a 3D compositing environment, like Shake, while more desktop solutions are adding what was considered high-end features such as precise tracking and vector-based warping to their arsenal. As toolsets get a lot more complex, a lot of legacy compositing software is being rewritten from the ground up: there's only so much you can put on top of old code."


The work can engergize you

 

DARIUS FISHER: "The first time I wore the hat with the label 'compositor' emblazoned on it was in 1991. I was using Cosa After Effects V.1 for bluescreen compositing for the interactive game title Burn Cycle. Since then the machines have gotten faster, the monitors bigger, the software more powerful. I started with motion JPEG half-size PAL QuickTimes, and now I'm doing up to 4K uncompressed compositing on features. They take about the same amount of time to render [as the games did]."

 

 

 

 

PETER MOYER: "When I started compositing for television, the [Abekas] A84 had just been invented, and I was compositing in realtime. As the industry transitioned to computer-based effects in the early '90s, the compositing power increased, but the processing slowed down as everything was done frame-by-frame. Today, with the realtime advances in computers and video cards, we have tremendous compositing power in realtime; the process has come full circle."

What is your preferred platform for compositing, and why?


For Cadillac´s Bonfire, Below the Radar´s Ricardo Torres, who uses Flame and Shake, composited in this very believable coastline. The spot was actually shot in the desert.
BARNETT: "I've tried many packages but my favorite is [Apple's] Shake. Everything is right there and I can get under the hood - it's easy to customize at code level. Here at Krypton VFX we're moving some of our workstations to Linux so we can make Shake available for all of our artists. We will also be running Shake on Mac. Shake Version 4 is supposed to be solid on Mac, and our buddies at Melrose Mac in Burbank keep us stocked with Mac gear."

AMANTE: "Flame and Inferno... for commercial work. I've never run into a situation I couldn't handle in Flame. And the right artist makes anything possible. For the independent feature The Last Winter by writer/director Larry Fessenden, I'm creating the VFX in Shake and After Effects. I can see some of the limitations of Shake compared to Inferno, but with a tight budget and a flexible schedule, the cost effective solution is Shake. But my tool of choice is always by Discreet."

EBNER: "For CafeFX the clear choice is [Eyeon's] Digital Fusion on an XP 64 system. It is very powerful, responsive, more intuitive with each new release and, most of all, superfast. It has twice the tools built in as Shake, runs in a 64-bit environment, supports floating point very well and has a true 3D compositing environment. Its particle system is also quite amazing."


For Mortuary, Krypton VFX created a mixture of 2D and 3D smoke using Combustion and Maya. The studio's compositing tool of choice is Shake.

TORRES: "It's a toss up between Shake and Flame. I use both equally. It used to be that you had to have Flame for clients to take you seriously. Now the majority realize it's not the software that makes the difference but the artist. They don't care what I use. I tend to use Flame mainly for SD commercial work; it's a very fast tool when I have supervised sessions. If I'm doing high-rez feature work, Shake is my tool of choice. The quality of its software-based renders is, I think, better at high-resolution, especially when projected on the screen. Often I switch between Flame and Shake on a shot."

 

FISHER: "I'm a little biased since I've been using [Adobe] After Effects for such a long time. But it's my favorite for blue- and greenscreen compositing and 2D and 2.5D VFX work. It's always been incredibly versatile and, I feel, intuitive. After Effects continues to meet my needs and expand into them as the product evolves. It's extremely affordable and third-party developers continue to come up with great solutions not necessarily housed within the core product. It's a very exciting time in terms of all things to do with the moving image, particularly since we're on the verge of some sort of transition to digital cinema. Photoshop CS2 now supports HDRI and we're hoping After Effects will soon also. We're looking forward to exciting new possibilities in forthcoming versions of Adobe products."

MOYER: "We've been working with Shake almost since the beginning. Digital FilmWorks' senior compositor, Marco Paolini, wrote the book on Shake - literally. But, although we've found that Shake can do everything, it's not for the faint of heart, due to the extensive use of individual nodes. We used to rely solely on SGI systems, but today we prefer Linux. While the serious graphic applications have been ported to Linux, not all applications are available yet. We are awaiting some new rotoscoping and paint tools geared more toward compositing."

What advice would you give to someone who wants to become a compositor?


For the indie film The Last Winter, Peter Amante composited this greenscreen shot with a practical explosion via Inferno.
BARNETT: "I'd say compositing isn't for everyone. It's problem solving and art rolled into one; a blend of technology and art. So along with technical knowledge, it's just as important to study art, study color science, go to museums. After a while you'll step outside and in your mind you'll be replacing the sky with a giant matte painting!"

AMANTE: "You have to have an eye for it. Too many people get fixated on the software, but you have to know what looks good and how to get [the shot] to that state. So the advice I'd give is to train your eye and don't get so caught up in the technology. Understanding why something looks good is more important than knowing which sliders to use. Besides, by the time you're out of school there will be new software to learn."

EBNER: "Start with photography and learn digital manipulation. It is important to understand photography, composition, color, exposure, etc. Pick up a program like After Effects since it is so inexpensive and start working with footage. Learn the basics such as roto, matte extraction, layering. Make some creative demos that show your talents and try to get people in the industry to look at it and tell you where to go from there."


Creative democratization
TORRES: "More important than learning software is [taking time] to do things like watch a sunset or sunrise: study it because you will have to build it from scratch in some composite. It will have to look real, and what makes it look real are the many little subtleties, things we don't take the time to look at. If I had known I'd be [compositing], I would have taken a photography class early on. Compositing is taking different elements and making them look like they're shot on the same day, with the same light, with the same camera. You need to look at the reality around you and apply that to your composites. That's the difference between a compositing artist and a compositing operator."

FISHER: "Create an excellent reel that shows you have a good eye and know when a composite looks real. Show how you did the composite with a good break down. The entry point for compositing is often rotoscoping so find a creative way to show off your rotoscoping skills. If you can [demonstrate that] you have studied reality with the eye of an artist, show what you did and how you did it, and [display] your sensibilities, you have a good chance of breaking in."

MOYER: "Every year I host the Television Academy's intern for special effects and compositing. The advice I give them is that to start, the menial tasks you learn, such as rotoscoping, dirt clean-ups and wire fixes, get you working on images. But you also need to develop a creative eye, and you can't learn that at a computer. You have to discover it by observing real-life objects, and once you can reproduce that on screen, you have the eye. I give every intern the same two-element shot to composite and from that composite, you can see what kind of eye that person has. One person will just put the object over the background. Another will add color correction. A third will add shadows to match the lighting, but the fourth will put it all together, and that's the guy who will be a great compositor."

What are some compositing challenges you have faced?


DNA had to composite Cate Blanchett out of one background and into another for this scene in The Aviator.
BARNETT: "I was recently working on Toby Hooper's Mortuary, where the creature gets inside of people and animates them. In one shot a girl's breasts explode out of her blouse. There were no practical effects and it took a lot of finagling, a lot of smoke and 3D elements. In Chris Solomine's Moscow Chill, two different planes were used for two shots but they were supposed to be the same plane. We were asked to take the stripes off the second plane, but that was very complicated with the light reflections on the aircraft. So, instead, we created a 3D plane, removed the original plane and flew the 3D plane through the scene."

AMANTE: "My biggest challenges have been time constraints rather than shots. Clients are asking for more and more scenes to be treated without extending schedules. I had four weeks to do four spots for Best Buy, which required me to fly to Minneapolis every week for two or three days of on-set supervision. I'd fly home and work on the spot for the next five days, then fly back to supervise the next shoot. On top of that, they were all in HD, and all required a massive amount of rotoscoping. But using Flame did make them manageable."

EBNER: "For many challenges we have had to come up with innovations in technique or combine tools in new ways or program a new tool - some of which have turned into common practices or tools now. A real difficult shot I did for [the feature] Dracula 2000 involved removing a wall of trees and blending CG battered-down trees, water and plane crash debris. What made it especially hard was the camera move: the camera approached, lifted and then circled the area. I had to come up with a way of doing 3D roto mattes and replace and blend the 3D water with the real water and extend the water that was covered by the original trees. The water glistened and changed color all throughout the shot. I created a CG glistening layer as well; color correction was animated frame by frame."

TORRES: "Sometimes circumstances can turn a regular shoot into a challenge. In the Cadillac spot Bonfire, these cars were supposed to be driving by a coastline with a sandy beach on one side of the road and the ocean on the other. The day before the shoot in Malibu, they found that a protected bird was mating in the area so they were not allowed to shoot there anymore. They decided to shoot in the desert and add the water later. They didn't have time to plan for a VFX supervisor so it was shot as it was boarded for the coastline with standard camera moves and no motion control. Then later [the production] shot locked-off water plates of the ocean in Malibu. I used tracking and warping tricks and other visual deterrents like pretending there was some wide-angle lens distortion to integrate the plates of the cars with the water. The director and agency were amazed. I actually got a call from a location scout asking me where the location was - they needed it for a movie. I had to say, it doesn't exist!"

FISHER: "We had a shot on The Aviator with a three-day turnaround which had been described two weeks earlier as verging on impossible. The editor, Thelma Schoonmaker, was set on using two shots [of Leonardo DiCaprio and Cate Blanchett] on the red carpet at a premiere back to back, and they had the same extra in a leopard-skin outfit passing in the background. The VFX supervisor found a piece of footage not intended for use: The camera was rolling at the end of a take. It wasn't quite the same angle and it was just the people and the press lining the red carpet, no principal actors. The challenge was to remove Cate from her environment, with the camera flashes illuminating her and her wispy hair, and composite her into the found footage, which had its own cadence of flash bulbs going off. Since the shot was no more than two seconds long, we did it frame by frame in After Effects. We rotoscoped Cate in sections and then color matched each frame and each firing of the flash bulbs to blend her in with the new background. When our first version was screened cut into the sequence the VFX supervisor Rob Legato asked which shot we had removed Cate from. He apparently couldn't tell which was the composite shot, which made us feel good."

MOYER: "The worst was a shot for a movie where the greenscreen was shot with the wrong film stock. The grain in the image was larger than some of the objects in the shot! We tried every idea and asked others how to handle it. We ended up hand rotoscoping the whole thing, and it never looked quite right. Due to a technical mistake, this was a shot that should never have been in the movie, but it was considered too important to cut. On the other hand, for A Midsummer Night's Dream, we were on set for a dress rehearsal of a greenscreen shot with Michelle Pfeiffer's stand-in who was backlit on a rig, her hair floating and thin. The DP tried different film stocks, exposures and filters, and together we found the combination that would composite best. So when the scene was shot the next day with Michelle we got the best greenscreen ever. This little bit of planning made the end result way easier."

What common misconceptions do clients have about problem solving via compositing?

BARNETT: "Some clients put all digital effects in a box labeled 'black magic,' but more of them are getting savvy and understand the process. Some misconceptions come from denial: they say, 'Make my photography better than it is,' but you can't make a silk purse out of a sow's ear. If the photography is good and the greenscreen is bad we can fix that. But when the photography is really bad we would have to replace everything in the scene to improve it."

AMANTE: "Scenes can be much more involved than they appear. What looks simple can, in fact, be very complicated. Lighting, tracking, roto and animation all play a part in creating a seamless composite. But I have to say that many of my clients are well versed in what is involved, which helps move things along."

EBNER: "Mainly that it is real easy to integrate anything. Today clients have a lot of faith about what can be done. Because we live in a world where many movies have such amazing effects, some clients are not well educated and take for granted that anything can be done. This frees them up to not worry about planning for effects, which is a bad trend for the industry - especially when we keep making everything look good despite the footage we have to work with. But there are many clients who are on the ball."

TORRES: "One of our roles is to educate the client on what's possible and not possible. That gap is getting smaller and smaller. You can attack almost any problem. But a lot of filmmakers are getting lazy. They say, 'Don't worry; we'll fix it in post.' But compositing is only as good as what you give me to start with. I'd rather spend eight hours making a shot look great instead of fixing it to make it look like it's shot correctly."

FISHER: "People think if they put a bit of blue or green in front of something that things just disappear. This can actually make it harder for you. We do a fair amount of screen replacement - on monitors, PDAs, cell phones - and I find myself wishing sometimes that the screens weren't covered by a bit of green cloth. There are so many options with tracking nowadays, that if a reflective gray or black material is used instead, you don't get spill and can retain beautiful reflective detail."

MOYER: "People do come in thinking anything can be fixed in post. In the old days, those shots weren't in the movie. And in some cases, that's as it should be. It's not a question of fixing things in post, it's a question of to what degree can you fix them. Just because a shot can be done, doesn't mean the end result should be on screen. And sometimes, this is difficult to convince a client."

What was the most bizarre thing you have ever had to composite or remove from a shot?

BARNETT: "We had to remove an annoying sunlight reflection off a car windshield. It was so long ago, I don't even remember what film it was. [But] with the camera angle and lens parallax it took forever to remove. We had to create keyframes, which were clean plates, and morph them together. In another project, the MPAA thought the gun in a movie trailer was pointed too much toward the camera, so we did a 2D cheat to rotate the gun to a 45-degree angle and repaint part of it. That was not difficult; just seemed like an odd request."

AMANTE: "I once worked on a commercial in which there was an elderly man standing up on his front lawn... the client wanted me to put some movement into him, move an arm or leg so he wasn't static because it looked like he was dead."

EBNER: "I've patched in hair, covered or reframed breasts. Some of our guys smoothed over nipples that were prominent under a T-shirt. The most bizarre to me was removing a team of videographers who were running through a war sequence in a feature. They were covering it for documentary reasons for the film but were not supposed to be in the shot."

TORRES: "In the mid-'90s when I was at 4MC there was a scene with Woody Harrelson in the TV version of the movie The Cowboy Way. He comes walking out of a cabin naked, with a cowboy hat covering his front. One of the decency rules for NBC was you couldn't show naked thighs. The movie was ready to air that night, and I had to enlarge the hat - there was no other solution. The tools back then weren't as good as today, so that hat became a sombrero and the hand holding the hat was two to three times the size of Woody's other hand! Similarly, when I was at Anderson Video, [Steven] Spielberg was asked to remove the scenes with nudity from Schindler's List so it could air. He said no, they're part of the story, so we had to come up with more sophisticated tricks like extending bed sheets or playing with dark shadows to hide certain body areas."

FISHER: "There was a shot in Starship Troopers 2 where we had to remove a G-string from a woman in a side shot to give the illusion she was wearing less than she was. My partner, Melanie Franciosi, did it beautifully!"

MOYER: "There was a scene in a motion picture with an 'A' actor who had a cold. It was a long dialogue shot and he was sniffling and stuff was coming out of his nose. We had to remove all that."

Do you think compositing is something you will be doing for the duration of your career?

BARNETT: "I did VFX supervision for Mortuary, and I'd like to do more of that. But no matter what I'll eventually be doing I will always enjoy sitting at a workstation and cranking out shots as a compositor."

AMANTE: "I think I will always have a hand in compositing. I don't want to get off the box any time soon. But if I were to look at a next step, it would be directing. I had a tremendous experience as the effects supervisor for The Last Winter. Larry Fessenden was great; he allowed me to do a bit of 2nd unit directing for several scenes. Having a compositing and VFX background could be very beneficial for a director."

EBNER: "The blending of elements, 3D and 2D, including acquisition footage, is something I'll be involved with for quite some time. It is part of filmmaking. And it is really fun to see the evolution of raw footage into a work of art."

TORRES: "It's the goal of a lot of my co-workers to be on the set and become VFX supervisors or technical directors. For some reason I don't have that desire yet. I still love compositing. It's an art form, not a job. I don't see compositing as being 'stuck' because it's always changing. A few years ago I knew nothing about 3D tracking and now I do."

FISHER: "I hope I won't personally be doing hands-on compositing for the duration of my career, but I will continue to draw on the knowledge I've gained by doing it. It will be part of my toolset, like editing and motion graphics as I realize my goal of becoming a storyteller and filmmaker in my own right."

MOYER: "Yes, I'll be compositing. Every project has its challenges; each is unique in its own way. And that's what drives me and the people who work here."

Back to Top