"History is a set of lies agreed upon, said Napoléon. So are movies." I often notice that my fellow filmmakers (including myself) seem to forget that movies are technically just elaborated lies. The elaboration of CGI animation is demanding enough by itself, and it has the additional challenge needing to be fast while creating content with a mix of 2D and 3D elements. This tutorial will show the basic required artistic elements to create immersive 3D scenes. The band Muse found me through some of my short film animation and asked me to create an animation for their The Handler music video. This tutorial and accompanied video is an overview and story of how the elements were created and animated to create the illusion.
iClone Muse circus tutorial:
Muse's The Handler
I will try to show that said question is happier without answer and that burden can be undauntingly playful.
As a base, I use Reallusion's software iClone; a very capable realtime animation making tool. Regardless of what we use - 2D or 3D elements - iClone (www.Reallusion.com) makes sure that the final product looks impressive and it allows us to visualize our ideas the easiest possible way.
As an example, here is one short clip from one of my movies. (Work in progress...)
Visibly, it's a 3D scene: we have the central machine mesh, a round object, cables are hanging
in the foreground and the character that moves is in between. That central machine has
some metal parts that seem perpendicular to the viewers and, as the camera moves, the
feeling of depth and perspective is undeniable. Well...yes, but it was basically a flat 2D
picture.
To get a 3D mesh, we need a height map or a displacement map. We can make it by using some photo editor tool, in my case it was Photoshop. A height map is supposed to be a greyscale representation of an original image, but not ordinary grayscale. Each shade of grey indicates different amount of depth. In this case, the things I want to bring closer to the viewers are brighter and vice versa.
I start with a simple move by turning the photo into a grayscale image, then I delete the parts
I don’t need. It's important to blur the image, because I don’t want to get sharp borderlines
between the “lower“ and “higher“ parts. After some paint work (using different shades of
white and black), this is how to achieve a quick and dirty, but functional enough height map
look.
Now we have the luxury of using the height map for a couple of different purposes. Deforming the flat image into a 3D vision is my most obvious goal now, but we shouldn't forget that this same image can be used as a specular or as an occlusion map if necessary.
To make a 3D mesh, we can engage many different kinds of 3D modelers. I stick to good old Photoshop's basic 3D tools. The simple and short command "New layer mesh from depth map to plane“ instantly gives us this result.
If we are smart enough to duplicate the layer of the original image at the beginning (which is
always a useful thing to do), we can simply put back that image on the newly-made model. Now this is a fully textured 3D machine mesh. Of course, we can shape the model further if we need to, by using some proper 3D modeler. (Photoshop's 3D capability is quite limited.)
Speaking of needs, it is very important to be aware of what is actually needed in a scene to be visible - what the whole scene should look like by the end. One of the common mistakes of animation movie makers is that they overdo the visuals. For instance, making a whole street for one short scene where only a house and a lamppost are supposed to be visible. All we need is what the camera needs. Not a single pixel or polygon more.
So, if we now import that model to 3DXchange and then to iClone, we have the basics for the scene. Of course, it's still just an image "glued" to a deformed plane, but the profile image shows enough three-dimensional data to fool the lights, the shadows and, eventually, the viewer.
Using Photoshop or any other similar tool, we can make a bump map or a normal map, but now I’m using the main texture of the model as a bump map, to achieve a rough, dirty and slightly “overbumped“ look.
As we see in the initial video, the camera has just a gentle pan move from one side of the machine model to another. Since the pan move is quite small, it doesn't reveal the edges of our deformed image (the model), but it's big enough to allow me to create the illusion of the depth between the camera and the machine, by placing different objects in the middle. These machine extensions basically help me selling the trick, so I use them shamelessly often.
So let's put some objects in the foreground, closer to the camera. With the presence of some hanging pipes, by changing the perspective and exploiting the parallax, the illusion of the scene is complete.
Allow me to insert some science at the end of this part.
Parallax is the way an object’s position or direction seems to change depending on viewing angle. As the eyes of humans and other animals are in different positions on the head, the present different views simultaneously. This is the basis of stereopsis, the process by which the brain exploits the parallax due to the different views from the eye to gain depth perception and estimate distances to objects.
Some animals also use motion parallax, in which the animals move their heads to gain different viewpoints. For example, pigeons whose eyes do not have overlapping fields of view and thus cannot use stereopsis bob their heads up and down to see depth. For the rest of us, non-pigeons, to percept depth it's enough to keep our heads steady and use iClone.
Here is another quick example clip, from the same movie I'm recently working on. It's again a factory scene based on the flat pic, made by using the aforementioned tricks:
Here are the ingredients:
Finally, let's use a similar technique, but on a slightly higher level. We, "Anymation" or "Machinima " authors and users of realtime engines as movie making environments, are usually obsessed with exporting characters and objects to some advanced game engines, often from iClone itself.
Why not? All the major game engines have excellent graphics these days and most of them are
more or less free. But, instead of "Why not?" we could also ask ourselves "Why yes?" Why
wouldn't we do things the other way around?
Here is an example as a starting point. It's one of my old videos, made in Unity3D. In this one,
both the character and the flying machine were animated in iClone:
It looks decent, especially that mountain with the waterfalls in the background. It also looks
like something that can't be easily done with iClone. That is a wrong idea. It can be done
rather simply.
Let's borrow that mountain with the waterfalls from Unity3D for iClone in a few easy steps.
1. As per usual, let's get it started with a still, flat picture of said mountains with waterfalls.
2. After that, we have to get a height map, which can be exported from Unity3D or we can
make one by ourselves.
3. A 3D model based on a height map (created in Photoshop), using the original captured
photo as the texture of the model.
4. Import the model to iClone
After these four quick steps, we have got a good looking mountain with the water, frozen in
time.
But what if we capture the video of the mountain with the flowing water in Unity3D, like this:
And then, perhaps we can use that video as a texture for the mountain?
The clever use of the nonexisting borders between 2D and 3D. Playing with the height or the displacement maps, especially with the video textures. There are endless possibilities. That kind of freedom represents – and not just in the technical sense - the very essence of my "Anymation" philosophy, which is based on the the idea of “everything goes if we know where it goes.“
For us animators, one of the most powerful tools is the ability of demystification. Let’s be aware that all those glorious realtime or non realtime working environments, all those eye candy software generators or even the most powerful tools on the market, finally produce just two dimensional still pictures. Plus some additional tricks that are generated fast and disguised well enough to convince the viewers about the motion or a third dimension. We also shouldn't forget that movie making is not much more than dealing with the economy of the viewers’ attention.
If we are aware of those things, we are able to shape the unshapable. In realtime, in no time, anytime.
Tom Jantol is a film director and animator.