Eye Replacement Test

Alot of pararell work going on at the moment!
Here is a short clip demonstrating the matched rotations of the eyes using trackers and 3d spheres in Nuke. (Track has not yet been completely stabilized + see some problems with the dot removal roto, ..will fix)


Since I want to get spherical distortions of 2d images I am using for the eye effects, I used Nuke’s tracking and 3d system. This is however all thanks to a lot of research and tutorials, since I did not have much former experience in Nuke’s 3d system. –>

In order to get a working 3d scene in Nuke, I began with putting in a Scene-, Cam-, Constant – and ScanlineReder-node. The Scanline taking care of the objects, and the Constant taking care of the format. I connected these to my footage, and then created two spheres, onto which I mapped the 2d images. A reformat node was also used to set the image size so that any images I choose to use wont behave strangely.

To make the sphere rotate and translate correctly with the eye, I first tracked 3 points of the eye, the left corner, the right and the pupil. Some frames had to be tracked manually, like the blink and the headturn, since the track points disappear. 3 points are needed, since the trackers are translated using calculation into rotation.

I used this expression, which is a calculation of rotation in Nuke-terms, using the 3 trackers;

degrees(atan( (parent.Tracker3.track1.y – (parent.Tracker3.track2.y + (parent.Tracker3.track3.y-parent.Tracker3.track2.y)/2 ))/((parent.Tracker3.track3.x-parent.Tracker3.track2.x)/2)))

Using the pupil tracker – the track translation was draged onto the spheres rotate y in the properties windows. The expression above was then added in to the rotate y, which makes the sphere rotate, but in the wrong direction –>
Nuke automatically puts the image on the backside of the sphere (away from the cam) as well as tries to expand the image onto the entire sphere – it becomes distorted. This was solved by changing the u + v extent parameters in the spheres properties, and in an added TransformGeo node, where you can simply transform/rotate/scale the geometry. The TransformGeo node is then connected to the Sphere to make its pivot correct.

To Stabilize the track I used the Curve Editor. The lft and rt trackers were filtered to a smoother movement. The rt trackers keys were also moved or flattened in places where the sphere jumps around a lot due to the tracking. The rt tracker was used because it was the easiest to track during the whole shot.

I then used a Bezier to roto the eye, and animated it using the most stable tracks translate x+y and center x+y. The image was also offset and I needed to insert an expression to get it into the right place, simply using the same numbers already in the translation, only changing the – and the + values.

The 2d image(sphere) was now rotating correctly with the eye. To solve the scaling that occurs throughout the shot I simply put in a transform node with an animated scale value. Added onto this is alot of color correcting and exposure. However, I am thinking to use this “pupil-effect” only in the last frames, as a “finale”. In the rest of the clip I am planning to use entirely black eyes, like the concept. But to do that I only have to remove the pupil effect (I already have a black image behind it) and put on some reflection.

Much work for a 3 second clip, but I want to learn how to use Nuke as much as possible…

Be Sociable, Share!

Leave a Reply