Background of the project:
In this project, we are asked to morph a face to another face, and create an animation (gif) for that. We also need to compute the "mean" face of a population, and extrapolate from the "mean" face to create a caricature of myself.
Part 1. Defining Correspondences
In order for the morphing to work, we need consistent labeling point on both
faces. Which we need to define the correspondence manually on face,
ear, nose, etc...
In this case, I used this labeling tool that is
provided in the assignment specs for labeling the correspondence.
Original Image:
After we get the lableing point, we can plug it into the Delaunay method that will create the triangulation that we need for morphing. Note that the green dots you see below is the labeling that we chose using the tool.
Image with Triangulation:
Part 2. Computing the "Mid-way Face":
Before we get into the morphing sequence for animation, we need to first get the "mid-way" face. Here is the process:
- computing the average shape of the 2 faces
- warping both faces into that average shape
- averaging the colors together (Cross-Dissolving)
For computing in the average shape, we just need to add all the labeling points in 2
faces together and divided by 2 (or times 0.5 to each of them, which will
make more sense in the next part when we do sequence). Same thing with the
average color, just times 0.5 to each of the faces and add them together.
The main issue here is the warping, what we are doing here is that,
we need to compute the Affine transformation for all the triangles
in the triangulation to the avaerage shape.
And for the pixels, we need to extrapolate the original pixels into the
pixels that we need in the mean face. In this case, in order to get a better
results and prevent holes, we will use inverse-warping.
Original Images with Mid Face:
Part 3. The Morph Sequence
After we get the mean face, what we need to do now is implement the
idea and make a gif or video using the "morphing sequence". Which we
want to make an animation that a face gradually changing to another face.
The morphing sequence is bascially just a bunch of morphing faces
stacked together. Same logic as the mean face, but this time, instead
of the fixed 0.5 that we times each image, we have one of the image started as 1, and the other
started at 0. Depends on how many morphing faces you want for creating the
animation, you have an alpha = 1 / num_of_face. And you basically use a for loop
to compute the shape and the color for each step using a formula like the
following:
average_shape = img1Points * (1 - t) + img2Points * t (same logic for the color)
My gif from Ado to Suisei:
Part 4. The "Mean face" of a population:
In this part, we need to compute the mean face for a population of people. And warp our own face into its geimetry and also wrap the mean face into mt face geometry. Over here, I use the first 100 faces in the FEI database. Note that there are some picture that have labeling points outside of the image where I just get rid of them as the outliers.
Some Examples of warped no expression face:
Some Examples of warped smile expression face:
Average no expression face and smiling face:
My face warped into average face geomerty and average face warped in my face geomerty (no expression):
Part 5. Caricatures: Extrapolating from the mean:
In this part, we are asked to create a caricature of our own face from the population mean face that we computed in the last part. Since we are suggested that this may work better with some characteristics specific mean, so I decided to used the mean smiling face that we got in the last part.
My face extrapolated into the smile face with alpha = 2:
As you can see, my face is now smiling, although it is not that obvious because my mouth is not opened.
Bells and Whistle - Change Gender:
In this bells and whistle, I tried to use the average Chinese women face that I found online to change the gender of my face. We are also asked to show the example where we only morph the shape or appearance.
My original face and average Chinese woman face:
Morphing for shape only and appearance only:
Final result (morph both shape and appearance):
Sorry for hurting your eyes, I apologize to who ever is reading this. But I think it is quite successful that my face now look more like a woman than a man.