One more degree of freedom in our 7 parameter set, sy has yet to be
set. To determine the optimal value of this parameter, we shall now consider
the position of the mouth on our model m3 and on our image i3. The other
vertices (eyes and nose) have been exactly mapped to their destinations on the
image via the WP3P computation. However, the same is not true for the mouth
whose locus in the image has had no bearing so far on the calculation of our
WP3P solution. In fact, we expect an error, Emouth in the projected model
mouth locus (mp3) and the image's mouth locus (i3). Emouth is the
distance between the model's mapped mouth and the image's mouth as shown in
Equation :
To minimize Emouth, we solve the WP3P for several iterations of sy.
We try the following values of sy in attempting to find the best fit to
the face:
.
The model and its four
anchor points are scaled iteratively in the y dimension by each value of
sy, and the WP3P is solved. The WP3P solution which generates the smallest
error, Emouth, is then found corresponding to the best sy parameter.
We have thus solved the problem of fitting to the mouth point by allowing our
model to stretch along y to fit to ``squashed'' and elongated faces.
Incidentally, since only a simple vertical stretching is involved in this
minimization, it is evident that only the perpendicular distance of the mouth
from the eyes will affect the deformation and not the horizontal displacement
of the mouth. No matter where the mouth is horizontally, the model fitting
will remain the same. Figure
depicts a few sample
stretches to find the best model to fit the image. The texture on the face is
a mesh representing the frontal face of the 3D model. The mesh undergoes
multiple stretches and is aligned to the eyes and nose each time with the WP3P
computation. The stretched mesh whose mouth point best aligns with the mouth point
in the image is then used as the 3D structure which will be coated with the
face in the intensity image.
Computing the above total 4-point fitting yields the rotation, deformation and translation to apply to the 3D model to fit it to the image. Our implementation is also highly computationally efficient, requiring less than 1 millisecond on an SGI Indy workstation.