Selecting the Optimal P3P for Deformation

One more degree of freedom in our 7 parameter set, s_y has yet to be set. To determine the optimal value of this parameter, we shall now consider the position of the mouth on our model m₃ and on our image i₃. The other vertices (eyes and nose) have been exactly mapped to their destinations on the image via the WP3P computation. However, the same is not true for the mouth whose locus in the image has had no bearing so far on the calculation of our WP3P solution. In fact, we expect an error, E_mouth in the projected model mouth locus (mp₃) and the image's mouth locus (i₃). E_mouth is the distance between the model's mapped mouth and the image's mouth as shown in Equation :

$$E_{mouth}=\sqrt{(i_{3_x}-mp_{3_x})^{2}+(i_{3_y}-mp_{3_x})^{2}}$$ (4.19)

To minimize E_mouth, we solve the WP3P for several iterations of s_y. We try the following values of s_y in attempting to find the best fit to the face: $s_y = 0.5 + 0.05i , \: i \epsilon [0,14]$. The model and its four anchor points are scaled iteratively in the y dimension by each value of s_y, and the WP3P is solved. The WP3P solution which generates the smallest error, E_mouth, is then found corresponding to the best s_y parameter. We have thus solved the problem of fitting to the mouth point by allowing our model to stretch along y to fit to ``squashed'' and elongated faces. Incidentally, since only a simple vertical stretching is involved in this minimization, it is evident that only the perpendicular distance of the mouth from the eyes will affect the deformation and not the horizontal displacement of the mouth. No matter where the mouth is horizontally, the model fitting will remain the same. Figure  depicts a few sample stretches to find the best model to fit the image. The texture on the face is a mesh representing the frontal face of the 3D model. The mesh undergoes multiple stretches and is aligned to the eyes and nose each time with the WP3P computation. The stretched mesh whose mouth point best aligns with the mouth point in the image is then used as the 3D structure which will be coated with the face in the intensity image.

  

Figure 4.11: Stretching the model in search of the best mouth fit.

Computing the above total 4-point fitting yields the rotation, deformation and translation to apply to the 3D model to fit it to the image. Our implementation is also highly computationally efficient, requiring less than 1 millisecond on an SGI Indy workstation.