Inverse 3D Projection

It is essential to fit the 3D range data of the mean face so that it models a face found in a 2D image. Assume we have a 2D image with the coordinates of the eyes, the nose and the mouth perfectly pinpointed, as in Figure . Denote the 2D positions of the eyes, nose and mouth as $(\vec{i}_1,\vec{i}_2,\vec{i}_3,\vec{i}_4)$. The parameters of the 3D model $(t_{x},t_{y},t{z},\theta_{x},\theta_{y},\theta_{z},s_{y})$ must be tuned to align its 3D anchor points $(\vec{m}_1,\vec{m}_2,\vec{m}_3,\vec{m}_4)$ so that their 2D projections coincide with the set of 2D anchor points $(\vec{i}_1,\vec{i}_2,\vec{i}_3,\vec{i}_4)$. Note that the alignment to the destination points $(\vec{i}_1,\vec{i}_2,\vec{i}_3,\vec{i}_4)$ involves minimizing 8 distances since each of these points is a 2D position. We observe that Equation , which only has 7 degrees of freedom, is over-specified. Thus, there is usually no exact solution to the fitting problem, only approximations.

  

Figure 4.9: Image of U.S. President Ford with eyes, nose and mouth located