The Proposed Approach

We propose a hybrid system that combines the robust detection of feature points with a holistic and precise linear transform analysis of the face data. The detection of feature points uses a robust model capable of detecting individual features despite a wide range of translations, scale changes, 3D-pose changes and background clutter. This allows us to locate faces in an arbitrary, un-contrived image. Since we wish to utilize linear transform techniques, however, a consistent, normalized frontal mug-shot view of the face is needed. Thus, we propose synthesizing the required mug-shot view from the one detected in the original image. This is performed by inverting the 3D projection of the original face in the image and re-mapping it into frontal view via a deformable 3D model. Then, we perform illumination correction and segmentation to obtain an ideal mug-shot view of the individual in question. At this stage, we can safely apply holistic linear transform techniques (namely, the Karhunen-Loeve decomposition) and use the vector-representation of the face to recognize its identity.

Thus, our 3D normalization technique acts as a bridge between feature-detection and holistic face recognition. When guided with the results of the feature-detection, normalization removes the non-linear variations in the image and generates a face that is aligned and ready for recognition. Thus, the holistic face recognition stage is preceded by feature-based detection and normalization for increased robustness. In fact, the bridge is bidirectional since the feature-detection can also be complemented by the holistic face recognition stage. The Karhunen Loeve decomposition has demonstrated the ability provide a statistical measure of how face-like an image vector seems [44]. This measure can be fed back to the feature-detection algorithm to inform it if it has poorly localized the face to begin with. This will force the feature-detection to keep searching for the face and improve its detection results. Thus, holistic face recognition can also be used to assist the feature-detection stage and, ultimately, improve its precision.

Figure  depicts the interaction between the two stages: feature detection and holistic recognition. Although feature localization is robust, when used alone it is too insensitive for recognition. Although holistic face recognition is precise, its use of linear transformation is not robust to large non-linear face variations. Thus, combining the two approaches provides a superior overall system.

  

Figure 1.1: 3D Normalization as a Bridge Between Feature Detection and Face Recognition