Automatic landmarking of 2D images

Facial landmark detection, or localisation, is an essential preprocessing step in any automatic face analysis system. A 2D FLD algorithm is usually applied to the content of a face bounding box output by a face detector, and attempts to locate the positions of a set of pre-defined landmarks (key points) such as eyebrows, eye centres, nose tip or mouth corners of the facial parts in a 2D image.

We have made three contributions to the field:

• Cascaded collaborative regression trained on a hybrid dataset [1]: A large amount of training data is usually crucial for successful supervised learning. However, the task of providing training samples is often time-consuming, involving a considerable amount of tedious manual work. To augment our training set, we used a 3D morphable face model to generate synthesised faces in supervised cascaded regression training. These were used in conjunction with a smaller set of natural images. Initially the training uses mainly the synthetic data, as this can model the gross variations between the various poses. As the training proceeds, progressively more of the natural images are incorporated, as these can model finer detail. The results indicated the benefits of this approach over the state of the art.
• An adaptive random cascaded regression copse [2]: In light of the difficulties posed by scale variations of image photography in practical scenarios, we developed an adaptive scheme for scale-invariant shape-indexed feature extraction and shape update in cascaded-regression-based FLD. Furthermore, to enhance the generalization capacity of this framework, we present a random CR "copse", fusing information by a simple voting approach. Experimental results on several well-known face databases show the benefits of the proposed algorithm.
• Tensor-based AAM with missing values [3]: The use of tensor algebra requires a complete training dataset that consists of the training samples under all different variations. It is usually very difficult to collect such a dataset in practice, especially when we have many variation types. We developed two solutions to build a unified T-AAM from an incomplete training set with missing training examples using two state-of-the-art tensor augmentation algorithms, M²SA and CP-WOPT. The application of this method to landmark detection validated the approach.

1. Feng, Hu, Kittler, Christmas, Wu: Cascaded Collaborative Regression for Robust Facial Landmark Detection Trained Using a Mixture of Synthetic and Real Images With Dynamic Weighting. IEEE Transactions on Image Processing, Vol.24(11), 2015, pp:3425-3440.
em
2. Feng, Huber, Kittler, Christmas and Wu: Random Cascaded-Regression Copse for Robust Facial Landmark Detection. IEEE Signal Processing Letters, Vol.22(1), 2015, pp:76-80.
3. Feng, Kittler, Christmas, Wu and Pfeiffer: Automatic face annotation by multilinear AAM with Missing Values. ICPR, 2012, pp: 2586-2589