3D face modelling using a 3D morphable model

3D Morphable Models are used for face analysis because the intrinsic properties of 3D faces provide a representation that is immune to intra-personal variations such as pose and illumination. Given a single facial input image, a 3DMM can recover 3D face (shape and texture) and scene properties (pose and illumination) via a fitting process.

The Surrey 3D face models

Each of our face models is created from a set of 3D face scans. Each scan is in the form of a graph, where the vertices are locations on the surface of the face, and the edges connect the vertices to form a triangulated mesh. Each vertex also has a colour; hence the vertices define both the shape and the texture of a face. Each face is registered to a standard mesh, so that each vertex has the same location on any registered face.

The model has two components: (i) a mesh consisting of the mean face, and (ii) two matrices, one each for shape and texture that describe the various modes of variations from the mean. The number of modes of variation depends on the size of the mesh, and also is different for shape and texture. Hence the appearance of a given face can be summarised by a set of coefficients that describe how much there is of each mode of variation.

If you would like to download and use any of the University of Surrey 3D face models, details of their availability are here.

Development of 3DMMs for face modelling at the University of Surrey

The work of our group on 3D Morphable Models followed on from early work of Blanz and Vetter, and also from the PhD thesis of S. Romdhani. The development has taken place in several phases:

1. The PhD work of Rafael Tena fused that of the above researchers, resulting in the creation of a morphable model built from a collection of 169 high-quality 3D face images. The main software components of this work are:
   • registering the 3D images onto a common mesh
   • creating the 3DMM from the registered images
   • creation of a specific 3D model of an individual by fitting a 2D image to the 3DMM
2. Pouria Mortazavian extended this method to work with low-resolution images:
   • Mortazavian, Kittler and Christmas: 3D Morphable Model Fitting For Low-Resolution Facial Images, ICB 2012
   • Mortazavian: Face Recognition in Low Resolution Using a 3D Morphable Model, PhD thesis 2013
3. The main problem with the above fitting methods is that they use gradient descent optimisation, and hence are slow. Guosheng Hu built on this work to provide a faster method using closed-form optimisation solutions:
   • Hu: Face Analysis using 3D Morphable Models, PhD thesis 2015
   • Hu, Chan, Yan, Christmas and Kittler: Robust face recognition by an albedo based 3D morphable model. IJCB 2014.
   • Hu, Mortazavian, Kittler and Christmas: A facial symmetry prior for improved illumination fitting of 3D morphable model. ICB 2013
   • Hu, Chan, Kittler and Christmas: Resolution-aware 3D morphable model. BMVC 2012.
4. The above work concentrated on fitting a 3DMM to a single image. Patrik Huber has extended this work to fit a 3DMM to frames from a video. Suitable frames are automatically extracted from the video, and the model fittings of these fames is combined to provide stronger evidence of identity.
   • P. Huber, G. Hu, R. Tena, P. Mortazavian, W. Koppen, W. Christmas, M. Rätsch and J. Kittler: A Multiresolution 3D Morphable Face Model and Fitting Framework, VISAPP 2016 [PDF] [GitHub]
   • P. Huber, Z. Feng, W. Christmas, J. Kittler, M. Rätsch: Fitting 3D Morphable Models using Local Features, ICIP 2015 [PDF]