Photometric normalisation

The performance of face verification algorithms can be dramatically affected by changes in the illumination conditions of image capture. In general the variation between images of different faces captured in the same conditions is smaller than that of the same face taken in a variety of environments.

The diagrams below (see Figure 1), show that a variation in illumination conditions can seriously alter the appearance of a face in the image plane to the extent where the images on the far right appear more similar to one another than to their respective frontally illuminated faces.

Figure 1 (Faces with varying illumination): Examples of the varying illumination conditions from the Yale B database.

Light can be reflected from a surface due to two effects; Lambertian reflectance and specular reflectance. Lambertian reflectance occurs when the light falling on a surface is reflected equally in all directions.

Specular reflectance occurs when the surface acts like a mirror and reflects the light from the surface in a single direction.

These types of reflection give rise to two effects of illumination. These are shading and specular reflections. An additional effect of illumination is that of shadows.

Shading occurs when light falls on a surface with changing gradient. In the image the brightness of the corresponding pixels will decrease as the surface normal diverges from the direction of the incident light (to reach a minimum when the angle between the incident light and the surface normal is ninety degrees). Specular reflections are regions of brightness on the surface. These occur when light is reflected from the surface to the camera such that the angle of incidence of light at the surface is equal to the angle of reflectance. Shadows occur when a light source is occluded. In an image, a shadow is a region of reduced brightness with a sharp edge.

The effect of illumination variation in image capture can not be easily separated from the important discriminative information between face images captured under identical lighting conditions. This is the task of illumination invariant face verification.

Pre-Processing

A number of image pre-processing techniques of various complexity have been proposed to remove the unwanted illumination effects, including histogram equalisation, homomorphic filtering, PCA reconstruction, isotropic and anisotropic smoothing.

The diagrams below shown in Figures 2, 3 and 4 present some of processed images under well, poor and very poor illumination. On the top of each figure we can see the original image, in the middle the image homomorphic filtered and in the bottom the image after applying an anisotropic smoothing method.

Figure 2: Examples of processed well illuminated images (left:original image, middle: homomorphic filtered, right: anisotropic smoothing method.

Figure 3: Examples of processed images under poor illumination (left:original image, middle: homomorphic filtered, right: anisotropic smoothing method.

Figure 4: Examples of processed images under very poor illumination (left:original image, middle: homomorphic filtered, right: anisotropic smoothing method.

Work carried out in CVSSP has shown that the anisotropic smoothing method developed by Gross and Brajovic is very effective and can be further improved by image dependent optimisation of the smoothing parameters. In Table 1 below we can see the performance results (in terms of half total error rates HTER=(FAR+FRR)/2) on all protocols of the BANCA database.

Table 1: Performance on all protocols of the BANCA database. Values shown are the HTER: half total error rate.

Component-Based Face Verification

Illumination varies by a large amount over a whole face image, but can be considered to have a smaller effect over small local areas. It is possible to carry out recognition separately on a number of facial components and fuse the results of these separate classifiers into one final decision.

A number of areas of investigation are raised by this approach, such as the selection of components, the choice of normalisation and the method of combining the individual classifiers.

The diagram below shown in Figures 5, presents the system diagram of a component-based face verification system.

Figure 5: Diagram of Component-Based Face Verification System.

Work carried out in CVSSP has shown that a component-based approach with simple photometric normalisation offers large improvements in face verification error rates. In Table 2 below we can see the performance results of both the complete and the component-based face verification system.

Table 2: Performance on all protocols of the BANCA database using histogram equalisation. Values shown are the HTER: Half Total Error Rate.

Publications

1. J. Short, J. Kittler, K. Messer, "A Comparison of Photometric Normalisation Algorithms for Face Verification", IEEE Int. Conf. Automatic Face and Gesture Recognition, pp. 254--259, (2004).

2. J. Short, J. Kittler, K. Messer, "Photometric Normalisation for Face Verification", Audio- and Video-Based Biometric Person Authentication, pp. 617--626, (2005.)

3. J. Short, J. Kittler, K. Messer, "Photometric Normalisation for Component-based Face Verification", IEEE Int. Conf. Automatic Face and Gesture Recognition, (2006).

For more information, please contact Prof. Josef Kittler or Mr James Short.