5.3. Skin locus from basis functions

At least in theory, when all spectral information about imaging is available, then the skin locus can be calculated off-line using Equation 5. This allows us the possibility of very easy and fast simulation of different illumination and white balancing conditions without taking any images. Again, results are sensor-dependent due to the use of the camera’s spectral sensitivities. To obtain a more device-independent approach, one can utilize the skin colour signals (Paper V). Generally, the colour signal CS is described as the light which is reflected from a point with a certain reflectance. It can be expressed as a product of the illuminant SPD effecting over the point and the reflectance at that point:

Equation 18.

in which I = SPD of the effective illuminant over the point,

R = reflectance of the object at the point, and

λ = wavelength.

Basis functions for skin colour signals are not always available, but they are very simple and quick to calculate by applying principal component analysis (PCA) when illumination and reflectance data are available (i.e. from Physics-based Face Database). In many earlier studies, it has been shown that the obtained basis functions for spectral reflectances of different objects are useful for colour constancy (Maloney & Wandell 1986), colour correction (Lenz et al. 1999), segmentation (Hauta-Kasari et al. 2000) and colour image synthesis and analysis (Wandell 1987), but here they are used exclusively for skin appearance modelling. Furthermore, according to the author’s best knowledge the PCA has usually been applied separately to illumination SPDs and an object’s reflectance functions for data compression and reducing complexity (Maloney 1986, Maloney & Wandell 1986, Parkkinen et al. 1989). The inspiration for employing basis functions is that they offer camera-independent information in compressed form about skin colour signals and make it possible to simulate outputs of different cameras easily.

The procedure for obtaining basis functions starts with the choice of illumination to be used. In these experiments, two illumination groups were used, artificial sources and Planckian illuminants. The first group consists of four light sources obtained from the Physics-based Face Database, namely the sources A, Horizon, TL84 and D65. The illuminants used in the Planckian group were calculated using black body radiator formulae (Wyszecki & Stiles 2000) with the colour temperature parameter changing from 2300 K to 6500 K in steps of 100 K. For both of these illumination groups, the results obtained from two commonly used normalization methods, the Euclidean rule and a scaling normalization (Eqs. 1-2 in Section 3.2), are compared and evaluated. When the normalization of illumination is done, the ensemble of skin colour signals is calculated using the illuminant groups and three spectral reflectances of eight persons from each skin type (totalling 72 reflectances). PCA was applied to the zero centered ensemble. The resulting basis functions are displayed in Figure 20. The basis functions for the Planckian colour signal set are smoother than for the artificial set, but this is to be expected due to their different forms of SPDs. In addition, the illuminant normalization method used effects the shape of the basis functions. Another interesting observation is shown in Figure 21.

The coefficients of the three first functions are shown in Figure 21 for the Planckian set. The coefficients form a slope in quadratic shape because the SPDs are smooth and similar in shape. This is not true for real illuminants and their slope form is different. The coefficients are more sparse with lower colour temperatures because Planckian SPDs do not change uniformly with colour temperature.

The reconstruction quality of the basis function obtained is tested both for new reflectances and for new illuminants. The function for approximation error E per wavelength was selected so that the error increases rapidly when the difference between the original and reconstructed signal increases. The selected function is

Equation 19.

where CS = the original skin colour signal, and

CS_REC = the reconstructed signal.

The total mean and standard errors are calculated as the mean approximation error over the wavelength and the number of samples. The reconstruction errors for skin colour signals calculated using 303 reflectances not used in basis function computations are shown in Tables 9 and 10, as well. Due to the division of data between test and training sets, the test set produces a smaller reconstruction error than the training set which was used in the computation of the basis functions. This is caused by reflectances included in the test set: it contains only Caucasian and Asian reflectances, which are more similar to each other than the Negroid reflectances. By comparing these two tables, it can be observed that the Euclidean normalization yields poorer performance than the scaling normalization and it is therefore excluded from the further research.

Because it is common for real scenes to have more than one illuminant shining on a point of the object, the basis functions are tested for skin signals under mixed illumination conditions. It is easy to show that this can be modelled by using the basis functions calculated earlier only for one illuminant cases. First, let’s assume that we have N illuminants giving their contribution to a point with reflectance R. The new relative, effective illuminant I_new can be expressed as a weighted sum of different light sources as shown in Section 3.2, Eq. 3. The colour signal with this new illuminant CS_new,

Equation 20.

is a weighted sum of colour signals of each individual illuminant. The new colour signal can be reconstructed using basis functions ε and mean µ calculated for the colour signals of each individual illuminant:

Equation 21.

where the combination C is a combination of coefficients and weights,

Equation 22.

and coeff = the coefficients of basis functions for an individual illuminant.

The reconstruction quality with colour signals using a new illuminant is displayed in Table 11. It shows the reconstruction error with 5 basis functions. The new illuminants are calculated from two, equally weighted “old” illumination SPDs and they can be used to model a mixture of light sources. The purpose is now to simulate cases where two light sources shine on a point in the face equally. The reconstruction error is small, which means that the basis functions can be used to represent new skin colour signals. In fact, the error is smaller than for the average one illuminant case (Table 9) because the illuminant combination smooths out big differences. Moreover, the different degrees of mixtures can be simply simulated by scaling coefficients of the illuminants and summing them as shown in Eq 3. Figure 21 displays coefficients for a mixture of two and Figure 22 for three illuminants. The coefficients calculated for mixture colour signals form straight lines.

Next, the basis functions are used to calculate the skin locus: first, the illuminants are chosen for the locus modelling and the number of the basis functions are selected to achieve the needed reconstruction accuracy. Using these two factors, the skin colour signal set is reconstructed. For each output channel, the set is weighted by the camera’s channel sensitivity and the sum of the weighted data is calculated. Then the sum is normalized using Eq. 6 with the desired white balancing illuminants. The simulated skin RGB values can be converted to some other colour space like normalized colour coordinates which are then used for further processing. The obtained loci can be seen in Figure 23. When only three basis functions are used, the obtained locus is somehow smeared as shown in Figure 23b and resembles the loci obtained with a Winnov camera (see Figure 24). The results with five basis functions, Figure 23c are comparable to those obtained using all data in Figure 23a.