| Home | PhD applicants | Research | Publications | Google Scholar | Teaching | Bio |
Reliable localisation of facial landmarks is an essential precursor to many face processing algorithms, such as face recognition. In our work, we have devised both new descriptors and new techniques for 3D facial landmarking and 3D facial pose normalisation.
In the figure below left, a colour-mapped radial basis function (RBF) respresents a 'signed distance to surface' (blue is inside the head, red outside). The zero-isosurface representing the facial surface is coloured turquoise. The yellow points on a sphere represent a spherical sampling of the RBF function, where the sphere centre is at a point on the facial surface. A 2D histogram of RBF signed 'distance to surface' (DTS) values, for a range of sphere radii, forms a pose invariant surface descriptor that is robust to missing data, due to the natural interpolation properties of the RBF function. (Since it is defined everywhere in 3D, irrespective of missing data). We call this descriptor a spherically sampled RBF (SSR) histogram or, colloquially, a 'balloon image', as shown below right, since one can imagine a spherical balloon being inflated in size and periodically sampling the RBF function as it does so. We have employed this descriptor to reliably locate the facial nose-tip.
In the figure below, a set of space curves are shown, where a set of concentric spheres centered on the nose-tip intersects with the facial surface (the zero isosurface of the RBF function). We call these 'isoradius contours' as every point on a given contour is the same Euclidean distance from the nose tip. From any contour, we can extract a curvature signal, which represents how the tangential curvature changes as we travel around the nose-tip on a sphere. Irrespective of head pose, the curvature signals are very similar, modulo a phase change. Hence correlation of curvature signals across a pair of different poses can recover the relative pose of a subject's head. The curvature signals for a variety of subject head poses are shown in the figure below right.
In the figure below left, we have the axis of a set of concentric cylinders aligned with the nose tip and inner eye corner. At these sample points, we evaluate the RBF 'signed distance to surface' function and we can accumulate these values into a histogram, as shown below right. We call this a 'cylindrically sampled RBF' (CSR) descriptor. Such descriptors can be used as attributes for edges in graph matching approaches.
Suppose we are trying to verify the nose-tip and inner-eye corners, as a triplet of facial landmarks. In this case, we use a sampling pattern based on the barycenters of triangular subdivisions of the patch, as shown in the figure below left. The RBF function is sampled at a variety of heights about and below the triplet's plane. The RBF 'distance to surface' (DTS) values can be constructed into a feature vector, if the values are scanned out in some particular order, as shown by the numbering in the figure. The data can also be used in schemes in which the point triplet is unoriented and, in such cases, we can accumulate DTS values into a pose-invariant shape histogram, as shown in the figure, below right. These 'point triplet' ideas are presented in detail in Marcelo Romero's PhD thesis (published 2010).
BACK to Nick Pears' Research Projects page.