| Home | PhD applicants | Research | Publications | Google Scholar | Teaching | Bio |
Reliable localisation of landmarks is an essential precursor to many 3D mesh processing algorithms. For example, if the mesh is generated from a facial scan, keypoint extraction and landmarking is required for the automation of face recognition in non-cooperative subject scenarios. In our work, we have devised a new framework for learning how to extract keypoints from 3D meshes, using a learnt combination of local shape descriptors. Techniques used to learn the descriptor combinations are Linear Discriminant Analysis (LDA) and Adaboost.
In the figure below, the first image shows the 14 labelled points that we have labelled on a set of 200 training faces. For each of these points, we learn the combination of local descriptors that best distinguishes it from its non-neighbouring vertices. This allows us to generate scores which tell us, for some input vertex, how alike it is to the mean of any one of the 14 labelled points. A colour-map of such 'likeness' scores is shown in the 3rd image, where bright blue regions correspond to the local areas that we are looking for. If we extract the locally maximal values of this "keypoint score map", then we have a set of unlabelled keypoints, as shown in the fourth image of the figure below. (Note that additional points are generated that are locally like one of the 14 manually labelled points, but they are not the points that we wish to extract.)
Note that keypoints are unlabelled and hence we need to label them using learnt configural information, which captures the landmark's global spatial relationships. The basic process is shown below. We have evaluated both configural models and hypergraph matching approaches to labelling.
Final labellings for a set of input faces is shown below. The green points are the manual labels, serving as an approximate ground truth, and the blue points are extracted landmarks, each of which is labelled (eg. nose-tip). Our system performs well under pose variation and occlusion where many other systems fail, particularly those systems that find landmarks in specific sequences (eg. nose and inner eye corners first).
More details may be found on Clement Creusot's PhD web page
BACK to Nick Pears' Research Projects page.