Dense Wide-Baseline Stereo with Varying Illumination and its Application to Face Recognition

Abstract

We study the problem of dense wide baseline stereo with varying illumination. We

are motivated by the problem of face recognition across pose. Stereo matching

allows us to compare face images based on physically valid, dense

correspondences. We show that the stereo matching cost provides a very robust

measure of the similarity of faces that is insensitive to pose variations. We

build on the observation that most illumination insensitive local comparisons

require the use of relatively large windows. The size of these windows is

affected by foreshortening. If we do not account for this effect, we incur

misalignments that are systematic and significant and are exacerbated by wide

baseline conditions.

We present a general formulation of dense wide baseline stereo with varying

illumination and provide two methods to solve them. The first method is based on

dynamic programming (DP) and fully accounts for the effect of slant. The second

method is based on graph cuts (GC) and fully accounts for the effect of both slant

and tilt. The GC method finds a global solution using the unary function from

the general formulation and a novel smoothness term that encodes surface

orientation.

Our experiments show that DP dense wide baseline stereo achieves superior

performance compared to existing methods in face recognition across pose. The

experiments with the GC method show that accounting for both slant and tilt can

improve performance in situations with wide baselines and lighting variation.

Our formulation can be applied to other more sophisticated window based image

comparison methods for stereo.