Face recognition has been a long standing problem in computer vision. General

face recognition is challenging because of large appearance variability due to

factors including pose, ambient lighting, expression, size of the face, age, and distance

from the camera, etc. There are very accurate techniques to perform face

recognition in controlled environments, especially when large numbers of samples

are available for each face (individual). However, face identification under uncontrolled(

unconstrained) environments or with limited training data is still an unsolved

problem. There are two face recognition tasks: face identification (who is who in

a probe face set, given a gallery face set) and face verification (same or not, given

two faces). In this work, we study both face identification and verification in unconstrained

environments.

Firstly, we propose a face verification framework that combines Partial Least

Squares (PLS) and the One-Shot similarity model[1]. The idea is to describe a

face with a large feature set combining shape, texture and color information. PLS

regression is applied to perform multi-channel feature weighting on this large feature

set. Finally the PLS regression is used to compute the similarity score of an image

pair by One-Shot learning (using a fixed negative set).

Secondly, we study face identification with image sets, where the gallery and

probe are sets of face images of an individual. We model a face set by its covariance

matrix (COV) which is a natural 2nd-order statistic of a sample set.By exploring an

efficient metric for the SPD matrices, i.e., Log-Euclidean Distance (LED), we derive

a kernel function that explicitly maps the covariance matrix from the Riemannian

manifold to Euclidean space. Then, discriminative learning is performed on the

COV manifold: the learning aims to maximize the between-class COV distance and

minimize the within-class COV distance.

Sparse representation and dictionary learning have been widely used in face

recognition, especially when large numbers of samples are available for each face

(individual). Sparse coding is promising since it provides a more stable and discriminative

face representation. In the last part of our work, we explore sparse

coding and dictionary learning for face verification application. More specifically,

in one approach, we apply sparse representations to face verification in two ways

via a fix reference set as dictionary. In the other approach, we propose a dictionary

learning framework with explicit pairwise constraints, which unifies the discriminative

dictionary learning for pair matching (face verification) and classification (face

recognition) problems.