The goal of this research is to model an activity" performed by a group of moving and interacting objects (which can be people or cars or robots or different rigid components of the human body) and use these models for abnormal activity detection, tracking and segmentation. Previous approaches to modeling group activity include co-occurrence statistics (individual and joint histograms) and Dynamic Bayesian Networks, neither of which is applicable when the number of interacting objects is large. We treat the objects as point objects (referred to as landmarks'') and propose to model their changing configuration as a moving and deforming shape" using ideas from Kendall's shape theory for discrete landmarks. A continuous state HMM is defined for landmark shape dynamics in an activity". The configuration of landmarks at a given time forms the observation vector and the corresponding shape and scaled Euclidean motion parameters form the hidden state vector. The dynamical model for shape is a linear Gauss-Markov model on shape velocity". The shape velocity" at a point on the shape manifold is defined in the tangent space to the manifold at that point. Particle filters are used to track the HMM, i.e. estimate the hidden state given observations.

An abnormal activity is defined as a change in the shape activity model, which could be slow or drastic and whose parameters are unknown. Drastic changes can be easily detected using the increase in tracking error or the negative log of the likelihood of current observation given past (OL). But slow changes usually get missed. We have proposed a statistic for slow change detection called ELL (which is the Expectation of negative Log Likelihood of state given past observations) and shown  analytically and experimentally the complementary behavior of ELL and OL for slow and drastic changes. We have established the stability (monotonic decrease) of the errors in approximating the ELL for changed observations using a particle filter that is optimal for the unchanged system. Asymptotic stability is shown under stronger assumptions. Finally, it is shown that the upper bound on ELL error is an increasing function of the ``rate of change" with increasing derivatives of all orders, and its implications are discussed. 

Another contribution of the thesis is a linear subspace algorithm for pattern classification, which we call Principal Components' Null Space Analysis (PCNSA). PCNSA was motivated by Principal Components' Analysis (PCA) and it approximates the optimal Bayes classifier for Gaussian distributions with unequal covariance matrices. We have derived classification error probability expressions for PCNSA and compared its performance with that of subspace Linear Discriminant Analysis (LDA) both analytically and experimentally. Applications to abnormal activity detection, human action retrieval, 

object/face recognition are discussed.% with experimental results.