Multi-Object Tracking, Event Modeling, and Activity Discovery in Video Sequences

Abstract

One of the main goals of computer vision is video understanding, where objects in the video are detected, tracked, and their behavior is analyzed. In this dissertation, several key problems in video understanding are addressed, focusing on video surveillance applications. Moving target detection and tracking is one of the most fundamental tasks in visual surveillance. A new moving target detection method is proposed where the temporal variance is used as a measure for characterizing object motion. Our method is experimentally shown to produce high detection rates while keeping low false positive rates. In tracking multiple objects, it is essential to correctly associate targets and measurements. We describe an efficient multi-object tracking approach that maintains multiple hypotheses over time regarding the association of targets and measurements. The data association problem is solved by a combinatorial optimization technique which finds the most likely association allowing track initiation, termination, merge, and split. Experimental results show that our method tracks through varying degrees of interactions among the targets with high success rate. Recognizing complex high-level events requires an explicit model of the structure of the events. Our approach uses attribute grammar for representing such event, which formally specifies the syntax of the symbols and the conditions on the attributes. Events are recognized using an extension of the Earley parser that handles attributes and concurrent event threads. Various examples of recognizing specific events of interest and detecting abnormal events are demonstrated using real data. Unsupervised methods for learning human activities have been largely based on clustering trajectories from a given scene. However, conventional clustering algorithms are not suitable for scenes that have many outlier trajectories. We describe a method for finding only salient groups of trajectories, using the probability of trajectories accidentally forming a group as the measure of significance of the group. The grouping algorithm finds groups that maximizes significance, while automatically determining the threshold for significance. We validate our approach on real data and analyze its performance using simulated data.