Beyond Nouns and Verbs

dc.contributor.advisorDavis, Larryen_US
dc.contributor.authorGupta, Abhinaven_US
dc.contributor.departmentComputer Scienceen_US
dc.contributor.publisherDigital Repository at the University of Marylanden_US
dc.contributor.publisherUniversity of Maryland (College Park, Md.)en_US
dc.description.abstractDuring the past decade, computer vision research has focused on constructing image based "appearance" models of objects and action classes using large databases of examples (positive and negative) and machine learning to construct models. Visual inference however involves not only detecting and recognizing objects and actions but also extracting rich relationships between objects and actions to form storylines or plots. These relationships also improve recognition performance of appearance-based models. Instead of identifying individual objects and actions in isolation, such systems improve recognition rates by augmenting appearance based models with contextual models based on object-object, action-action and object-action relationships. In this thesis, we look at the problem of using contextual information for recognition from three different perspectives: (a) Representation of Contextual Models (b) Role of language in learning semantic/contextual models (c) Learning of contextual models from weakly labeled data. Our work departs from the traditional view of visual and contextual learning where individual detectors and relationships are learned separately. Our work focuses on simultaneous learning of visual appearance and contextual models from richly annotated, weakly labeled datasets. Specifically, we show how rich annotations can be utilized to constrain the learning of visually grounded models of nouns, prepositions and comparative adjectives from weakly labeled data. I will also show how visually grounded models of prepositions and comparative adjectives can be utilized as contextual models for scene analysis. We also present storyline models for interpretation of videos. Storyline models go beyond pair-wise contextual models and represent higher order constraints by allowing only specific possible action sequences (stories). Visual inference using storyline models involve inferring the ``plot'' of the video (sequence of actions) and recognizing individual activities in the plot.en_US
dc.format.extent14714769 bytes
dc.subject.pqcontrolledArtificial Intelligenceen_US
dc.subject.pqcontrolledComputer Scienceen_US
dc.subject.pquncontrolledComputer Visionen_US
dc.subject.pquncontrolledImage Annotationen_US
dc.subject.pquncontrolledProbabilisitc Graphical Modelsen_US
dc.subject.pquncontrolledStoryline Modelsen_US
dc.titleBeyond Nouns and Verbsen_US


Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
14.03 MB
Adobe Portable Document Format