Reasoning about Geometric Object Interactions in 3D for Manipulation Action Understanding

Simple item page
dc.contributor.advisorAloimonos, Yiannisen_US
dc.contributor.authorZampogiannis, Konstantinosen_US
dc.contributor.departmentComputer Scienceen_US
dc.contributor.publisherDigital Repository at the University of Marylanden_US
dc.contributor.publisherUniversity of Maryland (College Park, Md.)en_US
dc.date.accessioned2019-09-27T05:38:25Z
dc.date.available2019-09-27T05:38:25Z
dc.date.issued2019en_US
dc.description.abstractIn order to efficiently interact with human users, intelligent agents and autonomous systems need the ability of interpreting human actions. We focus our attention on manipulation actions, wherein an agent typically grasps an object and moves it, possibly altering its physical state. Agent-object and object-object interactions during a manipulation are a defining part of the performed action itself. In this thesis, we focus on extracting semantic cues, derived from geometric object interactions in 3D space during a manipulation, that are useful for action understanding at the cognitive level. First, we introduce a simple grounding model for the most common pairwise spatial relations between objects and investigate the descriptive power of their temporal evolution for action characterization. We propose a compact, abstract action descriptor that encodes the geometric object interactions during action execution, as captured by the spatial relation dynamics. Our experiments on a diverse dataset confirm both the validity and effectiveness of our spatial relation models and the discriminative power of our representation with respect to the underlying action semantics. Second, we model and detect lower level interactions, namely object contacts and separations, viewing them as topological scene changes within a dense motion estimation setting. In addition to improving motion estimation accuracy in the challenging case of motion boundaries induced by these events, our approach shows promising performance in the explicit detection and classification of the latter. Building upon dense motion estimation and using detected contact events as an attention mechanism, we propose a bottom-up pipeline for the guided segmentation and rigid motion extraction of manipulated objects. Finally, in addition to our methodological contributions, we introduce a new open-source software library for point cloud data processing, developed for the needs of this thesis, which aims at providing an easy to use, flexible, and efficient framework for the rapid development of performant software for a range of 3D perception tasks.en_US
dc.identifierhttps://doi.org/10.13016/62hp-hjsi
dc.identifier.urihttp://hdl.handle.net/1903/25024
dc.language.isoenen_US
dc.subject.pqcontrolledArtificial intelligenceen_US
dc.subject.pqcontrolledRoboticsen_US
dc.subject.pqcontrolledComputer scienceen_US
dc.subject.pquncontrolledcomputer visionen_US
dc.subject.pquncontrolledgeometric registrationen_US
dc.subject.pquncontrolledinteractionen_US
dc.subject.pquncontrolledpoint clouden_US
dc.subject.pquncontrolledspatial relationsen_US
dc.subject.pquncontrolledtopologyen_US
dc.titleReasoning about Geometric Object Interactions in 3D for Manipulation Action Understandingen_US
dc.typeDissertationen_US

Files

Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
Zampogiannis_umd_0117E_20211.pdf
Size:
26.3 MB
Format:
Adobe Portable Document Format
Download

Collections

UMD Theses and Dissertations
Computer Science Theses and Dissertations