ADAPTIVE SAMPLING METHODS FOR TESTING AUTONOMOUS SYSTEMS

Simple item page
dc.contributor.advisorGupta, Satyandra Ken_US
dc.contributor.authorMullins, Galen Edwarden_US
dc.contributor.departmentMechanical Engineeringen_US
dc.contributor.publisherDigital Repository at the University of Marylanden_US
dc.contributor.publisherUniversity of Maryland (College Park, Md.)en_US
dc.date.accessioned2018-09-12T05:38:36Z
dc.date.available2018-09-12T05:38:36Z
dc.date.issued2018en_US
dc.description.abstractIn this dissertation, I propose a software-in-the-loop testing architecture that uses adaptive sampling to generate test suites for intelligent systems based upon identifying transitions in high-level mission criteria. Simulation-based testing depends on the ability to intelligently create test-cases that reveal the greatest information about the performance of the system in the fewest number of runs. To this end, I focus on the discovery and analysis of performance boundaries. Locations in the testing space where a small change in the test configuration leads to large changes in the vehicle's behavior. These boundaries can be used to characterize the regions of stable performance and identify the critical factors that affect autonomous decision making software. By creating meta-models which predict the locations of these boundaries we can efficiently query the system and find informative test scenarios. These algorithms form the backbone of the Range Adversarial Planning Tool (RAPT): a software system used at naval testing facilities to identify the environmental triggers that will cause faults in the safety behavior of unmanned underwater vehicles (UUVs). This system was used to develop UUV field tests which were validated on a hardware platform at the Keyport Naval Testing Facility. The development of test cases from simulation to deployment in the field required new analytical tools. Tools that were capable of handling uncertainty in the vehicle's performance, and the ability to handle large datasets with high-dimensional outputs. This approach has also been applied to the generation of self-righting plans for unmanned ground vehicles (UGVs) using topological transition graphs. In order to create these graphs, I had to develop a set of manifold sampling and clustering algorithms which could identify paths through stable regions of the configuration space. Finally, I introduce an imitation learning approach for generating surrogate models of the target system's control policy. These surrogate agents can be used in place of the true autonomy to enable faster than real-time simulations. These novel tools for experimental design and behavioral modeling provide a new way of analyzing the performance of robotic and intelligent systems, and provide a designer with actionable feedback.en_US
dc.identifierhttps://doi.org/10.13016/M2PZ51Q5N
dc.identifier.urihttp://hdl.handle.net/1903/21225
dc.language.isoenen_US
dc.subject.pqcontrolledRoboticsen_US
dc.subject.pquncontrolledAdaptive Samplingen_US
dc.subject.pquncontrolledAutonomous Vehiclesen_US
dc.subject.pquncontrolledMeta-modelingen_US
dc.subject.pquncontrolledScenario Generationen_US
dc.subject.pquncontrolledTesting and Evaluationen_US
dc.subject.pquncontrolledUnmanned Systemsen_US
dc.titleADAPTIVE SAMPLING METHODS FOR TESTING AUTONOMOUS SYSTEMSen_US
dc.typeDissertationen_US

Files

Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
Mullins_umd_0117E_19140.pdf
Size:
10.16 MB
Format:
Adobe Portable Document Format
Download

Collections

UMD Theses and Dissertations
Mechanical Engineering Theses and Dissertations