Theses and Dissertations from UMD
Permanent URI for this communityhttp://hdl.handle.net/1903/2
New submissions to the thesis/dissertation collections are added automatically as they are received from the Graduate School. Currently, the Graduate School deposits all theses and dissertations from a given semester after the official graduation date. This means that there may be up to a 4 month delay in the appearance of a give thesis/dissertation in DRUM
More information is available at Theses and Dissertations at University of Maryland Libraries.
Browse
153 results
Search Results
Item Deployment of Large Vision and Language Models for Real-Time Robotic Triage in a Mass Casualty Incident(2024) Mangel, Alexandra Paige; Paley, Derek; Aerospace Engineering; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)In the event of a mass casualty incident, such as a natural disaster or war zone, having a system of triage in place that is efficient and accurate is critical for life-saving intervention, but medical personnel and resources are often strained and struggle to provide immediate care to those in need. This thesis proposes a system of autonomous air and ground vehicles equipped with stand-off sensing equipment designed to detect and localize casualties and assess them for critical injury patterns. The goal is to assist emergency medical technicians in identifying those in need of primary care by using generative AI models to analyze casualty images and communicate with the victims. Large language models are explored for the purpose of developing a chatbot that can ask a casualty where they are experiencing pain and make an informed assessment about injury classifications, and a vision language model is prompt engineered to assess a casualty image to produce a report on designated injury classifiers.Item TOWARDS EFFICIENT OCEANIC ROBOT LEARNING WITH SIMULATION(2024) LIN, Xiaomin; Aloimonos, Yiannis; Electrical Engineering; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)In this dissertation, I explore the intersection of machine learning, perception, and simulation-based techniques to enhance the efficiency of underwater robotics, with a focus on oceanic tasks. My research begins with marine object detection using aerial imagery. From there, I address oyster detection using Oysternet, which leverages simulated data and Generative Adversarial Networks for sim-to-real transfer, significantly improving detection accuracy. Next, I present an oyster detection system that integrates diffusion-enhanced synthetic data with the Aqua2 biomimetic hexapedal robot, enabling real-time, on-edge detection in underwater environments. With detection models deployed locally, this system facilitates autonomous exploration. To enhance this capability, I introduce an underwater navigation framework that employs imitation learning, enabling the robot to efficiently navigate over objects of interest, such as rock and oyster reefs, without relying on localization. This approach improves information gathering while ensuring obstacle avoidance. Given that oyster habitats are often in shallow waters, I incorporate a deep learning model for real/virtual image segmentation, allowing the robot to differentiate between actual objects and water surface reflections, ensuring safe navigation. I expand on broader applications of these techniques, including olive detection for yield estimation and industrial object counting for warehouse management, using simulated imagery. In the final chapters, I address unresolved challenges, such as RGB/sonar data integration, and propose directions for future research to enhance underwater robotic learning through digital simulation further. Through these studies, I demonstrate how machine learning models and digital simulations can be used synergistically to address key challenges in underwater robotic tasks. Ultimately, this work advances the capabilities of autonomous systems to monitor and preserve marine ecosystems through efficient and robust digital simulation-based learning.Item Applied Aerial Autonomy for Reliable Indoor Flight and 3D Mapping(2024) Shastry, Animesh Kumar; Paley, Derek; Aerospace Engineering; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)Uncrewed Aerial Systems (UAS) are essential for safely exploring indoor environments damaged by shelling, fire, floods, and structural collapse. These systems can gather critical visual and locational data, aiding in hazard assessment and rescue planning without risking human lives. Reliable UAS deployments requires advanced sensors and robust algorithms for real-time data processing and safe navigation, even in GPS-denied and windy conditions. This dissertation details three research projects to improve UAS performance: (1) in-flight calibration to improve estimation and control, (2) system identification for wind rejection, and (3) indoor aerial 3D mapping. The dissertation begins with introducing a comprehensive nonlinear filtering framework for UAV parameter estimation, which considers factors such as external wind, drag coefficients, IMU bias, and center of pressure. Additionally, it establishes optimized flight trajectories for parameter estimation through empirical observability. Moreover, an estimation and control framework is implemented, utilizing the mean of state and parameter estimates to generate suitable control inputs for vehicle actuators. By employing a square-root unscented Kalman filter (sq-UKF), this framework can derive a 23-dimensional state vector from 9-dimensional sensor data and 4-dimensional control inputs. Numerical results demonstrate enhanced tracking performance through the integration of the estimation framework with a conventional model-based controller. The estimation of unsteady winds results in improved gust rejection capabilities of the onboard controller as well. Closely related to parameter estimation is system identification. Combining with the previous work a comprehensive system identification framework with both linear offline and nonlinear online methods is introduced. Inertial parameters are estimated using frequency-domain linear system identification, incorporating control data from motor-speed sensing and state estimates from automated frequency sweep maneuvers. Additionally, drag-force coefficients and external wind are recursively estimated during flight using a sq-UKF. A custom flight controller is developed to manage the computational demands of online estimation and control. Flight experiments demonstrate the tracking performance of the nonlinear controller and its improved capability in rejecting gust disturbances. Aside from wind rejection, aerial indoor 3D mapping is also required for indoor navigation, and therefore, the dissertation introduces a comprehensive pipeline for real-time mapping and target detection in indoor environments with limited network access. Seeking a best-in-class UAS design, it provides detailed analysis and evaluation of both hardware and software components. Experimental testing across various indoor settings demonstrates the system's efficacy in producing high-quality maps and detecting targets.Item Multi-Domain Human-Robot Interfaces(2024) Abdi, Sydrak Solomon; Paley, Derek; Aerospace Engineering; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)As autonomous robots become more capable and integrated into daily society, it becomes crucial to consider how a user will interact with them, how a robot will perceive a user, and how a robot will comprehend a user’s intentions. This challenge increases in difficulty when the user is required to interact with and control multiple robots simultaneously. Human intervention is often required during autonomous operations, particularly in scenarios that involve complex decision-making or where safety concerns arise. Thus, the methods by which users interact with multi-agent systems is an important area of research. These interactions should be intuitive, efficient, and effective all while preserving the operator's safety. We present a novel human swarm interface (HSI) that utilizes gesture control and haptic feedback to interact with and control a swarm of quadrotors in a confined space. This human swarm interface prioritizes operator safety while reducing cognitive load during control of an aerial swarm. Human-robot interfaces (HRIs) are mechanisms designed to facilitate communication between humans and robots, enhancing the user's ability to command and collaborate with robots in an intuitive and user-friendly manner. One challenge is providing mobile robotic systems with the capability to localize and interact with a user in their environment. Localization involves estimating the pose (position and orientation) of the user relative to the robot, which is essential for tasks that require close interactions or navigation in shared spaces. We present a novel method for obtaining user pose as well as other anthropometric measurements useful for human-robot interactions. Another challenge is extending these HRI and HSI paradigms to the outdoors. Unlike controlled laboratory conditions, outdoor environments involve a variety of variables such as fluctuating weather conditions as well as a mix of static and dynamic obstacles. In this dissertation, we design a portable human swarm interface that allows an operator to interact with and control a multi-agent system outdoors. The portable HSI takes the form of smart binoculars. The user uses the smart binoculars to select an outdoor location and assign a task for the multi-agent system to complete given the targeted area. This system allows for new methods of multi-agent operation, that will leverage a user's on-the-ground knowledge while utilizing autonomous vehicles for line-of-sight operations, without compromising their situational awareness.Item INGESTIBLE BIOIMPEDANCE SENSING DEVICE FOR GASTROINTESTINAL TRACT MONITORING(2024) Holt, Brian Michael; Ghodssi, Reza; Electrical Engineering; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)Gastrointestinal (GI) diseases, such as inflammatory bowel disease (IBD), result in dilated adherens and tight junctions, altering mucosal tissue permeability. Few monitoring techniques have been developed for in situ monitoring of local mucosal barrier integrity, and none are capable of non-invasive measurement beyond the esophagus. In this work, this technology gap is addressed through the development of a noise-resilient, flexible bioimpedance sensor integrated ingestible device containing electronics for low-power, four-wire impedance measurement and Bluetooth-enabled wireless communication. Through electrochemical deposition of a conductive polymeric film, the sensor charge transfer capacity is increased 51.4-fold, enabling low-noise characterization of excised intestinal tissues with integrated potentiostat circuitry for the first time. A rodent animal trial is performed, demonstrating successful differentiation of healthy and permeable mice colonic tissues using the developed device. In accordance with established mucosal barrier evaluation methodologies, mucosal impedance was reduced between 20.3 ± 9.0% and 53.6 ± 10.7% of its baseline value in response to incrementally induced tight junction dilation. Ultimately, this work addresses the fundamental challenges of electrical resistance techniques hindering localized, non-invasive IBD diagnostics. Through the development of a simple and reliable bioimpedance sensing module, the device marks significant progress towards explicit quantification of “leaky gut” patterns in the GI tract.Item Machine Learning with Differentiable Physics Priors(2024) Qiao, Yiling; Lin, Ming ML; Computer Science; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)Differentiable physics priors enable gradient-based learning systems to adhere to physical dynamics. By making physics simulations differentiable, we can backpropagate through the physical consequences of actions. This pipeline allows agents to quickly learn to achieve desired effects in the physical world and is an effective technique for solving inverse problems in physical or dynamical systems. This new programming paradigm bridges model-based and data-driven methods, mitigating data scarcity and model bias simultaneously. My research focuses on developing scalable, powerful, and efficient differentiable physics simulators. We have created state-of-the-art differentiable physics for rigid bodies, cloth, fluids, articulated bodies, and deformable solids, achieving performance orders of magnitude better than existing alternatives. These differentiable simulators are applied to solve inverse problems, train control policies, and enhance reinforcement learning algorithms.Item Enhanced Robot Planning and Perception Through Environment Prediction(2024) Sharma, Vishnu Dutt; Tokekar, Pratap; Computer Science; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)Mobile robots rely on maps to navigate through an environment. In the absence of any map, the robots must build the map online from partial observations as they move in the environment. Traditional methods build a map using only direct observations. In contrast, humans identify patterns in the observed environment and make informed guesses about what to expect ahead. Modeling these patterns explicitly is difficult due to the complexity in the environments. However, these complex models can be approximated well using learning-based methods in conjunction with large training data. By extracting patterns, robots can use not only direct observations but also predictions of what lies ahead to better navigate through an unknown environment. In this dissertation, we present several learning-based methods to equip mobile robots with prediction capabilities for efficient and safer operation. In the first part of the dissertation, we learn to predict using geometrical and structural patterns in the environment. Partially observed maps provide invaluable cues for accurately predicting the unobserved areas. We first demonstrate the capability of general learning-based approaches to model these patterns for a variety of overhead map modalities. Then we employ task-specific learning for faster navigation in indoor environments by predicting 2D occupancy in the nearby regions. This idea is further extended to 3D point cloud representation for object reconstruction. Predicting the shape of the full object from only partial views, our approach paves the way for efficient next-best-view planning, which is a crucial requirement for energy-constrained aerial robots. Deploying a team of robots can also accelerate mapping. Our algorithms benefit from this setup as more observation results in more accurate predictions and further improves the task efficiency in the aforementioned tasks. In the second part of the dissertation, we learn to predict using spatiotemporal patterns in the environment. We focus on dynamic tasks such as target tracking and coverage where we seek decentralized coordination between robots. We first show how graph neural networks can be used for more scalable and faster inference while achieving comparable coverage performance as classical approaches. We find that differentiable design is instrumental here for end-to-end task-oriented learning. Building on this, we present a differentiable decision-making framework that consists of a differentiable decentralized planner and a differentiable perception module for dynamic tracking. In the third part of the dissertation, we show how to harness semantic patterns in the environment. Adding semantic context to the observations can help the robots decipher the relations between objects and infer what may happen next based on the activity around them. We present a pipeline using vision-language models to capture a wider scene using an overhead camera to provide assistance to humans and robots in the scene. We use this setup to implement an assistive robot to help humans with daily tasks, and then present a semantic communication-based collaborative setup of overhead-ground agents, highlighting the embodiment-specific challenges they may encounter and how they can be overcome. The first three parts employ learning-based methods for predicting the environment. However, if the predictions are incorrect, this could pose a risk to the robot and its surroundings. The third part of the dissertation presents risk management methods with meta-reasoning over the predictions. We study two such methods: one extracting uncertainty from the prediction model for risk-aware planning, and another using a heuristic to adaptively switch between classical and prediction-based planning, resulting in safe and efficient robot navigation.Item Planning and Perception for Unmanned Aerial Vehicles in Object and Environmental Monitoring(2024) Dhami, Harnaik; Tokekar, Pratap; Computer Science; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)Unmanned Aerial vehicles (UAVs) equipped with high-resolution sensors are enabling data collection from previously inaccessible locations on a remarkable spatio-temporal scale. These systems hold immense promise for revolutionizing various fields such as precision agriculture and infrastructure inspection where access to data is important. To fully exploit their potential, the development of autonomy algorithms geared toward planning and perception is critical. In this dissertation, we develop planning and perception algorithms, specifically when UAVs are used for data collection in monitoring applications. In the first part of this dissertation, we study problems of object monitoring and the planning challenges that arise with them. Object monitoring refers to the continuous observation, tracking, and analysis of specific objects within an environment. We start with the problem of visual reconstruction where the planner must maximize visual coverage of a specific object in an unknown environment while minimizing the time and cost. Our goal is to gain as much information about the object as quickly as possible. By utilizing shape prediction deep learning models, we leverage predicted geometry for efficient planning. We further extend this to a multi-UAV system. With a reconstructed 3D digital model, efficient paths around an object can be created for close-up inspection. However, the purpose of inspection is to detect changes in the object. The second problem we study is inspecting an object when it has changed or no prior information about it is known. We study this in the context of infrastructure inspection. We validate our planning algorithm through real-world experiments and high-fidelity simulations. Further, we integrate defect detection into the process. In the second part, we study planning for monitoring entire environments rather than specific objects. Unlike object monitoring, we are interested in environmental monitoring of spatio-temporal processes. The goal of a planner for environmental monitoring is to maximize coverage of an area to understand the spatio-temporal changes in the environment. We study this problem in slow-changing and fast-changing environments. Specifically, we study it in the context of vegetative growth estimation and wildfire management. For the fast-changing wildfire environments, we utilize informative path planning for wildfire validation and localization. Our work also leverages long short-term memory (LSTM) networks for early fire detection.Item Inertial Parameter Identification of a Captured Payload Attached to a Robotic Manipulator on a Free-Flying Spacecraft(2024) Limparis, Nicholas Michael; Akin, David L; Aerospace Engineering; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)The groundwork for the dynamics of a free-flyer with a manipulator has been laid out by Yoshida, Vafa and Dubowsky, and Papadopoulos and Moosovian with the Generalized Jacobian Matrix, Virtual Manipulator, and Barycentric Vector Approach respectively. The identification of parameters for a robot manipulator has also been approached for industrial robots as well as through adaptive control theory. What is proposed is a method for inertial parameter identification and verification for a spacecraft with an attached manipulator that is an extension of the ground-fixed Inverse Direct Dynamic Model to function for a free-flying spacecraft. This method for inertial parameter identification for a spacecraft-manipulator system with an attached client spacecraft, debris, or other grappled payload is developed in this thesis and is experimentally tested using results for a servicer and an "unknown" grappled payload using three separate test beds. The results of the experiments show that the proposed method is capable of identifying the inertial parameters of the servicer and the grappled payload.Item Autonomous Robot Navigation in Challenging Real-World Indoor and Outdoor Environments(2024) Sathyamoorthy, Adarsh Jagan; Manocha, Dr. Dinesh; Electrical Engineering; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)The use of autonomous ground robots for various indoor and outdoor applications has burgeoned over the years. In indoor settings, their applications range from waiters in hotels, helpers in hospitals, cleaners in airports and malls, transporters of goods in warehouses, surveillance robots, etc. In unstructured outdoor settings, they have been used for exploration in off-road environments, search and rescue, package delivery, etc. To successfully accomplish these tasks, robots must overcome several challenges and navigate to their goal. In this dissertation, we present several novel algorithms for learning-based perception combined with model-based autonomous navigation in real-world indoor and outdoor environments. The presented algorithms address the problems of avoiding collisions in dense crowds (< 1 to 2 persons/sq.meter), reducing the occurrence of the freezing robot problem, navigating in a socially compliant manner without being obtrusive to humans, and avoiding transparent obstacles in indoor settings. In outdoor environments, they address challenges in estimating the traversabilityof off-road terrains and vegetation, and understanding explicit social rules (e.g. crossing streets using crosswalks). The presented algorithms are designed to operate in real-time using the limited computational capabilities on-board real wheeled and legged robots such as the Turtlebot 2, Clearpath Husky, and Boston Dynamics Spot. Furthermore, the algorithms have been evaluated in real-world environments with dense crowds, transparent obstacles, off-road terrains, and vegetation such as tall grass, bushes, trees, etc. They have demonstrated significant improvements in terms of several metrics such as increasing success rates by at least 50% (robot avoids collisions and reaches its goal), lowering freezing rates by at least 80% (robot does not halt/oscillate indefinitely), increasing pedestrian friendliness up to 100% higher, reducing vibrations experienced in off-road terrains by up to 22%, etc over the state-of-the-art algorithms in various test scenarios. The first part of this dissertation deals with socially-compliant navigation approaches for crowded indoor environments. The initial methods focus on collision avoidance, handling the freezing robot problem in crowds of varying densities by tracking individual pedestrians, and modeling regions the robot must avoid based on their future positions. Subsequent works expand on these models by considering pedestrian group behaviors. The next part of this dissertation focuses on outdoor navigation methods that estimate the traversability of various terrains, and complex vegetation (e.g. pliable obstacles such as tall grass) using perception inputs to navigate on safe, and stable terrains. The final part of the dissertation elaborates on methods designed for detecting and navigating complex obstacles in indoor and outdoor environments. It also explores a technique leveraging recent advancements in large vision language models for navigation in both settings. All proposed methods have been implemented and evaluated on real wheeled and legged robots.