A. James Clark School of Engineering
Permanent URI for this communityhttp://hdl.handle.net/1903/1654
The collections in this community comprise faculty research works, as well as graduate theses and dissertations.
Browse
105 results
Search Results
Item Deployment of Large Vision and Language Models for Real-Time Robotic Triage in a Mass Casualty Incident(2024) Mangel, Alexandra Paige; Paley, Derek; Aerospace Engineering; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)In the event of a mass casualty incident, such as a natural disaster or war zone, having a system of triage in place that is efficient and accurate is critical for life-saving intervention, but medical personnel and resources are often strained and struggle to provide immediate care to those in need. This thesis proposes a system of autonomous air and ground vehicles equipped with stand-off sensing equipment designed to detect and localize casualties and assess them for critical injury patterns. The goal is to assist emergency medical technicians in identifying those in need of primary care by using generative AI models to analyze casualty images and communicate with the victims. Large language models are explored for the purpose of developing a chatbot that can ask a casualty where they are experiencing pain and make an informed assessment about injury classifications, and a vision language model is prompt engineered to assess a casualty image to produce a report on designated injury classifiers.Item TOWARDS EFFICIENT OCEANIC ROBOT LEARNING WITH SIMULATION(2024) LIN, Xiaomin; Aloimonos, Yiannis; Electrical Engineering; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)In this dissertation, I explore the intersection of machine learning, perception, and simulation-based techniques to enhance the efficiency of underwater robotics, with a focus on oceanic tasks. My research begins with marine object detection using aerial imagery. From there, I address oyster detection using Oysternet, which leverages simulated data and Generative Adversarial Networks for sim-to-real transfer, significantly improving detection accuracy. Next, I present an oyster detection system that integrates diffusion-enhanced synthetic data with the Aqua2 biomimetic hexapedal robot, enabling real-time, on-edge detection in underwater environments. With detection models deployed locally, this system facilitates autonomous exploration. To enhance this capability, I introduce an underwater navigation framework that employs imitation learning, enabling the robot to efficiently navigate over objects of interest, such as rock and oyster reefs, without relying on localization. This approach improves information gathering while ensuring obstacle avoidance. Given that oyster habitats are often in shallow waters, I incorporate a deep learning model for real/virtual image segmentation, allowing the robot to differentiate between actual objects and water surface reflections, ensuring safe navigation. I expand on broader applications of these techniques, including olive detection for yield estimation and industrial object counting for warehouse management, using simulated imagery. In the final chapters, I address unresolved challenges, such as RGB/sonar data integration, and propose directions for future research to enhance underwater robotic learning through digital simulation further. Through these studies, I demonstrate how machine learning models and digital simulations can be used synergistically to address key challenges in underwater robotic tasks. Ultimately, this work advances the capabilities of autonomous systems to monitor and preserve marine ecosystems through efficient and robust digital simulation-based learning.Item Applied Aerial Autonomy for Reliable Indoor Flight and 3D Mapping(2024) Shastry, Animesh Kumar; Paley, Derek; Aerospace Engineering; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)Uncrewed Aerial Systems (UAS) are essential for safely exploring indoor environments damaged by shelling, fire, floods, and structural collapse. These systems can gather critical visual and locational data, aiding in hazard assessment and rescue planning without risking human lives. Reliable UAS deployments requires advanced sensors and robust algorithms for real-time data processing and safe navigation, even in GPS-denied and windy conditions. This dissertation details three research projects to improve UAS performance: (1) in-flight calibration to improve estimation and control, (2) system identification for wind rejection, and (3) indoor aerial 3D mapping. The dissertation begins with introducing a comprehensive nonlinear filtering framework for UAV parameter estimation, which considers factors such as external wind, drag coefficients, IMU bias, and center of pressure. Additionally, it establishes optimized flight trajectories for parameter estimation through empirical observability. Moreover, an estimation and control framework is implemented, utilizing the mean of state and parameter estimates to generate suitable control inputs for vehicle actuators. By employing a square-root unscented Kalman filter (sq-UKF), this framework can derive a 23-dimensional state vector from 9-dimensional sensor data and 4-dimensional control inputs. Numerical results demonstrate enhanced tracking performance through the integration of the estimation framework with a conventional model-based controller. The estimation of unsteady winds results in improved gust rejection capabilities of the onboard controller as well. Closely related to parameter estimation is system identification. Combining with the previous work a comprehensive system identification framework with both linear offline and nonlinear online methods is introduced. Inertial parameters are estimated using frequency-domain linear system identification, incorporating control data from motor-speed sensing and state estimates from automated frequency sweep maneuvers. Additionally, drag-force coefficients and external wind are recursively estimated during flight using a sq-UKF. A custom flight controller is developed to manage the computational demands of online estimation and control. Flight experiments demonstrate the tracking performance of the nonlinear controller and its improved capability in rejecting gust disturbances. Aside from wind rejection, aerial indoor 3D mapping is also required for indoor navigation, and therefore, the dissertation introduces a comprehensive pipeline for real-time mapping and target detection in indoor environments with limited network access. Seeking a best-in-class UAS design, it provides detailed analysis and evaluation of both hardware and software components. Experimental testing across various indoor settings demonstrates the system's efficacy in producing high-quality maps and detecting targets.Item Multi-Domain Human-Robot Interfaces(2024) Abdi, Sydrak Solomon; Paley, Derek; Aerospace Engineering; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)As autonomous robots become more capable and integrated into daily society, it becomes crucial to consider how a user will interact with them, how a robot will perceive a user, and how a robot will comprehend a user’s intentions. This challenge increases in difficulty when the user is required to interact with and control multiple robots simultaneously. Human intervention is often required during autonomous operations, particularly in scenarios that involve complex decision-making or where safety concerns arise. Thus, the methods by which users interact with multi-agent systems is an important area of research. These interactions should be intuitive, efficient, and effective all while preserving the operator's safety. We present a novel human swarm interface (HSI) that utilizes gesture control and haptic feedback to interact with and control a swarm of quadrotors in a confined space. This human swarm interface prioritizes operator safety while reducing cognitive load during control of an aerial swarm. Human-robot interfaces (HRIs) are mechanisms designed to facilitate communication between humans and robots, enhancing the user's ability to command and collaborate with robots in an intuitive and user-friendly manner. One challenge is providing mobile robotic systems with the capability to localize and interact with a user in their environment. Localization involves estimating the pose (position and orientation) of the user relative to the robot, which is essential for tasks that require close interactions or navigation in shared spaces. We present a novel method for obtaining user pose as well as other anthropometric measurements useful for human-robot interactions. Another challenge is extending these HRI and HSI paradigms to the outdoors. Unlike controlled laboratory conditions, outdoor environments involve a variety of variables such as fluctuating weather conditions as well as a mix of static and dynamic obstacles. In this dissertation, we design a portable human swarm interface that allows an operator to interact with and control a multi-agent system outdoors. The portable HSI takes the form of smart binoculars. The user uses the smart binoculars to select an outdoor location and assign a task for the multi-agent system to complete given the targeted area. This system allows for new methods of multi-agent operation, that will leverage a user's on-the-ground knowledge while utilizing autonomous vehicles for line-of-sight operations, without compromising their situational awareness.Item INGESTIBLE BIOIMPEDANCE SENSING DEVICE FOR GASTROINTESTINAL TRACT MONITORING(2024) Holt, Brian Michael; Ghodssi, Reza; Electrical Engineering; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)Gastrointestinal (GI) diseases, such as inflammatory bowel disease (IBD), result in dilated adherens and tight junctions, altering mucosal tissue permeability. Few monitoring techniques have been developed for in situ monitoring of local mucosal barrier integrity, and none are capable of non-invasive measurement beyond the esophagus. In this work, this technology gap is addressed through the development of a noise-resilient, flexible bioimpedance sensor integrated ingestible device containing electronics for low-power, four-wire impedance measurement and Bluetooth-enabled wireless communication. Through electrochemical deposition of a conductive polymeric film, the sensor charge transfer capacity is increased 51.4-fold, enabling low-noise characterization of excised intestinal tissues with integrated potentiostat circuitry for the first time. A rodent animal trial is performed, demonstrating successful differentiation of healthy and permeable mice colonic tissues using the developed device. In accordance with established mucosal barrier evaluation methodologies, mucosal impedance was reduced between 20.3 ± 9.0% and 53.6 ± 10.7% of its baseline value in response to incrementally induced tight junction dilation. Ultimately, this work addresses the fundamental challenges of electrical resistance techniques hindering localized, non-invasive IBD diagnostics. Through the development of a simple and reliable bioimpedance sensing module, the device marks significant progress towards explicit quantification of “leaky gut” patterns in the GI tract.Item Inertial Parameter Identification of a Captured Payload Attached to a Robotic Manipulator on a Free-Flying Spacecraft(2024) Limparis, Nicholas Michael; Akin, David L; Aerospace Engineering; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)The groundwork for the dynamics of a free-flyer with a manipulator has been laid out by Yoshida, Vafa and Dubowsky, and Papadopoulos and Moosovian with the Generalized Jacobian Matrix, Virtual Manipulator, and Barycentric Vector Approach respectively. The identification of parameters for a robot manipulator has also been approached for industrial robots as well as through adaptive control theory. What is proposed is a method for inertial parameter identification and verification for a spacecraft with an attached manipulator that is an extension of the ground-fixed Inverse Direct Dynamic Model to function for a free-flying spacecraft. This method for inertial parameter identification for a spacecraft-manipulator system with an attached client spacecraft, debris, or other grappled payload is developed in this thesis and is experimentally tested using results for a servicer and an "unknown" grappled payload using three separate test beds. The results of the experiments show that the proposed method is capable of identifying the inertial parameters of the servicer and the grappled payload.Item Autonomous Robot Navigation in Challenging Real-World Indoor and Outdoor Environments(2024) Sathyamoorthy, Adarsh Jagan; Manocha, Dr. Dinesh; Electrical Engineering; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)The use of autonomous ground robots for various indoor and outdoor applications has burgeoned over the years. In indoor settings, their applications range from waiters in hotels, helpers in hospitals, cleaners in airports and malls, transporters of goods in warehouses, surveillance robots, etc. In unstructured outdoor settings, they have been used for exploration in off-road environments, search and rescue, package delivery, etc. To successfully accomplish these tasks, robots must overcome several challenges and navigate to their goal. In this dissertation, we present several novel algorithms for learning-based perception combined with model-based autonomous navigation in real-world indoor and outdoor environments. The presented algorithms address the problems of avoiding collisions in dense crowds (< 1 to 2 persons/sq.meter), reducing the occurrence of the freezing robot problem, navigating in a socially compliant manner without being obtrusive to humans, and avoiding transparent obstacles in indoor settings. In outdoor environments, they address challenges in estimating the traversabilityof off-road terrains and vegetation, and understanding explicit social rules (e.g. crossing streets using crosswalks). The presented algorithms are designed to operate in real-time using the limited computational capabilities on-board real wheeled and legged robots such as the Turtlebot 2, Clearpath Husky, and Boston Dynamics Spot. Furthermore, the algorithms have been evaluated in real-world environments with dense crowds, transparent obstacles, off-road terrains, and vegetation such as tall grass, bushes, trees, etc. They have demonstrated significant improvements in terms of several metrics such as increasing success rates by at least 50% (robot avoids collisions and reaches its goal), lowering freezing rates by at least 80% (robot does not halt/oscillate indefinitely), increasing pedestrian friendliness up to 100% higher, reducing vibrations experienced in off-road terrains by up to 22%, etc over the state-of-the-art algorithms in various test scenarios. The first part of this dissertation deals with socially-compliant navigation approaches for crowded indoor environments. The initial methods focus on collision avoidance, handling the freezing robot problem in crowds of varying densities by tracking individual pedestrians, and modeling regions the robot must avoid based on their future positions. Subsequent works expand on these models by considering pedestrian group behaviors. The next part of this dissertation focuses on outdoor navigation methods that estimate the traversability of various terrains, and complex vegetation (e.g. pliable obstacles such as tall grass) using perception inputs to navigate on safe, and stable terrains. The final part of the dissertation elaborates on methods designed for detecting and navigating complex obstacles in indoor and outdoor environments. It also explores a technique leveraging recent advancements in large vision language models for navigation in both settings. All proposed methods have been implemented and evaluated on real wheeled and legged robots.Item Metareasoning Strategies to Correct Navigation Failures of Autonomous Ground Robots(2024) Molnar, Sidney Leigh; Herrmann, Jeffrey; Mechanical Engineering; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)Due to the complexity of autonomous systems, theoretically perfect path planning algorithms sometimes fail due to emergent behaviors that arise when interacting with different perception, mapping and goal planning subprocesses. These failures prevent mission success, especially in complex environments that have not previously been explored by the robot. To overcome these failures, many researchers have sought to develop parameter learning methods to improve either mission success or path planning convergence. Metareasoning, which can be simply described as “thinking about thinking,” offers another possible solution for mitigating these planning failures. This project offers a novel metareasoning approach that uses different methods of monitoring and control to detect and overcome path planning irregularities that contribute to path planning failures. All methods for the approaches were implemented as a part of the ARL ground autonomy stack which uses both global and local path planning ROS nodes. The proposed monitoring methods include listening to messages published to the system by the planning algorithms themselves, evaluating for the environmental context that the robot is in, the expected progress methods which use the robot’s movement capabilities to evaluate for progress that has been made from a milestone checkpoint, and the fixed radius methods which use user-selected parameters based on mission objectives to evaluate for the progress that has been made from a milestone checkpoint. The proposed control policies are the metric-based sequential policies which use benchmark robot performance metrics to select the order in which the planner combinations are to be launched, the context-based pairs policies which evaluate what happens when switching between only two planner combinations, and the restart policy which simply relaunches a new instance of the same planner combination. The study evaluated which monitoring and control policies, when paired, contributed to improved navigation performance and which policies contributed to degraded navigation performance by evaluating how close the robot was able to get to the final mission goal. Although specific methods were evaluated, the contributions of the project extend beyond the results by offering both a template for metareasoning approaches with regard to navigation as well as replicable algorithms that may be applied to any autonomous ground robot system. Additionally, this thesis presents ideas for additional research in order to determine under which conditions metareasoning will improve navigation.Item ON DATA-BASED MAPPING AND NAVIGATION OF UNMANNED GROUND VEHICLES(2024) Herr, Gurtajbir Singh; Chopra, Nikhil; Mechanical Engineering; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)Unmanned ground vehicles (UGVs) have seen tremendous advancement in their capabilities and applications in the past two decades. With several key algorithmic and hardware breakthroughs and advancements in deep learning, UGVs are quickly becoming ubiquitous (finding applications as self-driving cars, for remote site inspections, in hospitals and shopping malls, among several others). Motivated by their large-scale adoption, this dissertation aims to enable the navigation of UGVs in complex environments. In this dissertation, a supervised learning-based navigation algorithm that utilizes model predictive control (MPC) for providing training data is developed. Improving MPC performance by data-based modelling of complex vehicle dynamics is then addressed. Finally, this dissertation deals with detecting and registering transparent objects that may deteriorate navigation performance. Navigation in dynamic environments poses unique challenges, particularly due to the limited knowledge of the decisions made by other agents and their objectives. In this dissertation, a solution that utilizes an MPC-based planner as an \textit{expert} to generate high-quality motion commands for a car-like robot operating in a simulated dynamic environment is proposed. These commands are then used to train a deep neural network, which learns to navigate. The deep learning-based planner is further enhanced with safety margins to improve its effectiveness in collision avoidance. The performance of the proposed method through simulations and real-world experiments, demonstrating its superiority in terms of obstacle avoidance and successful mission completion is showcased. This research has practical implications for the development of safer and more efficient autonomous vehicles. Many real-world applications rely on MPC to control UGVs due to its safety guarantees and constraint satisfaction properties. However, the performance of such MPC-based solutions is heavily reliant on the accuracy of the motion model. This dissertation addresses this challenge by exploring a data-based approach to discovering vehicle dynamics. Unlike existing physics-based models that require extensive testing setups and manual tuning for new platforms and driving surfaces, our approach leverages the universal differential equations (UDEs) framework to identify unknown dynamics from vehicle data. This innovative approach, which does not make assumptions about the unknown dynamics terms and directly models the vector field, is then deployed to showcase its efficacy. This research opens up new possibilities for more accurate and adaptable motion models for UGVs. With the increasing adoption of glass and other transparent materials, UGVS must be able to detect and register them for reliable navigation. Unfortunately, such objects are not easily detected by LiDARs and cameras. In this dissertation, algorithms for detecting and including glass objects in a Graph SLAM framework were studied. A simple and computationally inexpensive glass detection scheme to detect glass objects is utilized. The methodology to incorporate the identified objects into the occupancy grid maintained by such a framework is the presented. The issue of \textit{drift accumulation} that can affect mapping performance when operating in large environments is also addressed.Item INDOOR TARGET SEARCH, DETECTION, AND INSPECTION WITH AN AUTONOMOUS DRONE(2024) Ashry, Ahmed; Paley, Derek; Aerospace Engineering; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)This thesis investigates the deployment of unmanned aerial vehicles (UAVs) in indoor search and rescue (SAR) operations, focusing on enhancing autonomy through the development and integration of advanced technological solutions. The research addresses challenges related to autonomous navigation and target inspection in indoor environments. A key contribution is the development of an autonomous inspection routine that allows UAVs to navigate to and meticulously inspect targets identified by fiducial markers, replacing manual piloted inspection. To enhance the system’s target recognition, a custom-trained object detection model identifies critical markers on targets, operating in real-time on the UAV’s onboard computer. Additionally, the thesis introduces a comprehensive mission framework that manages transitions between coverage and inspection phases, experimentally validated using a quadrotor equipped with onboard sensing and computing across various scenarios. The research also explores integration and critical analysis of state-of-the-art path planning algorithms, enhancing UAV autonomy in cluttered settings. This is supported by evaluations conducted through software-in-the-loop simulations, setting the stage for future real-world applications.