Computer Science Theses and Dissertations

Permanent URI for this collectionhttp://hdl.handle.net/1903/2756

Browse

Search Results

Now showing 1 - 10 of 27

Machine Learning with Differentiable Physics Priors
(2024) Qiao, Yiling; Lin, Ming ML; Computer Science; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)
Differentiable physics priors enable gradient-based learning systems to adhere to physical dynamics. By making physics simulations differentiable, we can backpropagate through the physical consequences of actions. This pipeline allows agents to quickly learn to achieve desired effects in the physical world and is an effective technique for solving inverse problems in physical or dynamical systems. This new programming paradigm bridges model-based and data-driven methods, mitigating data scarcity and model bias simultaneously. My research focuses on developing scalable, powerful, and efficient differentiable physics simulators. We have created state-of-the-art differentiable physics for rigid bodies, cloth, fluids, articulated bodies, and deformable solids, achieving performance orders of magnitude better than existing alternatives. These differentiable simulators are applied to solve inverse problems, train control policies, and enhance reinforcement learning algorithms.
Enhanced Robot Planning and Perception Through Environment Prediction
(2024) Sharma, Vishnu Dutt; Tokekar, Pratap; Computer Science; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)
Mobile robots rely on maps to navigate through an environment. In the absence of any map, the robots must build the map online from partial observations as they move in the environment. Traditional methods build a map using only direct observations. In contrast, humans identify patterns in the observed environment and make informed guesses about what to expect ahead. Modeling these patterns explicitly is difficult due to the complexity in the environments. However, these complex models can be approximated well using learning-based methods in conjunction with large training data. By extracting patterns, robots can use not only direct observations but also predictions of what lies ahead to better navigate through an unknown environment. In this dissertation, we present several learning-based methods to equip mobile robots with prediction capabilities for efficient and safer operation. In the first part of the dissertation, we learn to predict using geometrical and structural patterns in the environment. Partially observed maps provide invaluable cues for accurately predicting the unobserved areas. We first demonstrate the capability of general learning-based approaches to model these patterns for a variety of overhead map modalities. Then we employ task-specific learning for faster navigation in indoor environments by predicting 2D occupancy in the nearby regions. This idea is further extended to 3D point cloud representation for object reconstruction. Predicting the shape of the full object from only partial views, our approach paves the way for efficient next-best-view planning, which is a crucial requirement for energy-constrained aerial robots. Deploying a team of robots can also accelerate mapping. Our algorithms benefit from this setup as more observation results in more accurate predictions and further improves the task efficiency in the aforementioned tasks. In the second part of the dissertation, we learn to predict using spatiotemporal patterns in the environment. We focus on dynamic tasks such as target tracking and coverage where we seek decentralized coordination between robots. We first show how graph neural networks can be used for more scalable and faster inference while achieving comparable coverage performance as classical approaches. We find that differentiable design is instrumental here for end-to-end task-oriented learning. Building on this, we present a differentiable decision-making framework that consists of a differentiable decentralized planner and a differentiable perception module for dynamic tracking. In the third part of the dissertation, we show how to harness semantic patterns in the environment. Adding semantic context to the observations can help the robots decipher the relations between objects and infer what may happen next based on the activity around them. We present a pipeline using vision-language models to capture a wider scene using an overhead camera to provide assistance to humans and robots in the scene. We use this setup to implement an assistive robot to help humans with daily tasks, and then present a semantic communication-based collaborative setup of overhead-ground agents, highlighting the embodiment-specific challenges they may encounter and how they can be overcome. The first three parts employ learning-based methods for predicting the environment. However, if the predictions are incorrect, this could pose a risk to the robot and its surroundings. The third part of the dissertation presents risk management methods with meta-reasoning over the predictions. We study two such methods: one extracting uncertainty from the prediction model for risk-aware planning, and another using a heuristic to adaptively switch between classical and prediction-based planning, resulting in safe and efficient robot navigation.
Planning and Perception for Unmanned Aerial Vehicles in Object and Environmental Monitoring
(2024) Dhami, Harnaik; Tokekar, Pratap; Computer Science; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)
Unmanned Aerial vehicles (UAVs) equipped with high-resolution sensors are enabling data collection from previously inaccessible locations on a remarkable spatio-temporal scale. These systems hold immense promise for revolutionizing various fields such as precision agriculture and infrastructure inspection where access to data is important. To fully exploit their potential, the development of autonomy algorithms geared toward planning and perception is critical. In this dissertation, we develop planning and perception algorithms, specifically when UAVs are used for data collection in monitoring applications. In the first part of this dissertation, we study problems of object monitoring and the planning challenges that arise with them. Object monitoring refers to the continuous observation, tracking, and analysis of specific objects within an environment. We start with the problem of visual reconstruction where the planner must maximize visual coverage of a specific object in an unknown environment while minimizing the time and cost. Our goal is to gain as much information about the object as quickly as possible. By utilizing shape prediction deep learning models, we leverage predicted geometry for efficient planning. We further extend this to a multi-UAV system. With a reconstructed 3D digital model, efficient paths around an object can be created for close-up inspection. However, the purpose of inspection is to detect changes in the object. The second problem we study is inspecting an object when it has changed or no prior information about it is known. We study this in the context of infrastructure inspection. We validate our planning algorithm through real-world experiments and high-fidelity simulations. Further, we integrate defect detection into the process. In the second part, we study planning for monitoring entire environments rather than specific objects. Unlike object monitoring, we are interested in environmental monitoring of spatio-temporal processes. The goal of a planner for environmental monitoring is to maximize coverage of an area to understand the spatio-temporal changes in the environment. We study this problem in slow-changing and fast-changing environments. Specifically, we study it in the context of vegetative growth estimation and wildfire management. For the fast-changing wildfire environments, we utilize informative path planning for wildfire validation and localization. Our work also leverages long short-term memory (LSTM) networks for early fire detection.
Minimal Perception: Enabling Autonomy on Resource-Constrained Robots
(2023) Singh, Chahat Deep; Aloimonos, Yiannis; Computer Science; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)
Mobile robots are widely used and crucial in diverse fields due to their autonomous task performance. They enhance efficiency, and safety, and enable novel applications like precision agriculture, environmental monitoring, disaster management, and inspection. Perception plays a vital role in their autonomous behavior for environmental understanding and interaction. Perception in robots refers to their ability to gather, process, and interpret environmental data, enabling autonomous interactions. It facilitates navigation, object identification, and real-time reactions. By integrating perception, robots achieve onboard autonomy, operating without constant human intervention, even in remote or hazardous areas. This enhances adaptability and scalability. This thesis explores the challenge of developing autonomous systems for smaller robots used in precise tasks like confined space inspections and robot pollination. These robots face limitations in real-time perception due to computing, power, and sensing constraints. To address this, we draw inspiration from small organisms such as insects and hummingbirds, known for their sophisticated perception, navigation, and survival abilities despite their minimalistic sensory and neural systems. This research aims to provide insights into designing compact, efficient, and minimal perception systems for tiny autonomous robots. Embracing this minimalism is paramount in unlocking the full potential of tiny robots and enhancing their perception systems. By streamlining and simplifying their design and functionality, these compact robots can maximize efficiency and overcome limitations imposed by size constraints. In this work, a Minimal Perception framework is proposed that enables onboard autonomy in resource-constrained robots at scales (as small as a credit card) that were not possible before. Minimal perception refers to a simplified, efficient, and selective approach from both hardware and software perspectives to gather and process sensory information. Adopting a task-centric perspective allows for further refinement of the minimalist perception framework for tiny robots. For instance, certain animals like jumping spiders, measuring just 1/2 inch in length, demonstrate minimal perception capabilities through sparse vision facilitated by multiple eyes, enabling them to efficiently perceive their surroundings and capture prey with remarkable agility. This thesis introduces a cutting-edge exploration of the minimal perception framework, pushing the boundaries of robot autonomy to new heights. The contributions of this work can be summarized as follows:1. Utilizing minimal quantities such as uncertainty in optical flow and its untapped potential to enable autonomous navigation, static and dynamic obstacle avoidance, and the ability to fly through unknown gaps. 2. By utilizing the principles of interactive perception, the framework proposes novel object segmentation in cluttered environments eliminating the reliance on neural network training for object recognition. 3. Introducing a generative simulator called WorldGen that has the power to generate countless cities and petabytes of high-quality annotated data, designed to minimize the demanding need for laborious 3D modeling and annotations, thus unlocking unprecedented possibilities for perception and autonomy tasks. 4. Proposed a method to predict metric dense depth maps in never-seen or out-of-domain environments by fusing information from a traditional RGB camera and a sparse 64-pixel depth sensor. 5. The autonomous capabilities of the tiny robots are demonstrated on both aerial and ground robots: (a) autonomous car with a size smaller than a credit card (70mm), and (b) bee drone with a length of 120mm, showcasing navigation abilities, depth perception in all four main directions, and effective avoidance of both static and dynamic obstacles. In conclusion, the integration of the minimal perception framework in tiny mobile robots heralds a new era of possibilities, signaling a paradigm shift in unlocking their perception and autonomy potential. This thesis would serve as a transformative milestone that will reshape the landscape of mobile robot autonomy, ushering in a future where tiny robots operate synergistically in swarms, revolutionizing fields such as exploration, disaster response, and distributed sensing.
Outdoor Localization and Path Planning for Repositioning a Self-Driving Electric Scooter
(2023) Poojari, Srijal Shekhar; Paley, Derek; Systems Engineering; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)
The long-term goal of this research is to develop self-driving e-scooter technology to increase sustainability, accessibility, and equity in urban mobility. Toward this goal, in this work, we design and implement a self-driving e-scooter with the ability to safely travel along a pre-planned route using automated, onboard control without a rider. We also construct an autonomous driving framework by synthesizing open-source robotics software libraries with custom-designed modules specific to an e-scooter, including path planning and state estimation. The hardware and software development steps along with design choices and pitfalls are documented. Results of real-world autonomous navigation of our retrofitted e-scooter, along with the effectiveness of our localization methods are presented.
Learning Metareasoning Policies for Motion Planning
(2023) GOPAL, SIDDHARTH; Herrmann, Jeffrey W; Systems Engineering; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)
Metareasoning is the process of reasoning about reasoning. This thesis applies metareasoning to motion planning and evaluates three different metareasoning policies. Two policies are rule-based policies and are human specified. The third policy is a smart metareasoning policy that learns from the robot's past experiences, particularly the front camera images. The data is obtained by running the robot without a metareasoner in modular test scenarios which can be combined to form multiple real-world situations. The policy is stored in the form of the weights of a neural network. The neural network-based model used for this research is a multi-input classifier that chooses an optimal planner combination from amongst eight different planner combinations. The metareasoners are tested on a Unity simulator with a Clearpath Warthog ground robot. This thesis tests the performance of the robot under eight different test scenarios for eight different planner combinations and shows an improvement in the robot's success rate when using a metareasoner. Lastly, this thesis also provides a comparative study between a rule-based metareasoner and a smart metareasoner by introducing two new test scenarios which are not part of the robot's past experiences.
A FRAMEWORK FOR DEXTEROUS MANIPULATION THROUGH TACTILE PERCEPTION
(2022) Ganguly, Kanishka; Aloimonos, Yiannis; Computer Science; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)
A long-anticipated, yet hitherto unfilled goal in Robotics research has been to have robotic agents seamlessly integrating with humans in their natural environments, and performing useful tasks alongside humans. While tremendous progress has been made in allowing robots to perceive visually, and understand and reason about the scene, the act of manipulating said environment still remains a challenging and incomplete task.For robotic agents to have capabilities where they can perform useful tasks in environments that are not specifically designed for their operation, it is crucial to have dexterous manipulation capabilities guided by some form of tactile perception. While visual perception provides a large-scale understanding of the environment, tactile perception allows fine-grained understanding of objects and textures. For truly useful robotic agents, a tightly coupled system comprising both visual and tactile perception is a necessity. Tactile sensing hardware can be classified on a spectrum, organized by form-factor on one end to sensing accuracy and robustness on the other. Most off-the-shelf sensors available today trade off one of these features for the other. The tactile sensor used in this research, the BioTac SP, has been selected for its anthropomorphic qualities, such as its shape and sensing mechanism while compromising on quality of sensory outputs. This sensor provides a sensing surface, and returns 24 tactile points of data at each timestamp, along with pressure values. We first present a novel method for contact and motion estimation through visual perception, where we perform non-rigid registration of a human performing actions and compute dense motion estimation trajectories. This is used to compute topological scene changes, and is refined to get object and contact segmentation. We then ground these contact points and motion trajectories to an intermediate action-graph, which can then executed by a robot agent. Secondly, we introduce the concept of computational tactile flow, which is inspired by fMRI studies on humans where it was discovered that the same parts of the brain that react to optical motion stimulus also react to tactile stimulus. We mathematically model the BioTac SP sensor, and interpolate surfaces in two- and three dimensions, on which we compute tactile flow fields. We demonstrate the flow fields on various surfaces, and suggest various useful applications of tactile flow. We next apply tactile feedback to a novel controller, that is able to grasp objects without any prior knowledge about the shape, material, or weight of the objects. We apply tactile flow to compute slippage during grasp, and adjust the finger forces to maintain stable grasp during motion. We demonstrate success on transparent and soft, deformable objects, alongside other regularly shaped samples. Lastly, we take a different approach to processing tactile data, where we compute tactile events taking inspiration from neuromorphic computing literature. We compute spatio-temporal gradients on the raw tactile data, to generate event surfaces, which are more robust and reduces sensor noise. This intermediate surface is then used to track contact regions over the BioTac SP sensor skin, and allows us to detect slippage, track spatial edge contours, and magnitude of applied forces.
Efficient Environment Sensing and Learning for Mobile Robots
(2022) Suryan, Varun; Tokekar, Pratap; Computer Science; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)
Data-driven learning is becoming an integral part of many robotic systems. Robots can be used as mobile sensors to learn about the environment in which they operate. Robots can also seek to learn essential skills, such as navigation, within the environment. A critical challenge in both types of learning is sample efficiency. Acquiring samples with physical robots can be prohibitively time-consuming. As a result, when applying learning techniques in robotics that require physical interaction with the environment, minimizing the number of such interactions becomes a key. The key question we seek to answer is: How do we make robots learn efficiently with a minimal amount of physical interaction? We approach this question along two fronts: extrinsic learning and intrinsic learning. In extrinsic learning, we want the robot to learn about the external environment in which it is operating. In intrinsic learning, our focus is on the robot to learn a skill using reinforcement learning (RL) such as navigating in an environment. In this dissertation, we develop algorithms that carefully plan where the robots obtain samples in order to efficiently perform intrinsic and extrinsic learning. In particular, we exploit the structural properties of Gaussian Process (GP) regression to design efficient sampling algorithms. We study two types of problems under extrinsic learning. We start with the problem of learning a spatially varying field modeled by a GP efficiently. Our goal is to ensure that the GP posterior variance, which is also the mean square error between the learned and actual fields, is below a predefined value. By exploiting the underlying properties of GP, we present a series of constant-factor approximation algorithms for minimizing the number of stationary sensors to place, minimizing the total time taken by a single robot, and minimizing the time taken by a team of robots to learn the field. Here, we assume that the GP hyperparameters are known. We then study a variant where our goal is to identify the hotspot in an environment. Here we do not assume that hyperparameters are unknown. For this problem, we present Upper Confidence Bound (UCB) and Monte Carlo Tree Search (MCTS) based algorithms for a single robot and later extend them to decentralized multi-robot teams. We also validate their performance on real-world datasets. For intrinsic learning, our aim is to reduce the number of physical interactions by leveraging simulations often known as Multi-Fidelity Reinforcement Learning (MFRL). In the MFRL framework, an agent uses multiple simulators of the real environment to perform actions. We present two MFRL framework versions, model-based and model-free, that leverage GPs to learn the optimal policy in a real-world environment. By incorporating GPs in the MFRL framework, we empirically observe a significant reduction in the number of samples for model-based and model-free learning.
TOWARDS AUTONOMOUS DRIVING IN DENSE, HETEROGENEOUS, AND UNSTRUCTURED TRAFFIC
(2022) Chandra, Rohan; Manocha, Dinesh; Computer Science; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)
This dissertation addressed many key problems in autonomous driving towards handling dense, heterogeneous, and unstructured traffic environments. Autonomous vehicles (AV) at present are restricted to operating on smooth and well-marked roads, in sparse traffic, and among well-behaved drivers. We developed new techniques to perceive, predict, and plan among human drivers in traffic that is significantly denser in terms of number of traffic-agents, more heterogeneous in terms of size and dynamic constraints of traffic agents, and where many drivers do not follow the traffic rules. In this thesis, we present work along three themes—perception, driver behavior modeling, and planning. Our novel contributions include: 1. Improved tracking and trajectory prediction algorithms for dense and heterogeneous traffic using a combination of computer vision and deep learning techniques. 2. A novel behavior modeling approach using graph theory for characterizing human drivers as aggressive or conservative from their trajectories. 3. Behavior-driven planning and navigation algorithms in mixed (human driver and AV) and unstructured traffic environments using game theory and risk-aware control. Additionally, we have released a new traffic dataset, METEOR, which captures rare and interesting, multi-agent driving behaviors in India. These behaviors are grouped into traffic violations, atypical interactions, and diverse scenarios. We evaluate our perception work on tracking and trajectory prediction using standard autonomous driving datasets such as the Waymo Open Motion, Argoverse, NuScenes datasets, as well as public leaderboards where our tracking approach resulted in achieving rank 1 among over a 100 methods. We apply human driver behavior modeling in planning and navigation at unsignaled intersections and highways scenarios using state-of-the-art traffic simulators and show that our approach yields fewer collisions and deadlocks compared to methods based on deep reinforcement learning. We conclude the presentation with a discussion on future work.
Generating Feasible Spawn Locations for Autonomous Robot Simulations in Complex Environments
(2022) Ropelato, Rafael Florian; Herrmann, Jeffrey W; Systems Engineering; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)
Simulations have become one of the main methods in the development of autonomous robots. With the application of physical simulations that closely represent real-world environments, the behavior of a robot in a variety of situations can be tested in a more efficient manner than performing experiments in reality. With the implementation of ROS (Robot Operating System), the software of an autonomous system can be simulated separately without an existing robot. In order to simulate the physical environment surrounding the robot, a physics simulation has to be created through which the robot navigates and performs tasks. A commonly used platform for such simulations is Unity which provides a wide range of simulation capabilities as well as an interface for ROS. In order to perform multi-agent simulations or simulations with varying initial locations for the robot, it is crucial to find unobstructed spawn locations to avoid undesirable situations with collisions upon start of the simulation. For this purpose, multiple methods were implemented with this research, in order to generate feasible spawn locations within complex environments. Each of the three applied methods generates a set of valid spawn positions, which can be used to design simulations with varying initial locations for the agents. To assess the performance and functionality of these approaches, the algorithms were applied to several environments varying in complexity and scale. Overall, the implemented approaches performed very well in the applied environments, and generated mainly correctly classified locations which are suitable to spawn a robot. All approaches were tested for performance and compared in respect to their fitness to be applied to environments of varying complexity and scale. The resulting algorithms can be considered a efficient solutions to prepare simulations with multiple initial locations for robots and other test objects.

Computer Science Theses and Dissertations

Browse

Filters

Settings

Sort By

Results per page

Search Results