A. James Clark School of Engineering
Permanent URI for this communityhttp://hdl.handle.net/1903/1654
The collections in this community comprise faculty research works, as well as graduate theses and dissertations.
Browse
50 results
Search Results
Item Representation Learning for Reinforcement Learning: Modeling Non-Gaussian Transition Probabilities with a Wasserstein Critic(2024) Tse, Ryan; Zhang, Kaiqing; Electrical Engineering; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)Reinforcement learning algorithms depend on effective state representations when solving complex, high-dimensional environments. Recent methods learn state representations using auxiliary objectives that aim to capture relationships between states that are behaviorally similar, meaning states that lead to similar future outcomes under optimal policies. These methods learn explicit probabilistic state transition models and compute distributional distances between state transition probabilities as part of their measure of behavioral similarity. This thesis presents a novel extension to several of these methods that directly learns the 1-Wasserstein distance between state transition distributions by exploiting the Kantorovich-Rubenstein duality. This method eliminates parametric assumptions about the state transition probabilities while providing a smoother estimator of distributional distances. Empirical evaluation demonstrates improved sample efficiency over some of the original methods and a modest increase in computational cost per sample. The results establish that relaxing theoretical assumptions about state transition modeling leads to more flexible and robust representation learning while maintaining strong performance characteristics.xItem Efficient learning-based sound propagation for virtual and real-world audio processing applications(2024) Ratnarajah, Anton Jeran; Manocha, Dinesh; Electrical Engineering; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)Sound propagation is the process by which sound energy travels through a medium, such as air, to the surrounding environment as sound waves. The room impulse response (RIR) describes this process and is influenced by the positions of the source and listener, the room's geometry, and its materials. Physics-based acoustic simulators have been used for decades to compute accurate RIRs for specific acoustic environments. However, we have encountered limitations with existing acoustic simulators. For example, they require a 3D representation and detailed material knowledge of the environment. To address these limitations, we propose three novel solutions. First, we introduce a learning-based RIR generator that is two orders of magnitude faster than an interactive ray-tracing simulator. Our approach can be trained to input both statistical and traditional parameters directly, and it can generate both monaural and binaural RIRs for both reconstructed and synthetic 3D scenes. Our generated RIRs outperform interactive ray-tracing simulators in speech-processing applications, including Automatic Speech Recognition (ASR), Speech Enhancement, and Speech Separation, by 2.5%, 12%, and 48%, respectively. Secondly, we propose estimating RIRs from reverberant speech signals and visual cues in the absence of a 3D representation of the environment. By estimating RIRs from reverberant speech, we can augment training data to match test data, improving the word error rate of the ASR system. Our estimated RIRs achieve a 6.9% improvement over previous learning-based RIR estimators in real-world far-field ASR tasks. We demonstrate that our audio-visual RIR estimator aids tasks like visual acoustic matching, novel-view acoustic synthesis, and voice dubbing, validated through perceptual evaluation. Finally, we introduce IR-GAN to augment accurate RIRs using real RIRs. IR-GAN parametrically controls acoustic parameters learned from real RIRs to generate new RIRs that imitate different acoustic environments, outperforming Ray-tracing simulators on the Kaldi far-field ASR benchmark by 8.95%.Item Learning Autonomous Underwater Navigation with Bearing-Only Data(2024) Robertson, James; Duraiswami, Ramani; Electrical Engineering; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)Recent applications of deep reinforcement learning in controlling maritime autonomoussurface vessels have shown promise for integration into maritime transportation. These could have the potential to reduce at-sea incidents such as collisions and groundings which are majorly attributed to human error. With this in mind the goal of this work is to evaluate how well a similar deep reinforcement learning agent could perform the same task in submarines but using passive SONAR rather than the ranging data provided by active RADAR aboard surface vessels. A simulated submarine outfitted with a passive spherical, hull-mounted SONAR sensor is placed into contact scenarios under the control of a reinforcement learning agent and directed to make its way to a navigational waypoint while avoiding interfering surface vessels. In order to see how this best translates to lower power autonomous vessels (vice warship submarines), no estimation for the range of the surface vessels is maintained in order to cut down on computing requirements. Inspired by my time aboard U.S. Navy submarines, the agent is provided with simply the simulated passive SONAR data. I show that this agent is capable of navigating to a waypoint while avoiding crossing, overtaking, and head-on surface vessels and thus could provide a recommended course to a submarine contact management team in ample time since the maneuvers made by the agent are not instantaneous in contrast to the assumptions of traditional target tracking with bearing-only data. Additionally, an in-progress plugin for Epic Games’ Unreal Engine is presented with the ability to simulate underwater acoustics inside the 3D development software. Unreal Engine is a powerful 3D game engine that is incredibly flexible and capable of being integrated into many different forms of scientific research. This plugin could provide researchers with the ability to conduct useful simulations in intuitively designed 3D environments.Item Denoising the Design Space: Diffusion Models for Accelerated Airfoil Shape Optimization(2024) Diniz, Cashen; Fuge, Mark D; Mechanical Engineering; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)Generative models offer the possibility to accelerate and potentially substitute parts of the often expensive traditional design optimization process. We present Aero-DDM, a novel application of a latent denoising diffusion model (DDM) capable of generating airfoil geometries conditioned on flow parameters and an area constraint. Additionally, we create a novel, diverse dataset of optimized airfoil designs that better reflects a realistic design space than has been done in previous work. Aero-DDM is applied to this dataset, and key metrics are assessed both statistically and with an open-source computational fluid dynamics (CFD) solver to determine the performance of the generated designs. We compare our approach to an optimal transport GAN, and demonstrate that our model can generate designs with superior performance statistically, in aerodynamic benchmarks, and in warm-start scenarios. We also extend our diffusion model approach, and demonstrate that the number of steps required for inference can be reduced by as much as ~86%, compared to an optimized version of the baseline inference process, without meaningful degradation in design quality, simply by using the initial design to start the denoising process.Item Prediction of Marine Timber Pile Damage Ratings Using a Gradient Boosted Regression Model(2023) Willmott, Carly; Attoh-Okine, Nii O.; Civil Engineering; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)Marine pilings are critical structural elements exposed to harsh environmental conditions. Specialized routine inspection and regular maintenance are essential to keep marine facilities in good working condition. These activities generate data that can be exploited for knowledge gain with machine learning tools. A gradient boosted random forest regressor machine learning algorithm, XGBoost, was applied to datasets that contain timber pile underwater inspection and repair data over a period of 23 years. First, the data was visualized to show the longevity of different timber pile repair types. An XGBoost model was then tuned and trained on a dataset for timber piles at one pier. Variables in the dataset were evaluated for feature importance in predicting damage ratings assigned during routine underwater inspections. Next, an ensemble of XGBoost models was trained and applied to a second dataset containing the same features for an adjacent pier. This dataset was reserved for testing to demonstrate whether the ensemble trained on one pier’s data could be generalized to predict timber pile damage ratings at a nearby but separate pier. Finally, the ensemble was used to predict damage ratings on piles that had earlier data but were not rated in the two most recent inspection events. Results suggest that the ensemble is capable of predicting timber pile damage ratings to approximately +/- one damage rating on both the training and test datasets. Feature importances revealed that half of the variables (time since the first event, repair type, exposed pile length, and time since the last repair) contributed to two thirds of the relative importance in predicting damage ratings. Data visualization showed that a few repair types, such as pile replacements and encapsulations, appeared to be most successful over the long term compared with shorter-lived repairs like wraps and encasements. These results are promising indications of the advantages machine learning algorithms can offer in processing and gleaning new insights from structural repair and inspection data. Economic benefits to marine facility owners can potentially be realized through earlier anticipation of repairs and more targeted inspection and rehabilitation efforts. There are also opportunities for better understanding of deterioration rates if more data is gathered over the lifespans of structures, as well as more detailed data that can be introduced as new features.Item FAST FEASIBLE MOTION PLANNING WITHOUT TWO-POINT BOUNDARY VALUE SOLUTION(2023) Nayak, Sharan Harish; Otte, Michael; Aerospace Engineering; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)Autonomous robotic systems have seen extensive deployment across domains such as manufacturing, industrial inspection, transportation, and planetary surface exploration. A crucial requirement for these systems is navigating from an initial to a final position, while avoiding potential collisions with obstacles en route. This challenging task of devising collision-free trajectories, formally termed as motion planning, is of prime importance in robotics. Traditional motion planning approaches encounter scalability challenges when planning in higher-dimensional state-spaces. Moreover, they rarely consider robot dynamics during the planning process. To address these limitations, a class of probabilistic planning methods called Sampling-Based Motion Planning (SBMP) has gained prominence. SBMP strategies exploit probabilistic techniques to construct motion planning solutions. In this dissertation, our focus turns towards feasible SBMP algorithms that prioritize rapidly discovering solutions while considering robot kinematics and dynamics. These algorithms find utility in quickly solving complex problems (e.g., Alpha puzzle) where obtaining any feasible solution is considered as an achievement. Furthermore, they find practical use in computationally constrained systems and in seeding time-consuming optimal solutions. However, many existing feasible SBMP approaches assume the ability to find precise trajectories that exactly connect two states in a robot's state space. This challenge is framed as the Two-Point Boundary Value Problem (TPBVP). But finding closed-form solutions for the TPBVP is difficult, and numerical approaches are computationally expensive and prone to precision and stability issues. Given these complexities, this dissertation's primary focus resides in the development of SBMP algorithms for different scenarios where solving the TPBVP is challenging. Our work addresses four distinct scenarios -- two for single-agent systems and two for multi-agent systems. The first single-agent scenario involves quickly finding a feasible path from the start to goal state, using bidirectional search strategies for fast solution discovery. The second scenario focuses on performing prompt motion replanning when a vehicle's dynamical constraints change mid-mission. We leverage the heuristic information from the original search tree constructed using the vehicle's nominal dynamics to speed up the replanning process. Both these two scenarios unfold in static environments with pre-known obstacles. Transitioning to multi-agent systems, we address the feasible multi-robot motion planning problem where a robot team is guided towards predefined targets in a partially-known environment. We employ a dynamic roadmap updated from the current known environment to accelerate agent planning. Lastly, we explore the problem of multi-robot exploration in a completely unknown environment applied to the CADRE mission. We demonstrate how our proposed bidirectional search strategies can facilitate efficient exploration for robots with significant dynamics. The effectiveness of our algorithms is validated through extensive simulation and real-world experiments.Item Cardiovascular Physiological Monitoring Based on Video(2023) Gebeyehu, Henok; Wu, Min; Electrical Engineering; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)Regular, continuous monitoring of the heart is advantageous to maintaining one’s cardiovascular health as it enables the early detection of potentially life-threatening cardiovascular diseases. Typically, the required devices for continuous monitoring are found in a clinical setting, but recent research developments have advanced remote physiological monitoring capabilities and expanded the options for continuous monitoring from home. This thesis focuses on further extending the monitoring capabilities of consumer electronic devices to motivate the feasibility of reconstructing Electrocardiograms via a smartphone camera. First, the relationship between skin tone and remote physiological sensing is examined as variations in melanin concentrations for people of diverse skin tones can affect remote physiological sensing. In this work, a study is performed to observe the prospect of reducing the performance disparity caused by melanin differences by exploring the sites from which the physiological signal is collected. Second, the physiological signals obtained from the previous part are enhanced to improve the signal-to-noise ratio and utilized to infer ECG as parts of a novel technique that emphasizes interpretability as a guiding principle. The findings in this work have the potential to enable and promote the remote sensing of a physiological signal that is more informative than what is currently possible with remote sensing.Item Robust Reinforcement Learning via Risk-Sensitivity(2023) Noorani, Erfaun; Baras, John S; Electrical Engineering; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)The objective of this research is to develop robust-resilient-adaptive Reinforcement Learning (RL) systems that are generic, provide performance guarantees, and can generalize-reason-improve in complex and unknown task environments. To achieve this objective, we focus on exploring the concept of Risk-sensitivity in RL systems and its extensions to Multi-Agent (MA) systems. The development of robust reinforcement learning algorithms is crucial to address challenges such as model misspecification, parameter uncertainty, disturbances, and more. Risk-sensitive methods offer an approach to developing robust RL algorithms by hedging against undesirable outcomes in a probabilistic manner. The robustness properties of risk-sensitive controllers have long been established. We investigate risk-sensitive RL (as a generalization of risk-sensitive stochastic control), by theoretically analyzing the risk-sensitive exponential (exponential of the total reward) criteria and the benefits and improvements the introduction of risk-sensitivity brings to conventional RL. By considering exponential criteria as risk measures, we aim to enhance the reliability of our decision-making process. We explore the exponential criteria to better understand its representation, the implications of its optimization, and the behavioral characteristics exhibited by an agent optimizing this criterion. We demonstrate the advantages of utilizing exponential criteria for the development of RL algorithms. We then shift our focus to developing algorithms that effectively leverage these exponential criteria. To do that, we first focus on developing risk-sensitive RL algorithms within the framework of Markov Decision Processes (MDPs). We then broaden our scope by exploring the application of the Probabilistic Graphical Models (PGM) framework for developing risk-sensitive algorithms. Within this context, we delve into the PGM framework and examine its connection with the MDP framework. We proceed by exploring the effects of risk sensitivity on trust, collaboration, and cooperation in multi-agent systems. To conclude, we finally investigate the concept of risk sensitivity and the robust properties of risk-sensitive algorithms in decision-making and optimization domains beyond RL. Specifically, we focus on safe RL using risk-sensitive filters. Through our exploration, we seek to enhance the understanding and applicability of risk-sensitive approaches in various domains.Item Development of Low-Cost Autonomous Systems(2023) Saar, Logan Miles; Takeuchi, Ichiro; Material Science and Engineering; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)A central challenge of materials discovery for improved technologies arises from the increasing compositional, processing, and structural complexity involved when synthesizing hitherto unexplored material systems. Traditional Edisonian and combinatorial high-throughput methods have not been able to keep up with the exponential growth in potential materials and relevant property metrics. Autonomously operated Self-Driving Labs (SDLs) - guided by the optimal experiment design sub-field of machine learning, known as active learning - have arisen as promising candidates for intelligently searching these high-dimensional search spaces. In the fields of biology, pharmacology, and chemistry, these SDLs have allowed for expedited experimental discovery of new drugs, catalysts, and more. However, in material science, highly specialized workflows and bespoke robotics have limited the impact of SDLs and contributed to their exorbitant costs. In order to equip the next generation workforce of scientists and advanced manufacturers with the skills needed to coexist with, improve, and understand the benefits and limitations of these autonomous systems, a low-cost and modular SDL must be available to them. This thesis describes the development of such a system and its implementation in an undergraduate and graduate machine learning for materials science course. The low-cost SDL system developed is shown to be affordable for primary through graduate level adoption, and provides a hands-on method for simultaneously teaching active learning, robotics, measurement science, programming, and teamwork: all necessary skills for an autonomous compatible workforce. A novel hypothesis generation and validation active learning scheme is also demonstrated in the discovery of simple composition/acidity relationships.Item INTERPRETABLE AND SPEED ADAPTIVE CONVOLUTIONAL NEURAL NETWORK FOR PROGNOSTICS AND HEALTH MANAGEMENT OF ROTATING MACHINERY(2023) Lee, Nam Kyoung; Pecht, Michael; Azarian, Michael H; Mechanical Engineering; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)Faulty rotating machines exhibit vibrational characteristics that can be distinguished from healthy machines using prognostics and health management methods. These characteristics can be extracted using signal processing techniques. However, these techniques require certain inputs, or parameters, before the desired characteristics can be extracted. Setting the parameters requires skill and knowledge, as they should reflect the component geometries and the operational conditions. Using convolutional neural networks for diagnosing faults on a rotating machine eliminates the need for parameter setting by replacing signal processing with mathematical operations in the networks. The parameters that affect the outcomes of the operations are learned from data during the training of the neural networks. The networks can capture characteristics that are related to the health state of a machine, but their operations are not interpretable. Unlike signal processing, the internal operations of the networks have no constraints that guide the networks to transform vibrations into certain information, that is, vibrational characteristics. Without the constraints, there is no basis for understanding the characteristics in terms that can be associated with the physics of failure. The lack of interpretability impedes the physical validation of vibrational characteristics captured by the networks.This dissertation presents a method for changing the internal operations of a convolutional neural network to emulate a specific type of signal processing known as envelope analysis. Envelope analysis demodulates vibrations to extract vibrational signatures associated with mechanical impact on a defective rolling component. An understanding of envelope analysis, along with knowledge of the geometries of machine components and operational speeds, allows for a physical interpretation of the signatures. The dissertation develops speed adaptive convolutional layers and a rotational speed estimation algorithm to identify defect signatures whose frequency components change as the speed changes. The characteristics that are captured by the developed convolutional neural network are verified through a feature selection process that is designed to filter out physically implausible features. Case studies on three different systems demonstrate the feasibility of using the developed convolutional neural network for the diagnosis.