A. James Clark School of Engineering
Permanent URI for this communityhttp://hdl.handle.net/1903/1654
The collections in this community comprise faculty research works, as well as graduate theses and dissertations.
Browse
129 results
Search Results
Item TOWARDS EFFICIENT OCEANIC ROBOT LEARNING WITH SIMULATION(2024) LIN, Xiaomin; Aloimonos, Yiannis; Electrical Engineering; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)In this dissertation, I explore the intersection of machine learning, perception, and simulation-based techniques to enhance the efficiency of underwater robotics, with a focus on oceanic tasks. My research begins with marine object detection using aerial imagery. From there, I address oyster detection using Oysternet, which leverages simulated data and Generative Adversarial Networks for sim-to-real transfer, significantly improving detection accuracy. Next, I present an oyster detection system that integrates diffusion-enhanced synthetic data with the Aqua2 biomimetic hexapedal robot, enabling real-time, on-edge detection in underwater environments. With detection models deployed locally, this system facilitates autonomous exploration. To enhance this capability, I introduce an underwater navigation framework that employs imitation learning, enabling the robot to efficiently navigate over objects of interest, such as rock and oyster reefs, without relying on localization. This approach improves information gathering while ensuring obstacle avoidance. Given that oyster habitats are often in shallow waters, I incorporate a deep learning model for real/virtual image segmentation, allowing the robot to differentiate between actual objects and water surface reflections, ensuring safe navigation. I expand on broader applications of these techniques, including olive detection for yield estimation and industrial object counting for warehouse management, using simulated imagery. In the final chapters, I address unresolved challenges, such as RGB/sonar data integration, and propose directions for future research to enhance underwater robotic learning through digital simulation further. Through these studies, I demonstrate how machine learning models and digital simulations can be used synergistically to address key challenges in underwater robotic tasks. Ultimately, this work advances the capabilities of autonomous systems to monitor and preserve marine ecosystems through efficient and robust digital simulation-based learning.Item Systematic Analysis of Adversaries' Exploitations of the End-host(2024) Avllazagaj, Erin; Dumitras, Tudor; Kwon, Yonghwi; Electrical Engineering; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)In the pipeline of a cyber attack, the malicious actor will first gain a foothold in the target system through a malware. The malware detection is still a challenging problem, as the malware authors are constantly evolving their techniques to evade detection. Therefore, it is important for us to understand why that is the case and what can the defenders do to improve the detection of the malware. In this thesis, I explore the behavior of the malware in the real users’ machines and how it changes across different executions. I show that the malware exhibits more variability than benign samples and that certain actions are often more prone to variability than others. This is the first study that quantitatively analyzes the behavior of the malware in the wildI leverage an observation from the first project, where variability in the malware samples happens due to running privilege escalation exploits. The variability in behavior is due to the fact that the malware sometimes runs in non-privileged mode and tries to run an exploit to escalate its privileges. For these reasons, I propose a new methodology to systematically discover sensitive memory corruption targets that cause privilege escalation. At last, I explore the sensitive memory corruption targets in the Linux kernel. Specifically, I propose a methodology to systematically discover sensitive fields in the Linux kernel that, when corrupted, lead the system into an exploitable state. This system, called SCAVY, is based on a novel definition of the exploitable state that allows the attacker to read and write into files and memory locations that they would normally. SCAVY explores the exploitable states based on the threat model of a local unprivileged attacker with the ability to issue system calls and with the capability to read/write into a limited location in the kernel memory. The framework revealed that there are 17 sensitive fields across 12 Linux kernel C structs that, when overwritten with the correct value, lead the system into an exploitable state. In this definition, unlike prior work, I consider the system to be in an exploitable state when the weird machine allows the attacker to read and/or write into files and memory locations that they would normally not be able to. This state can be used to write into sensitive files such as //etc//passwd where the exploit author can create a new root account on the vulnerable host and log in as that. Additionally, if the attacker can read unreadable files such as //etc//shadow they can leak passwords of root accounts, de-hash them and log in as the root account. I utilize these targets to develop 6 exploits for 5 CVE vulnerabilities. I also demonstrated the severity of these fields and the applicability of the exploitable state by exploiting CVE-2022-27666. I overwrote the f mapping pointer in struct file and caused a write into //etc//passwd. Unlike the original exploit, ours didn’t need to break KASLR, modify global variables or require support of FUSE-fs from the vulnerable host. This makes our methodology more extensible and more stable, since the exploit requires fewer corruption in the kernel memory and it doesn’t rely on the need to have the addresses of the kernel’s symbols for calculating the KASLR offset. Additionally, our exploit doesn’t modify global variables, which makes it more stable and less likely to crash the kernel, during its runtime. Our findings show that new memory corruption targets can change the security implications of vulnerabilities, urging researchers to proactively discover memory corruption targets.Item POLYMORPHIC CIRCUITS: THE IDENTIFICATION OF POSSIBLE SOURCES AND APPLICATIONS(2024) Dunlap, Timothy; Qu, Gang; Electrical Engineering; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)Polymorphic gates are gates whose function depends on some external or environmental conditions. While there has been research into both the creation and applications of polymorphic gates, much remains unknown. This dissertation, motivated by the recent security applications of polymorphic gates, seeks a systematic approach to generating polymorphic gates.Its contributions include a polymorphic interoperability framework, the first study on the source of polymorphism, time-based polymorphic gates, and polymorphism in the sub-threshold design. Polymorphic circuits are commonly created with evolutionary algorithms [3]. Because the evolutionary algorithm operates in ways that are not always obvious, precise mechanisms of polymorphism are not immediately clear in the resulting gates and has not been reported before. This dissertation, for the first time, identifies multiple structures that impact the polymorphic nature of the gates, which sheds light on how to create polymorphic gates. This discovery is based on a categorization methodology that evaluates the quality of polymorphic gates and finds the robust ones for further investigation of polymorphism. By combining the discovered structures with the evolutionary algorithm, high quality polymorphic gates can be generated faster as demonstrated in the subthreshold design domain. Time-based polymorphism was discovered during the time analysis of evolved polymorphic circuits while searching for the sources of polymorphism. This occurs when the function of the circuit depends on the sample rate of the circuit and is based on some input combinations not quickly reaching the output they move towards. Therefore, when the circuit is running at different clock frequency, it may exhibit different functionality. This is time-based polymorphism. As one application of polymorphic gates, this dissertation presents a framework that can enhance the fault coverage of any fault testing method by utilizing polymorphic gates. The proposed framework starts with any traditional fault testing approach, and when it becomes less effective in covering uncovered faults, it employs a gate replacement strategy to selectively replace certain standard logic gates by polymorphic gates of specific polymorphism. This concept is demonstrated in the dissertation with examples of a D flip-flop and the ISCAS85 C17 benchmark. This work has high practical value in subthreshold design where circuit manufacture defects increase significantly. In summary, this dissertation presents multiple contributions to the study of polymorphic circuits. It discovers multiple sources of polymorphism and how the results of an evolutionary algorithm can be filtered into higher quality solutions. It also examines time-based polymorphism as a new form of polymorphism with security applications. Finally, an enhancement to stuck-at fault testing using polymorphic gates is presented. This allows for easier testing of corner-cases that are hard to detect using traditional methodologies and holds promise for improving the reliability of testing, particularly in the subthreshold domain.Item SYNPLAY: IMPORTING REAL-WORLD DIVERSITY FOR A SYNTHETIC HUMAN DATASET(2024) Yim, Jinsub; Bhattacharyya, Shuvra S.; Electrical Engineering; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)In response to the growing demand for large-scale training data, synthetic datasets have emerged as practical solutions. However, existing synthetic datasets often fall short of replicating the richness and diversity of real-world data. Synthetic Playground (SynPlay) is introduced as a new synthetic human dataset that aims to bring out the diversity of human appearance in the real world. In this thesis, We focus on two factors to achieve a level of diversity that has not yet been seen in previous works: i) realistic human motions and poses and ii) multiple camera viewpoints towards human instances. We first use a game engine and its library-provided elementary motions to create games where virtual players can take less-constrained and natural movements while following the game rules (i.e., rule-guided motion design as opposed to detail-guided design). We then augment the elementary motions with real human motions captured with a motion capture device. To render various human appearances in the games from multiple viewpoints, we use seven virtual cameras encompassing the ground and aerial views, capturing abundant aerial-vs-ground and dynamic-vs-static attributes of the scene. Through extensive and carefully-designed experiments, we show that using SynPlay in model training leads to enhanced accuracy over existing synthetic datasets for human detection and segmentation. Moreover, the benefit of SynPlay becomes even greater for tasks in the data-scarce regime, such as few-shot and cross-domain learning tasks. These results clearly demonstrate that SynPlay can be used as an essential dataset with rich attributes of complex human appearances and poses suitable for model pretraining.Item Advances in Concrete Cryptanalysis of Lattice Problems and Interactive Signature Schemes(2024) Kippen, Hunter Michael; Dachman-Soled, Dana; Electrical Engineering; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)Advanced cryptography that goes beyond what is currently deployed to service our basic internet infrastructure is continuing to see widespread adoption. The enhanced functionality achieved by these schemes frequently yields an increase in complexity. Solely considering the asymptotic security of the underlying computational assumptions is often insufficient to realize practical and secure instantiations.In these cases, determining the risk of any particular deployment involves analyzing the concrete security (the exact length of time it would take to break the encryption) as well as quantifying how concrete security can degrade over time due to any exploitable information leakage. In this dissertation, we examine two such cryptographic primitives where assessing concrete security is paramount. First, we consider the cryptanalysis of lattice problems (used as the basis for current standard quantum resistant cryptosystems). We develop a novel side-channel attack on the FrodoKEM key encapsulation mechanism as submitted to the NIST Post Quantum Cryptography (PQC) standardization process. Our attack involves poisoning the FrodoKEM Key Generation (KeyGen) process using a security exploit in DRAM known as “Rowhammer”. Additionally, we revisit the security of the lattice problem known as Learning with Errors (LWE) in the presence of information leakage. We further enhance the robustness of prior methodology by viewing side information from a geometric perspective. Our approach provides the rigorous promise that, as hints are integrated, the correct solution is a (unique) lattice point contained in an ellipsoidal search space. Second, we study the concrete security of interactive signature schemes (used as part of many Privacy Enhancing Technologies). To this end, we complete a new analysis of the performance of Wagner’s k-list algorithm [CRYPTO ‘02], which has found significant utility in computing forgeries on several interactive signature schemes that implicitly rely on the hardness of the ROS problem formulated by Schnorr [ICICS ‘01].Item Graph-based Methods for Efficient, Interpretable and Reliable Machine Learning(2024) Ma, Yujunrong; Bhattacharyya, Shuvra SSB; Electrical Engineering; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)Machine learning algorithms have revolutionized fields such as computer vision, natural language processing, and speech recognition by offering the capability to analyze and extract information from vast datasets, a task far beyond human capacity. The deployment of these algorithms in high-stakes applications, including medical diagnosis, computational finance and criminal justice, underscores their growing importance. However, the decision-making processes of the so-called black-box models used in such areas raise considerable concerns. Therefore, enhancing the interpretability of these models is crucial, as it helps address issues like biases and inconsistencies in predictions, thereby making the models more comprehensible and trustworthy to end-users. Moreover, interpretability facilitates a deeper understanding of model behavior, such as the distribution of contributions across inputs. This deeper understanding can be applied to significantly improve efficiency. This is especially relevant as machine learning models find applications on edge devices, where computational resources are often limited. For such applications, significant improvements in energy efficiency and resource requirements can be obtained by optimizing and adapting model implementations based on an understanding of the models' internal behavior. However, such optimization introduces new challenges that arise due to factors such as complex, dynamically-determined dependency management among computations. This thesis presents five main contributions. The first contribution is the development of a novel type of interpretable machine learning model for applications in criminology and criminal justice (CCJ). The model involves graphical representations in the form of single decision trees, where the trees are constructed in an optimized fashion using a novel evolutionary algorithm. This approach not only enhances intrinsic interpretability but also enables users to understand the decision-making process more transparently, addressing the critical need for clarity in machine learning models' predictions. At the same time, the application of evolutionary algorithm methods enables such interpretability to be provided without significant degradation in model accuracy. In the second contribution, we develop new multi-objective evolutionary algorithm methods to find a balance between fairness and predictive accuracy in CCJ applications. We build upon the single-decision-tree framework developed in the first contribution of the thesis, and systematically integrate considerations of fairness and multi-objective optimization. In the third contribution, we develop new methods for crime forecasting applications. In particular, we develop new interpretable, attention-based methods using convolutional long short-term memory (ConvLSTM) models. These methods combine the power of ConvLSTM models in capturing spatio-temporal patterns with the interpretability of attention mechanisms. This combination of capabilities allows for the identification of key geographic areas in the input data that contribute to predictions from the model. The fourth contribution introduces a dynamic dataflow-graph-based framework to enhance the computational efficiency and run-time adaptability of inference processes, considering the constraints of available resources. Our proposed model maintains a high degree of analyzability while providing greater freedom than static dataflow models in being able to manipulate the computations associated with inference process at run-time. The fifth contribution of the thesis builds on insights developed in the fourth, and introduces a new parameterized design approach for image-based perception that enables efficient and dynamic reconfiguration of convolutions using channel attention. Compared to switching among sets of multiple complete neural network models, the proposed reconfiguration approach is much more streamlined in terms of resource requirements, while providing a high level of adaptability to handle unpredictable and dynamically-varying operational scenarios.Item Dynamic EM Ray Tracing for Complex Outdoor and Indoor Environments with Multiple Receivers(2024) Wang, Ruichen; Manocha, Dinesh; Electrical Engineering; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)Ray tracing models for visual, aural, and EM simulations have advanced, gaining traction in dynamic applications such as 5G, autonomous vehicles, and traffic systems. Dynamic ray tracing, modeling EM wave paths and their interactions with moving objects, leads to many challenges in complex urban areas due to environmental variability, data scarcity, and computational needs. In response to these challenges, we've developed new methods that use a dynamic coherence-based approach for ray tracing simulations across EM bands. Our approach is designed to enhance efficiency by improving the recomputation of bounding volume hierarchy (BVH) and by caching propagation paths. With our formulation, we've observed a reduction in computation time by about 30%, all while maintaining a level of accuracy comparable to that of other simulators. Building on our dynamic approach, we've made further refinements to our algorithm to better model channel coherence, spatial consistency, and the Doppler effect. Our EM ray tracing algorithm can incrementally improve the accuracy of predictions relating to the movement and positioning of dynamic objects in the simulation. We've also integrated the Uniform Geometrical Theory of Diffraction (UTD) with our ray tracing algorithm. Our enhancement is designed to allow for more accurate simulations of diffraction around smooth surfaces, especially in complex indoor settings, where accurate prediction is important. Taking another step forward, we've combined machine learning (ML) techniques with our dynamic ray tracing framework. Leveraging a modified conditional Generative Adversarial Network (cGAN) that incorporates encoded geometry and transmitter location, we demonstrate better efficiency and accuracy of simulations in various indoor environments with 5X speedup. Our method aims to not only improve the prediction of received power in complex layouts and reduce simulation times but also to lay a groundwork for future developments in EM simulation technologies, potentially including real-time applications in 6G networks. We evaluate the performance of our methods in various environments to highlight the advantages. In dynamic urban scenes, we demonstrate our algorithm’s scalability to vast areas and multiple receivers with maintained accuracy and efficiency compared to prior methods; for complex geometries and indoor environments, we compare the accuracy with analytical solutions as well as existing EM ray tracing systems.Item STUDYING PRODUCT REVIEWS USING SENTIMENT ANALYSIS BASED ON INTERPRETABLE MACHINE LEARNING(2023) Atrey, Pranjal; Dutta, Sanghamitra; Wu, Min; Electrical Engineering; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)Consumers’ reliance on product reviews and ratings has been making substantial impacts on purchasing behaviors in e-commerce. However, the relationship between reviews and ratings has received limited attention. For instance, a product may have a high rating but average reviews. Such feedback can cause confusion and uncertainty about the products, leading to decreased trust in the product. This thesis carries out a natural-language based machine learning study to analyze the relationship from e-commerce big data of product reviews and ratings. Towards answering this relationship question using natural-language-processing (NLP), we first employ data-driven sentiment analysis to obtain a numeric sentiment score from the reviews, which are then used for studying the correlation with actual ratings. For sentiment analysis, we consider the use of both glass-box (rule-based) and black-box opaque (BERT) models. We find that while the black-box model is more correlated with product ratings, there are interesting counterexamples where the sentiment analysis results by the glass-box model are better aligned with the rating. Next, we explore how well ratings can be predicted from the text reviews, and if sentiment scores can further help improve classification of reviews. We find that neither opaque nor glass- box classification models yield better accuracy, and classification accuracy mostly improves when BERT sentiment scores are augmented with reviews. Furthermore, to understand what different models use to predict ratings from reviews, we employ Local Interpretable Model- Agnostic Explanations (LIME) to explain the impact of words in reviews on the decisions of the classification models. Noting that different models can give similar predictions, which is a phenomenon known as the Rashomon Effect, our work provides insights on which words actually contribute to the decision-making of classification models, even in scenarios where an incorrect classification is made.Item Leveraging Deep Generative Models for Estimation and Recognition(2023) PNVR, Koutilya; Jacobs, David W.; Electrical Engineering; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)Generative models are a class of statistical models that estimate the joint probability distribution on a given observed variable and a target variable. In computer vision, generative models are typically used to model the joint probability distribution of a set of real image samples assumed to be on a complex high-dimensional image manifold. The recently proposed deep generative architectures such as Variational Autoencoders (VAEs), Generative Adversarial Networks (GANs), and diffusion models (DMs) were shown to generate photo-realistic images of human faces and other objects. These generative models also became popular for other generative tasks such as image editing, text-to-image, etc. As appealing as the perceptual quality of the generated images has become, the use of generative models for discriminative tasks such as visual recognition or geometry estimation has not been well studied. Moreover, with different kinds of powerful generative models getting popular lately, it's important to study their significance in other areas of computer vision. In this dissertation, we demonstrate the advantages of using generative models for applications that go beyond just photo-realistic image generation: Unsupervised Domain Adaptation (UDA) between synthetic and real datasets for geometry estimation; Text-based image segmentation for recognition. In the first half of the dissertation, we propose a novel generative-based UDA method for combining synthetic and real images when training networks to determine geometric information from a single image. Specifically, we use a GAN model to map both synthetic and real domains into a shared image space by translating just the domain-specific task-related information from respective domains. This is connected to a primary network for end-to-end training. Ideally, this results in images from two domains that present shared information to the primary network. Compared to previous approaches, we demonstrate an improved domain gap reduction and much better generalization between synthetic and real data for geometry estimation tasks such as monocular depth estimation and face normal estimation. In the second half of the dissertation, we showcase the power of a recent class of generative models for improving an important recognition task: text-based image segmentation. Specifically, large-scale pre-training tasks like image classification, captioning, or self-supervised techniques do not incentivize learning the semantic boundaries of objects. However, recent generative foundation models built using text-based latent diffusion techniques may learn semantic boundaries. This is because they must synthesize intricate details about all objects in an image based on a text description. Therefore, we present a technique for segmenting real and AI-generated images using latent diffusion models (LDMs) trained on internet-scale datasets. First, we show that the latent space of LDMs (z-space) is a better input representation compared to other feature representations like RGB images or CLIP encodings for text-based image segmentation. By training the segmentation models on the latent z-space, which creates a compressed representation across several domains like different forms of art, cartoons, illustrations, and photographs, we are also able to bridge the domain gap between real and AI-generated images. We show that the internal features of LDMs contain rich semantic information and present a technique in the form of LD-ZNet to further boost the performance of text-based segmentation. Overall, we show up to 6% improvement over standard baselines for text-to-image segmentation on natural images. For AI-generated imagery, we show close to 20% improvement compared to state-of-the-art techniques.Item Methods and Tools for Real-Time Neural Image Processing(2023) Xie, Jing; Bhattacharyya, Shuvra; Chen, Rong; Electrical Engineering; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)As a rapidly developing form of bioengineering technology, neuromodulationsystems involve extracting information from signals that are acquired from the brain and utilizing the information to stimulate brain activity. Neuromodulation has the potential to treat a wide range of neurological diseases and psychiatric conditions, as well as the potential to improve cognitive function. Neuromodulation integrates neural decoding and stimulation. As one of the twocore parts of neuromodulation systems, neural decoding subsystems interpret signals acquired through neuroimaging devices. Neuroimaging is a field of neuroscience that uses imaging techniques to study the structure and function of the brain and other central nervous system functions. Extracting information from neuroimaging signals, as is required in neural decoding, involves key challenges due to requirements of real-time, energy-efficient, and accurate processing and for large-scale, high resolution image data that are characteristic of neuromodulation systems. To address these challenges, we develop new methods and tools for design andimplementation of efficient neural image processing systems. Our contributions are organized along three complementary directions. First, we develop a prototype system for real-time neuron detection and activity extraction called the Neuron Detection and Signal Extraction Platform (NDSEP). This highly configurable system processes neural images from video streams in real-time or off-line, and applies techniques of dataflow modeling to enable extensibility and experimentation with a wide variety of image processing algorithms. Second,we develop a parameter optimization framework to tune the performance of neural image processing systems. This framework, referred to as the NEural DEcoding COnfiguration (NEDECO) package, automatically optimizes arbitrary collections of parameters in neural image processing systems under customizable constraints. The framework allows system designers to explore alternative neural image processing trade-offs involving execution time and accuracy. NEDECO is also optimized for efficient operation on multicore platforms, which allows for faster execution of the parameter optimization process. Third, we develop a neural network inference engine targeted to mobile devices.The framework can be applied to neural network implementation in many application areas, including neural image processing. The inference engine, called ShaderNN, is the first neural network inference engine that exploits both graphics-centric abstractions (fragment shaders) and compute-centric abstractions (compute shaders). The integration of fragment shaders and compute shaders makes improved use of the parallel computing advantages of GPUs on mobile devices. ShaderNN has favorable performance especially in parametrically small models.