Electrical & Computer Engineering Theses and Dissertations
Permanent URI for this collectionhttp://hdl.handle.net/1903/2765
Browse
2 results
Search Results
Item Understanding and Improving Reliability of Predictive and Generative Deep Learning Models(2024) Kattakinda, Priyatham; Feizi, Soheil; Electrical Engineering; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)Deep learning models are prone to acquiring spurious correlations and biases during training and adversarial attacks during inference. In the context of predictive models, this results in inaccurate predictions relying on spurious features. Our research delves into this phenomenon specifically concerning objects placed in uncommon settings, where they are not conventionally found in the real world (e.g., a plane on water or a television in a cave). We introduce the "FOCUS: Familiar Objects in Common and Uncommon Settings" dataset which aims to stress-test the generalization capabilities of deep image classifiers. By leveraging the power of modern search engines, we deliberately gather data containing objects in common and uncommon settings in a wide range of locations, weather conditions, and time of day. Our comprehensive analysis of popular image classifiers on the FOCUS dataset reveals a noticeable decline in performance when classifying images in atypical scenarios. FOCUS only consists of natural images which are extremely challenging to collect as by definition it is rare to find objects in unusual settings. To address this challenge, we introduce an alternative dataset named Diffusion Dreamed Distribution Shifts (D3S). D3S comprises synthetic images generated through StableDiffusion, utilizing text prompts and image guides derived from placing a sample foreground image onto a background template image. This scalable approach allows us to create 120,000 images featuring objects from all 1000 ImageNet classes set against 10 diverse backgrounds. Due to the incredible photorealism of the diffusion model, our images are much closer to natural images than previous synthetic datasets. To alleviate this problem, we propose two methods of learning richer and more robust image representations. In the first approach, we harness the foreground and background labels within D3S to learn a foreground (background)representation resistant to changes in background (foreground). This is achieved by penalizing the mutual information between the foreground (background) features and the background (foreground) labels. We demonstrate the efficacy of these representations by training classifiers on a task with strong spurious correlations. Thus far, our focus has centered on predictive models, scrutinizing the robustness of the learned object representations, particularly when the contextual surroundings are unconventional. In the second approach, we propose to use embeddings of objects and their relationships extracted using off-the-shelf image segmentation models and text encoders respectively as input tokens to a transformer. This leads to remarkably richer features that improve performance on downstream tasks such as image retrieval. Large language models are also prone to failures during inference. Given the widespread use of LLMs, understanding the propensity of these models to fail given adversarial inputs is crucial. To that end we propose a series of fast adversarial attacks called BEAST that uses beam search to add adversarial tokens to a given input prompt. These attacks induce hallucination, cause the models to jailbreak and facilitate unintended membership inference from model outputs. Our attacks are fast and are executable in relatively compute constrained environments.Item Optimization-based Robustness and Stabilization in Decentralized Control(2017) Alavian, Alborz; Rotkowitz, Michael C; Electrical Engineering; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)This dissertation pertains to the stabilization, robustness, and optimization of Finite Dimensional Linear Time Invariant (FDLTI) decentralized control systems. We study these concepts for FDLTI systems subject to decentralizations that emerge from imposing sparsity constraints on the controller. While these concepts are well-understood in absence of an information structure, they continue to raise fundamental interesting questions regarding an optimal controller, or on suitable notions of robustness in presence of information structures. Two notions of stabilizability with respect to decentralized controllers are considered. First, the seminal result of Wang & Davison in 1973 regarding internal stabilizability of perfectly decentralized system and its connection to the decentralized fixed-modes of the plant is revisited. This seminal result would be generalized to any arbitrary sparsity-induced information structure by providing an inductive proof that verifies and shows that those mode of the plant that are fixed with respect to the static controllers would remain fixed with respect to the dynamic ones. A constructive proof is also provided to show that one can move any non-fixed mode of the plant to any arbitrary location within desired accuracy provided that they remain symmetric in the complex plane. A synthesizing algorithm would then be derived from the inductive proof. A second stronger notion of stability referred to as "non-overshooting stability" is then addressed. A key property called "feedthrough consistency" is derived, that when satisfied, makes extension of the centralized results to the decentralized case possible. Synthesis of decentralized controllers to optimize an H_Infinity norm for model-matching problems is considered next. This model-matching problem corresponds to an infinite-dimensional convex optimization problem. We study a finite-dimensional parametrization, and show that once the poles are chosen for this parametrization, the remaining problem of coefficient optimization can be cast as a semidefinite program (SDP). We further demonstrate how to use first-order methods when the SDP is too large or when a first-order method is otherwise desired. This leaves the remaining choice of poles, for which we develop and discuss several methods to better select the most effective poles among many candidates, and to systematically improve their location using convex optimization techniques. Controllability of LTI systems with decentralized controllers is then studied. Whether an LTI system is controllable (by LTI controllers) with respect to a given information structure can be determined by testing for fixed modes, but this gives a binary answer with no information about robustness. Measures have already been developed to determine how far a system is from having a fixed mode when one considers complex or real perturbations to the state-space matrices. These measures involve intractable minimizations of a non-convex singular value over a power-set, and hence cannot be computed except for the smallest of the plants. We replace these problem by equivalent optimization problems that involve a binary vector rather than the power-set minimization and prove their equality. Approximate forms are also provided that would upper bound the original metrics, and enable us to utilize MINLP techniques to derive scalable upper bounds. We also show that we can formulate lower bounds for these measures as polynomial optimization problems,and then use sum-of-squares methods to obtain a sequence of SDPs, whose solutions would lower bound these metrics.