Theses and Dissertations from UMD
Permanent URI for this communityhttp://hdl.handle.net/1903/2
New submissions to the thesis/dissertation collections are added automatically as they are received from the Graduate School. Currently, the Graduate School deposits all theses and dissertations from a given semester after the official graduation date. This means that there may be up to a 4 month delay in the appearance of a give thesis/dissertation in DRUM
More information is available at Theses and Dissertations at University of Maryland Libraries.
Browse
2 results
Search Results
Item Multi-Agent Reinforcement Learning: Systems for Evaluation and Applications to Complex Systems(2023) Terry, Jordan; Dickerson, John; Computer Science; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)Reinforcement learning is a field of artificial intelligence that studies methods for agents to learn by trial and error to take actions in a given system. Famous examples of it have included learning to control real robots, or achieving superhuman performance in most of the most popular and challenging games for humans. In order to conduct research in this space, researchers use standardized "environments", such as robotics simulations or video games, to evaluate the performance of learning methods. This thesis covers PettingZoo, a library that offers a standardized API and set of reference environments for multi-agent reinforcement learning that's become widely used, SuperSuit, a library that offers a easy-to-use standardized preprocessing wrappers for interfacing with learning libraries, and extensions to the Arcade Learning Environment (a popular tool which reinforcement learning researchers use to interact with Atari 2600 games) that allows for supporting multiplayer game modes. Using these tools, this thesis also uses multi-agent reinforcement learning to develop a new tool for natural science research. Emergent behaviors refer to the coordinated behaviors of groups of agents such as pedestrians in a crosswalk, birds in flocking formations, cars in traffic or traders in the stock market, and represent some of the most important things that we generally don't understand across many fields of science. In this work, we introduce the first mathematical formalism for the systematic search of all possible good ("mature") emergent behaviors within a multi-agent system through multi-agent reinforcement learning (MARL), and create a naive implementation of this search via deep reinforcement learning that can be applied in arbitrary environments. We show that in 12 multi-agent systems, this naive method is able to find over a hundred total emergent behaviors, the majority of which were previously unknown to the environment authors. Such methods could allow for answering various types of open scientific questions, such as "What behaviors are possible in this system", "What specific conditions in this system allow for this kind of emergent behavior", or "How can I change this system to prevent this emergent behavior."Item CONTEXTUALIZATION OF THE E. COLI LSR SYSTEM: RELATIVE ORTHOLOGY, RELATIVE QS ACTIVITY, AND EMERGENT BEHAVIOR(2015) Quan, David Nathan; Bentley, William E.; Bioengineering; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)Within bacterial consortia there exist innumerable combinatorial circumstances, some of which may tip the scale toward pathogenicity, some of which may favor asymptomatic phenotypes. Indeed, the lines and intersections between commensal, pathogenic, and opportunistic bacteria are not always clean. As a foothold to mediate pathogenicity arising from consortia, many have puzzled at communication between bacteria. Primary among such considerations is quorum sensing (QS). Analogous to autocrine signaling in multicellular organisms, QS is a self-signaling process involving small molecules. Generally, QS activation is believed to have pleiotropic effects, and has been associated with numerous pathogenic phenotypes. The research herein focuses on autoinducer-2 (AI-2) based QS signaling transduced through the Lsr system. Produced by over 80 species of bacteria, AI-2 is believed to be an interspecies signaling molecule. Outside of the marine bacteria genera Vibrio and Marinomonas, the only known AI-2 based QS transduction pathway is the Lsr system. We sought to deepen the characterization of the Lsr system in contexts outside of the batch cultures in which it was originally defined. First, we interrogated E. coli K-12 W3110 Lsr system orthologs relative to the same strain's lac system. Both systems are induced by the molecule which they import and catabolize. We searched for homologs by focusing on the gene order along a genome, as gene arrangement can bear signaling consequences for autoregulatory circuits. We found that the Lsr system signal was phylogenetically dispersed if not particularly deep, especially outside of Enterobacteriales and Pasteurellaceaes, indicating that the system has generally been conferred horizontally. This contrasts with the lac system, whose signal is strong but limited to a select group of highly related enterobacteria. We then modeled the Lsr system with ODEs, revealing bimodality in silico, bolstering preliminary experimental evidence. This bifurcated expression was seen to depend upon nongenetic heterogeneity, which we modeled as a variation of a single compound parameter, basal, representing the basal rate of AI-2 flux into the cell through a low flux pathway. Moreover, in our finite difference-agent based models, bimodal expression could not arise from spatial stochasticity alone. This lies in contrast with the canonical LuxIR QS system, which employs an intercellular positive feedback loop to activate the entire population. We examined the consequences of this contrast, by modeling both systems under conditions of colony growth using finite difference-agent based methods. We additionally investigated the confluence of Lsr signaling with chemotactic sensitivity to AI-2, which has been demonstrated in E. coli. Finally, the consequences of bimodality in interspecies interactions were assessed by posing two populations containing different Lsr systems against each other. While few natural consortia consist of only two interacting bacteria, these studies indicate that AI-2 based Lsr signaling may mediate a multitude of transitional intraspecies and interspecies bacterial dynamics, the specifics of which will vary with the context and the homologs involved.