Physics
Permanent URI for this communityhttp://hdl.handle.net/1903/2269
Browse
7 results
Search Results
Item Analyzing the Dynamics of Biological and Artificial Neural Networks with Applications to Machine Learning(2024) Srinivasan, Keshav; Girvan, Michelle; Biophysics (BIPH); Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)The study of the brain has profoundly shaped the evolution of computational learning models and the history of neural networks. This journey began in the 1940s with Warren McCulloch and Walter Pitts’ groundbreaking work on the first mathematical model of a neuron, laying the foundation for artificial neural networks. The 1950s and 60s witnessed a significant milestone with Frank Rosenblatt’s development of the perceptron, showcasing the potential of neural networks for complex computational tasks. Since then, the field of neural networks has witnessed explosive growth, and terms like “Artificial Intelligence” and “Machine Learning” have become commonplace across diverse fields, including finance,medicine, and science. This dissertation explores the symbiotic parallels between neuroscience and machine learning, focusing on the dynamics of biological and artificial neural networks. We begin by examining artificial neural networks, particularly in predicting the dynamics of large, complex networks—a paradigm where traditional machine learning algorithms often struggle. To address this, we propose a novel approach utilizing a parallel architecture that mimics the network’s structure, achieving scalable and accurate predictions. Shifting our focus to biological neuronal networks, we delve into the theory of critical systems. This theory posits that the brain, when viewed as a complex dynamical system, operates near a critical point, a state ideal for efficient information processing. A key experimental observation of this type of criticality is neuronal avalanches—scale-free cascades of neuronal activity—which have been documented both in vitro (in neuronal cultures and acute brain slices) and in vivo (in the brains of awake animals). Recent advancements in experimental techniques, such as multi-photon imaging and genetically encoded fluorescent markers, allow for the measurement of activity in living organisms with unparalleled single-cell resolution. Despite these advances, significant challenges remain when only a fraction of neurons can be recorded with sufficient resolution, leading to inaccurate estimations of power-law relationships in size, duration, and scaling of neuronal avalanches. We demonstrate that by analyzing simulated critical neuronal networks alongside real 2-photon imaging data, temporal coarse-graining can recover the critical value of the mean size vs. duration scaling of neuronal avalanches, allowing for more accurate estimations of critical brain dynamics even from subsampled data. Finally, we bridge the gap between machine learning and neuroscience by exploring the concept of excitatory-inhibitory balance, a crucial feature of neuronal networks in the brain, within the framework of reservoir computing. We emphasize the stabilizing role of inhibition in reservoir computers (RCs), mirroring its function in the brain. We propose a novel inhibitory adaptation mechanism that allows RCs to autonomously adjust inhibitory connections to achieve a specific firing rate target, motivated by the firing rate homeostasis observed in biological neurons. Overall, this dissertation strives to deepen the ongoing collaboration between neuroscience and machine learning, fostering advancements that will benefit both fields.Item Combining Physics-based Modeling, Machine Learning, and Data Assimilation for Forecasting Large, Complex, Spatiotemporally Chaotic Systems(2023) Wikner, Alexander Paul; Ott, Edward; Physics; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)We consider the challenging problem of forecasting high-dimensional, spatiotemporally chaotic systems. We are primarily interested in the problem of forecasting the dynamics of the earth's atmosphere and oceans, where one seeks forecasts that (a) accurately reproduce the true system trajectory in the short-term, as desired in weather forecasting, and that (b) correctly capture the long-term ergodic properties of the true system, as desired in climate modeling. We aim to leverage two types of information in making our forecasts: incomplete scientific knowledge in the form of an imperfect forecast model, and past observations of the true system state that may be sparse and/or noisy. In this thesis, we ask if machine learning (ML) and data assimilation (DA) can be used to combine observational information with a physical knowledge-based forecast model to produce accurate short-term forecasts and consistent long-term climate dynamics. We first describe and demonstrate a technique called Combined Hybrid-Parallel Prediction (CHyPP) that combines a global knowledge-based model with a parallel ML architecture consisting of many reservoir computers and trained using complete observations of the system's past evolution. Using the Kuramoto-Sivashinsky equation as our test model, we demonstrate that this technique produces more accurate short-term forecasts than either the knowledge-based or the ML component model acting alone and is scalable to large spatial domains. We further demonstrate using the multi-scale Lorenz Model 3 that CHyPP can incorporate the effect of unresolved short-scale dynamics (subgrid-scale closure). We next demonstrate how DA, in the form of the Ensemble Transform Kalman Filter (ETKF), can be used to extend the Hybrid ML approach to the case where our system observations are sparse and noisy. Using a novel iterative scheme, we show that DA can be used to obtain training data for successive generations of hybrid ML models, improving the forecast accuracy and the estimate of the full system state over that obtained using the imperfect knowledge-based model. Finally, we explore the commonly used technique of adding observational noise to the ML model input during training to improve long-term stability and climate replication. We develop a novel training technique, Linearized Multi-Noise Training (LMNT), that approximates the effect of this noise addition. We demonstrate that reservoir computers trained with noise or LMNT regularization are stable and replicate the true system climate, while LMNT allows for greater ease of regularization parameter tuning when using reservoir computers.Item PROBING BIOPHYSICAL INTERACTIONS TO UNDERSTAND VIRAL DIFFUSION AND PARTICLE FATE IN THE AIRWAY MUCOSAL BARRIER(2023) Kaler, Logan; Duncan, Gregg A; Biophysics (BIPH); Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)The mucus barrier in the airway is the first line of defense against inhaled particulates and pathogens. Within the mucus barrier, large, heavily glycosylated gel-forming mucin proteins form a network to trap particles for removal. Influenza A virus (IAV) must first cross the mucus barrier before reaching the underlying airway epithelial cells to cause infection. On the IAV envelope, hemagglutinin (HA) binds sialic acid on the surface of the cell to initiate viral entry. However, HA preferentially binds sialic acid attached to galactose by either an ⍺2,3 or ⍺2,6 linkage. In addition to the cell surface, sialic acid is found on mucins and is thought to act as a decoy receptor to entrap the IAV within the mucus layer. However, neuraminidase (NA) on the envelope of IAV cleaves the bond between HA and sialic acid, releasing the virus. While the mechanism of IAV infection has been characterized, the interplay between mucus biophysical properties and the binding of IAV within the mucus network prior to infection requires further investigation. The overall objective of this dissertation is to understand how IAV moves through the mucosal barrier to subsequently cause infection. We hypothesize the structural features of the mucus gel network are responsible for the changes in IAV movement, rather than the binding and unbinding of the virus. To investigate this, we first analyzed the movement of IAV in ex vivo mucus from human endotracheal tubes. In order to further analyze this movement, we developed a novel analysis to calculate the dissociation constant of IAV-mucus binding in a 3D gel network environment. Using this data, we established a pipeline for estimating the passage of particles, including IAV, through the airway mucosal barrier. A machine learning-based trajectory analysis was employed to classify individual trajectories in order to calculate the percentage of particles able to cross the mucus barrier within a physiologically relevant time frame. Lastly, we investigated the effect of sialic acid binding preference on diffusion of IAV through mucus collected from different in vitro human airway epithelial cell cultures. The combined results of these studies confirmed our hypothesis that the mucus microstructure rather than the adhesive interactions of IAV to the mucins was responsible for the differences in IAV diffusion. This work provides further insight into role of the mucosal barrier in IAV infection and identifies the mucus gel network microstructure as a target for the development of therapeutics against IAV.Item DEVELOPING MACHINE LEARNING TECHNIQUES FOR NETWORK CONNECTIVITY INFERENCE FROM TIME-SERIES DATA(2022) Banerjee, Amitava; Ott, Edward; Physics; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)Inference of the connectivity structure of a network from the observed dynamics of the states of its nodes is a key issue in science, with wide-ranging applications such as determination of the synapses in nervous systems, mapping of interactions between genes and proteins in biochemical networks, distinguishing ecological relationships between different species in their habitats etc. In this thesis, we show that certain machine learning models, trained for the forecasting of experimental and synthetic time-series data from complex systems, can automatically learn the causal networks underlying such complex systems. Based on this observation, we develop new machine learning techniques for inference of causal interaction network connectivity structures underlying large, networked, noisy, complex dynamical systems, solely from the time-series of their nodal states. In particular, our approach is to first train a type of machine learning architecture, known as the ‘reservoir computer’, to mimic the measured dynamics of an unknown network. We then use the trained reservoir computer system as an in silico computational model of the unknown network to estimate how small changes in nodal states propagate in time across that network. Since small perturbations of network nodal states are expected to spread along the links of the network, the estimated propagation of nodal state perturbations reveal the connections of the unknown network. Our technique is noninvasive, but is motivated by the widely used invasive network inference method, whereby the temporal propagation of active perturbations applied to the network nodes are observed and employed to infer the network links (e.g., tracing the effects of knocking down multiple genes, one at a time, can be used infer gene regulatory networks). We discuss how we can further apply this methodology to infer causal network structures underlying different time-series datasets and compare the inferred network with the ground truth whenever available. We shall demonstrate three practical applications of this network inference procedure in (1) inference of network link strengths from time-series data of coupled, noisy Lorenz oscillators, (2) inference of time-delayed feedback couplings in opto-electronic oscillator circuit networks designed the laboratory, and, (3) inference of the synaptic network from publicly-available calcium fluorescence time-series data of C. elegans neurons. In all examples, we also explain how experimental factors like noise level, sampling time, and measurement duration systematically affect causal inference from experimental data. The results show that synchronization and strong correlation among the dynamics of different nodal states are, in general, detrimental for causal network inference. Features that break synchrony among the nodal states, e.g., coupling strength, network topology, dynamical noise, and heterogeneity of the parameters of individual nodes, help the network inference. In fact, we show in this thesis that, for parameter regimes where the network nodal states are not synchronized, we can often achieve perfect causal network inference from simulated and experimental time-series data, using machine learning techniques, in a wide variety of physical systems. In cases where effects like observational noise, large sampling time, or small sampling duration hinder such perfect network inference, we show that it is possible to utilize specially-designed surrogate time-series data for assigning statistical confidence to individual inferred network links. Given the general applicability of our machine learning methodology in time-series prediction and network inference, we anticipate that such techniques can be used for better model-building, forecasting, and control of complex systems in nature and in the lab.Item WAVE CHAOS IN MICROWAVE COUPLED ENCLOSURES, RESERVOIR COMPUTING, AND PHOTONIC TOPOLOGICAL INSULATOR GRAPHS: THEORY AND EXPERIMENT(2022) Ma, Shukai; Anlage, Steven M; Physics; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)Complex scattering exists in many diverse physical and real-life scenarios. Examples include reactions of atomic nuclei, transport through quantum dots, and the propagation of electromagnetic (EM) waves in over-moded resonant systems. These systems are Wave Chaotic, meaning that minute perturbations will lead to a drastic change in the wave properties of the system. The underlying chaotic property in the short wavelength limit makes deterministic modeling of wave properties vulnerable to small perturbations. Because of this, statistical methods play a central role in wave chaotic system studies. The Random Coupling Model (RCM) has been successfully applied to predict the statistics of single chaotic EM enclosures. We here expand RCM to systems consisting of multiple volumes that are coupled together, and do so with highly reduced computational complexity. Going beyond knowledge-based modeling, we employ machine-learning techniques to identify hidden information embedded in the scattering properties of wave chaotic systems. Reservoir computing (RC) is a genre of neural networks employed in machine learning studies. Its training is radically simplified (compared to a full back-propagation process in neural networks) because the input and reservoir layers remain unchangedduring the process. Recent work shows that RC can reasonably predict the future evolution of spatio-temporal chaotic systems. We aim to reverse the thinking: to emulate a software RC using the spatio-temporal chaotic wave fields in physical EM enclosures. A proof-of-principle hardware RC is demonstrated experimentally, and tested through a series of complex tasks carried out at ns-time scales. The concept of photonic topological insulator (PTI) is translated from the study of topological insulators (TIs) in condensed matter physics. For a TI material, the charge will only flow in topologically-protected states on the boundary surrounding the material,rather than in the bulk. We experimentally demonstrated a novel composite PTI system with quantum Hall (QH) and quantum spin Hall (QSH) topological phases. The TI effect also introduces a synthetic spin-1/2 degree-of-freedom to the guided waves. The Fermionic two-state property, absent in the bosonic photon world, is now accessible using the PTI system. Using this PTI system, I realize a chaotic graph system that falls in the Gaussian Symplectic Ensemble (GSE) universality class, which in principle only exists in the Fermionic world. We use simulations to show that GSE statistics will emerge in an appropriately designed PTI graph obeying anti-unitary symmetry.Item UNCOVERING PATTERNS IN COMPLEX DATA WITH RESERVOIR COMPUTING AND NETWORK ANALYTICS: A DYNAMICAL SYSTEMS APPROACH(2020) Krishnagopal, Sanjukta; Girvan, Michelle; Physics; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)In this thesis, we explore methods of uncovering underlying patterns in complex data, and making predictions, through machine learning and network science. With the availability of more data, machine learning for data analysis has advanced rapidly. However, there is a general lack of approaches that might allow us to 'open the black box'. In the machine learning part of this thesis, we primarily use an architecture called Reservoir Computing for time-series prediction and image classification, while exploring how information is encoded in the reservoir dynamics. First, we investigate the ways in which a Reservoir Computer (RC) learns concepts such as 'similar' and 'different', and relationships such as 'blurring', 'rotation' etc. between image pairs, and generalizes these concepts to different classes unseen during training. We observe that the high dimensional reservoir dynamics display different patterns for different relationships. This clustering allows RCs to perform significantly better in generalization with limited training compared with state-of-the-art pair-based convolutional/deep Siamese Neural Networks. Second, we demonstrate the utility of an RC in the separation of superimposed chaotic signals. We assume no knowledge of the dynamical equations that produce the signals, and require only that the training data consist of finite time samples of the component signals. We find that our method significantly outperforms the optimal linear solution to the separation problem, the Wiener filter. To understand how representations of signals are encoded in an RC during learning, we study its dynamical properties when trained to predict chaotic Lorenz signals. We do so by using a novel, mathematical fixed-point-finding technique called directional fibers. We find that, after training, the high dimensional RC dynamics includes fixed points that map to the known Lorenz fixed points, but the RC also has spurious fixed points, which are relevant to how its predictions break down. While machine learning is a useful data processing tool, its success often relies on a useful representation of the system's information. In contrast, systems with a large numbers of interacting components may be better analyzed by modeling them as networks. While numerous advances in network science have helped us analyze such systems, tools that identify properties on networks modeling multi-variate time-evolving data (such as disease data) are limited. We close this gap by introducing a novel data-driven, network-based Trajectory Profile Clustering (TPC) algorithm for 1) identification of disease subtypes and 2) early prediction of subtype/disease progression patterns. TPC identifies subtypes by clustering patients with similar disease trajectory profiles derived from bipartite patient-variable networks. Applying TPC to a Parkinson’s dataset, we identify 3 distinct subtypes. Additionally, we show that TPC predicts disease subtype 4 years in advance with 74% accuracy.Item DYNAMICS OF LARGE SYSTEMS OF NONLINEARLY EVOLVING UNITS(2017) Lu, Zhixin; Ott, Edward; Chemical Physics; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)The dynamics of large systems of many nonlinearly evolving units is a general research area that has great importance for many areas in science and technology, including biology, computation by artificial neural networks, statistical mechanics, flocking in animal groups, the dynamics of coupled neurons in the brain, and many others. While universal principles and techniques are largely lacking in this broad area of research, there is still one particular phenomenon that seems to be broadly applicable. In particular, this is the idea of emergence, by which is meant macroscopic behaviors that “emerge” from a large system of many “smaller or simpler entities such that ... large entities” [i.e., macroscopic behaviors] arise which “exhibit properties the smaller/simpler entities do not exhibit.” [1]. In this thesis we investigate mechanisms and manifestations of emergence in four dynamical systems consisting many nonlinearly evolving units. These four systems are as follows. (a) We first study the motion of a large ensemble of many noninteracting particles in a slowly changing Hamiltonian system that undergoes a separatrix crossing. In such systems, we find that separatrix-crossing induces a counterintuitive effect. Specifically, numerical simulation of two sets of densely sprinkled initial conditions on two energy curves appears to suggest that the two energy curves, one originally enclosing the other, seemingly interchange their positions. This, however, is topologically forbidden. We resolve this paradox by introducing a numerical simulation method we call “robust” and study its consequences. (b) We next study the collective dynamics of oscillatory pacemaker neurons in Suprachiasmatic Nucleus (SCN), which, through synchrony, govern the circadian rhythm of mammals. We start from a high-dimensional description of the many coupled oscillatory neuronal units within the SCN. This description is based on a forced Kuramoto model. We then reduce the system dimensionality by using the Ott Antonsen Ansatz and obtain a low-dimensional macroscopic description. Using this reduced macroscopic system, we explain the east-west asymmetry of jet-lag recovery and discus the consequences of our findings. (c) Thirdly, we study neuron firing in integrate-and-fire neural networks. We build a discrete-state/discrete-time model with both excitatory and inhibitory neurons and find a phase transition between avalanching dynamics and ceaseless firing dynamics. Power-law firing avalanche size/duration distributions are observed at critical parameter values. Furthermore, in this critical regime we find the same power law exponents as those observed from experiments and previous, more restricted, simulation studies. We also employ a mean-field method and show that inhibitory neurons in this system promote robustness of the criticality (i.e., an enhanced range of system parameter where power-law avalanche statistics applies). (d) Lastly, we study the dynamics of “reservoir computing networks” (RCN’s), which is a recurrent neural network (RNN) scheme for machine learning. The ad- vantage of RCN’s over traditional RNN’s is that the training is done only on the output layer, usually via a simple least-square method. We show that RCN’s are very effective for inferring unmeasured state variables of dynamical systems whose system state is only partially measured. Using the examples of the Lorenz system and the Rossler system we demonstrate the potential of an RCN to perform as an universal model-free “observer”.