Physics
Permanent URI for this communityhttp://hdl.handle.net/1903/2269
Browse
8 results
Search Results
Item Combining Physics-based Modeling, Machine Learning, and Data Assimilation for Forecasting Large, Complex, Spatiotemporally Chaotic Systems(2023) Wikner, Alexander Paul; Ott, Edward; Physics; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)We consider the challenging problem of forecasting high-dimensional, spatiotemporally chaotic systems. We are primarily interested in the problem of forecasting the dynamics of the earth's atmosphere and oceans, where one seeks forecasts that (a) accurately reproduce the true system trajectory in the short-term, as desired in weather forecasting, and that (b) correctly capture the long-term ergodic properties of the true system, as desired in climate modeling. We aim to leverage two types of information in making our forecasts: incomplete scientific knowledge in the form of an imperfect forecast model, and past observations of the true system state that may be sparse and/or noisy. In this thesis, we ask if machine learning (ML) and data assimilation (DA) can be used to combine observational information with a physical knowledge-based forecast model to produce accurate short-term forecasts and consistent long-term climate dynamics. We first describe and demonstrate a technique called Combined Hybrid-Parallel Prediction (CHyPP) that combines a global knowledge-based model with a parallel ML architecture consisting of many reservoir computers and trained using complete observations of the system's past evolution. Using the Kuramoto-Sivashinsky equation as our test model, we demonstrate that this technique produces more accurate short-term forecasts than either the knowledge-based or the ML component model acting alone and is scalable to large spatial domains. We further demonstrate using the multi-scale Lorenz Model 3 that CHyPP can incorporate the effect of unresolved short-scale dynamics (subgrid-scale closure). We next demonstrate how DA, in the form of the Ensemble Transform Kalman Filter (ETKF), can be used to extend the Hybrid ML approach to the case where our system observations are sparse and noisy. Using a novel iterative scheme, we show that DA can be used to obtain training data for successive generations of hybrid ML models, improving the forecast accuracy and the estimate of the full system state over that obtained using the imperfect knowledge-based model. Finally, we explore the commonly used technique of adding observational noise to the ML model input during training to improve long-term stability and climate replication. We develop a novel training technique, Linearized Multi-Noise Training (LMNT), that approximates the effect of this noise addition. We demonstrate that reservoir computers trained with noise or LMNT regularization are stable and replicate the true system climate, while LMNT allows for greater ease of regularization parameter tuning when using reservoir computers.Item INFERENCE AND CONTROL IN NETWORKS FAR FROM EQUILIBRIUM.(2022) Sharma, Siddharth; Levy, Doron Prof.; Biophysics (BIPH); Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)This thesis focuses on two problems in biophysics.1. Inference in networks far from equilibrium. 2. Optimal transitions between network steady-states of unequal dimensions. The system used for development of the theory and design of computational algorithms is the fully connected and asymmetric version of the widely used Ising model. We begin with the basic concepts of biological networks and their emergence as an analytical paradigm over the last two decades due to advancements in high-throughput experimental methods. Biological systems are open and exchange both energy and matter with their environment. Their dynamics are far from equilibrium and don’t have well characterized steady-state distributions. This is in stark contrast to equilibrium dynamics with the Maxwell-Boltzmann distribution describing the histogram of microstates. The development of inference and control algorithms in this work is for nonequilibrium steady-states without detailed balance. Inferring the Ising model far from equilibrium requires solving the inverse problem in statistical mechanics. As opposed to using a known Hamiltonian to solve for the macroscopic averages, we calculate the couplings and fields, i.e., model parameters, given the microstates or stochastic snapshots as inputs. We first demonstrate a time-series calculation for the inverse problem and use Poisson and Polya-Gamma latent variables to construct a quadratic likelihood function which is then maximized using the expectation-maximization algorithm. In addition to the main calculation, properties of the Polya-Gamma variables are used to solve logistic regression on a Gaussian mixture. This has applications to problems like clustering and community detection. Not all available data in biology is time-ordered. In fact for some systems, e.g., gene-regulatory networks, most of the data is not in time-series. The solution to the inverse problem for such systems (data) is qualitatively different as it involves solving for the thermodynamic arrow of time. The present work uses the definition of a sufficient statistic based on equivalence classes to design a likelihood function through the disjoint cycles of the permutation group. The geometric intuition is provided using dihedral group of the same order. We state and prove that our likelihood function is minimally sufficient and present an optimization algorithm with computational results. The second problem, i.e., optimal network control is solved using optimal transport. We recognize that biological networks have the property to grow and shrink while remaining functional and robust. Recent works that have continued the progress made by earlier sem- inal results have concentrated on systems which do not undergo transitions that alter their dimensions. For example, a network increasing or decreasing its number of nodes. The connection between thermodynamics and optimal transport is well established through the Wasserstein metric being the minimal dissipation for stochastic dynamics. This result depends on narrow convergence which requires that the system size remains the same. Recently introduced Gromov-Wasserstein metric defined on a space of metric measure spaces, makes it possible to design optimal paths between probability distributions of different sizes. In context of networks, the GW metric can define geodesics between two network nonequilibrium steady-states with different number of vertices. The last two chapters discuss the mathematical concepts and results that are required to develop the GW metric on networks and the computational algorithms that follow as a result. We define the probability measures and loss functions as per the physical properties of the Ising model and demonstrate a geodesic calculation between two networks of different sizes.Item DEVELOPING MACHINE LEARNING TECHNIQUES FOR NETWORK CONNECTIVITY INFERENCE FROM TIME-SERIES DATA(2022) Banerjee, Amitava; Ott, Edward; Physics; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)Inference of the connectivity structure of a network from the observed dynamics of the states of its nodes is a key issue in science, with wide-ranging applications such as determination of the synapses in nervous systems, mapping of interactions between genes and proteins in biochemical networks, distinguishing ecological relationships between different species in their habitats etc. In this thesis, we show that certain machine learning models, trained for the forecasting of experimental and synthetic time-series data from complex systems, can automatically learn the causal networks underlying such complex systems. Based on this observation, we develop new machine learning techniques for inference of causal interaction network connectivity structures underlying large, networked, noisy, complex dynamical systems, solely from the time-series of their nodal states. In particular, our approach is to first train a type of machine learning architecture, known as the ‘reservoir computer’, to mimic the measured dynamics of an unknown network. We then use the trained reservoir computer system as an in silico computational model of the unknown network to estimate how small changes in nodal states propagate in time across that network. Since small perturbations of network nodal states are expected to spread along the links of the network, the estimated propagation of nodal state perturbations reveal the connections of the unknown network. Our technique is noninvasive, but is motivated by the widely used invasive network inference method, whereby the temporal propagation of active perturbations applied to the network nodes are observed and employed to infer the network links (e.g., tracing the effects of knocking down multiple genes, one at a time, can be used infer gene regulatory networks). We discuss how we can further apply this methodology to infer causal network structures underlying different time-series datasets and compare the inferred network with the ground truth whenever available. We shall demonstrate three practical applications of this network inference procedure in (1) inference of network link strengths from time-series data of coupled, noisy Lorenz oscillators, (2) inference of time-delayed feedback couplings in opto-electronic oscillator circuit networks designed the laboratory, and, (3) inference of the synaptic network from publicly-available calcium fluorescence time-series data of C. elegans neurons. In all examples, we also explain how experimental factors like noise level, sampling time, and measurement duration systematically affect causal inference from experimental data. The results show that synchronization and strong correlation among the dynamics of different nodal states are, in general, detrimental for causal network inference. Features that break synchrony among the nodal states, e.g., coupling strength, network topology, dynamical noise, and heterogeneity of the parameters of individual nodes, help the network inference. In fact, we show in this thesis that, for parameter regimes where the network nodal states are not synchronized, we can often achieve perfect causal network inference from simulated and experimental time-series data, using machine learning techniques, in a wide variety of physical systems. In cases where effects like observational noise, large sampling time, or small sampling duration hinder such perfect network inference, we show that it is possible to utilize specially-designed surrogate time-series data for assigning statistical confidence to individual inferred network links. Given the general applicability of our machine learning methodology in time-series prediction and network inference, we anticipate that such techniques can be used for better model-building, forecasting, and control of complex systems in nature and in the lab.Item Analysis of models of superfluidity(2022) Jayanti, Pranava Chaitanya; Trivisa, Konstantina; Physics; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)This thesis deals with the rigorous analysis of two models of superfluidity. One of them is a macro-scale description of the interacting dynamics of a mixture of superfluid Helium and normal Helium. The equations used are modifications of the incompressible Navier-Stokes equations in 2D, with a nonlinear \textit{mutual friction} that couples the two fluids. We show global well-posedness of strong solutions (with high-regularity data) to this model, by proving a Beale-Kato-Majda-type condition. This work has been published in the Journal of Nonlinear Science. \\ Next, we study a micro-scale model (the ``Pitaevskii’’ model) of superfluid-normal fluid interactions, derived by Lev Pitaevskii in 1959. This involves the nonlinear Schr\"odinger equation and incompressible inhomogeneous Navier-Stokes equations. Mass and momentum exchange between the two fluids is mediated through a nonlinear and bidirectional coupling. We establish the existence of local solutions (strong in wavefunction and velocity, weak in density) that satisfy an energy equality. The analysis of this model has been published in the Journal of Mathematical Fluid Mechanics. \\ Finally, we prove a weak-strong type uniqueness theorem for the solutions of the Pitaevskii model. We begin by arguing that the standard weak-strong uniqueness argument does not seem to work in the case of weak solutions whose regularity is governed purely by the energy balance equation, even if the strong solution is as smooth as one wishes. Thus, we are forced to consider slightly less weak solutions obtained from a higher-order energy bound. Owing to their better regularity, we can compare them to \textit{moderate} solutions $-$ which are rougher than conventional strong solutions used for this purpose $-$ and establish a \textit{weak-moderate uniqueness} theorem. Relative to the solutions actually constructed in the earlier part of this thesis, only some of the regularity properties are used, allowing room for improved existence theorems in the future, while maintaining compatible uniqueness results. The uniqueness results have been accepted for publication in Nonlinearity.Item SCALABLE MODELING APPROACHES IN SYSTEMS IMMUNOLOGY(2020) Park, Kyemyung; Levy, Doron; Tsang, John S; Biophysics (BIPH); Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)Systems biology seeks to build quantitative predictive models of biological system behavior. Biological systems, such as the mammalian immune system, operate across multiple spatiotemporal scales with a myriad of molecular and cellular players. Thus, mechanistic, predictive models describing such systems need to address this multiscale nature. A general outstanding problem is to cope with the high-dimensional parameter space arising when building reasonably detailed models. Another challenge is to devise integrated frameworks incorporating behavioral characteristics manifested at various organizational levels seamlessly. In this dissertation, I present two research projects addressing problems in immunological, or biological systems in general, using quantitative mechanistic models and machine learning, touching on the aforementioned challenges in scalable modeling. First, I aimed to understand how cell-to-cell heterogeneities are regulated through gene expression variations and their propagation at the single-cell level. To better understand detailed gene regulatory circuit models with many parameters without analytical solutions, I developed a framework called MAchine learning of Parameter-Phenotype Analysis (MAPPA). MAPPA combines machine learning approaches and stochastic simulation methods to dissect the mapping between high- dimensional parameters and phenotypes. MAPPA elucidated regulatory features of stochastic gene-gene correlation phenotypes. Next, I sought to quantitatively dissect immune homeostasis conferring tolerance to self-antigens and responsiveness to foreign antigens. Towards this goal, I built a series of models spanning from intracellular to organismal levels to describe the recurrent reciprocal relationships between self-reactive T cells and regulatory T cells in collaboration with an experimentalist. This effort elucidated critical immune parameters regulating the circuitry enabling the robust suppression of self-reactive T cells, followed by experimental validation. Moreover, by bridging these models across organizational scales, I derived a framework describing immune homeostasis as a dynamical equilibrium between self-activated T cells and regulatory T cells, typically operating well below thresholds that could result in clonal expansion and subsequent autoimmune diseases. I start with an introduction with a perspective linking seemingly contradictory behaviors of the immune system at different scales: microscopic “noise” and macroscopic deterministic outcomes. By connecting these aspects in the adaptive immune system analogously with an ansatz from statistical physics, I introduced a view on how robust immune homeostasis ensues.Item DYNAMICS OF LARGE SYSTEMS OF NONLINEARLY EVOLVING UNITS(2017) Lu, Zhixin; Ott, Edward; Chemical Physics; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)The dynamics of large systems of many nonlinearly evolving units is a general research area that has great importance for many areas in science and technology, including biology, computation by artificial neural networks, statistical mechanics, flocking in animal groups, the dynamics of coupled neurons in the brain, and many others. While universal principles and techniques are largely lacking in this broad area of research, there is still one particular phenomenon that seems to be broadly applicable. In particular, this is the idea of emergence, by which is meant macroscopic behaviors that “emerge” from a large system of many “smaller or simpler entities such that ... large entities” [i.e., macroscopic behaviors] arise which “exhibit properties the smaller/simpler entities do not exhibit.” [1]. In this thesis we investigate mechanisms and manifestations of emergence in four dynamical systems consisting many nonlinearly evolving units. These four systems are as follows. (a) We first study the motion of a large ensemble of many noninteracting particles in a slowly changing Hamiltonian system that undergoes a separatrix crossing. In such systems, we find that separatrix-crossing induces a counterintuitive effect. Specifically, numerical simulation of two sets of densely sprinkled initial conditions on two energy curves appears to suggest that the two energy curves, one originally enclosing the other, seemingly interchange their positions. This, however, is topologically forbidden. We resolve this paradox by introducing a numerical simulation method we call “robust” and study its consequences. (b) We next study the collective dynamics of oscillatory pacemaker neurons in Suprachiasmatic Nucleus (SCN), which, through synchrony, govern the circadian rhythm of mammals. We start from a high-dimensional description of the many coupled oscillatory neuronal units within the SCN. This description is based on a forced Kuramoto model. We then reduce the system dimensionality by using the Ott Antonsen Ansatz and obtain a low-dimensional macroscopic description. Using this reduced macroscopic system, we explain the east-west asymmetry of jet-lag recovery and discus the consequences of our findings. (c) Thirdly, we study neuron firing in integrate-and-fire neural networks. We build a discrete-state/discrete-time model with both excitatory and inhibitory neurons and find a phase transition between avalanching dynamics and ceaseless firing dynamics. Power-law firing avalanche size/duration distributions are observed at critical parameter values. Furthermore, in this critical regime we find the same power law exponents as those observed from experiments and previous, more restricted, simulation studies. We also employ a mean-field method and show that inhibitory neurons in this system promote robustness of the criticality (i.e., an enhanced range of system parameter where power-law avalanche statistics applies). (d) Lastly, we study the dynamics of “reservoir computing networks” (RCN’s), which is a recurrent neural network (RNN) scheme for machine learning. The ad- vantage of RCN’s over traditional RNN’s is that the training is done only on the output layer, usually via a simple least-square method. We show that RCN’s are very effective for inferring unmeasured state variables of dynamical systems whose system state is only partially measured. Using the examples of the Lorenz system and the Rossler system we demonstrate the potential of an RCN to perform as an universal model-free “observer”.Item Characterizing the Complex Spatial Patterns in Biological Systems(2015) Parker, Joshua; Losert, Wolfgang; Physics; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)Spatial point patterns are ubiquitous in natural systems, from the patterns of raindrops on a sidewalk to the organization of stars in a galaxy. In cell biology, these patterns can represent the locations of fluorescently-labeled molecules inside or on the surface of cells, or even represent the centers of the cells themselves. These patterns arise due to the signaling activity of the cells which are mediated by a broad range of chemicals, and understanding this activity is vital to investigating these complex systems. Luckily, though each pattern is unique, the statistical properties of the patterns embed information about the underlying pattern formation process. In this work, I demonstrate techniques to characterize the complex spatial patterns found in unicellular systems. Using topologically-derived measures, I demonstrated a technique to automatically classify sets of point patterns into groups to identify changes in higher order statistical moments due to experimental variation. This technique utilizes functional principal component analysis (FPCA) on the Minkowski functionals of a secondary pattern formed by imposing disks on each point center. I demonstrate that this better classifies a range of point pattern sets, and then applied this technique to pattern sets representing membrane-bound proteins in human immune cells, showing that this procedure correctly identifies non-interacting proteins. Further, I demonstrate a simulation-based technique to diminish the statistical impact of large-scale pattern features. In protein patterns, these represent the effects of membrane ruffling during pattern formation. These features dominate correlation measures, obscuring any hint of nanoscale clustering. Using heterogeneous Poisson null models for each cell to re-normalize their pairwise correlation functions, I found that patterns of LAT proteins ("linker for the activation of T-cells") do indeed cluster, with a characteristic length-scale of approximately 500 nm. By performing clustering analysis at this length scale on both the LAT patterns and their respective null models, I found that clusters are most commonly dimers, but that this clustering is strongly diminished upon T-cell activation. This loss of clustering may be due to the presence of unlabeled molecules that have been recruited to the cell membrane to form complexes with LAT. I also investigate both molecular and cell-center patterns in Dictyostellium discoideum cells, which are a model organism for amoeboid motion and G-protein receptor-mediated chemotaxis. These cells migrate using "autocrine" signal relay in that they both secrete and sense the same chemoattractant, cyclic adenosine monophosphate (cyclic AMP or cAMP). They also secrete phosphodiesterases that degrade the chemoattractant. This leads to streaming patterns of cells towards aggregation centers, which serve as sites of sporulation. To study these cells, I demonstrate an image analysis technique that statistically infers the local population of fluorescently-labeled mRNA units in fluorescent images of self-aggregating cells. The images were of experiments where two particular mRNAs were labeled along with their respective proteins, the first being adenylyl cyclase A (ACA), a molecule involved in the production of cAMP. ACA itself has already been seen to accumulate at the back of migrating cells. The location of these molecules were compared to that of the locations of cyclic AMP receptor 1 (cAR1), which is the cell's mechanism for gradient sensing. Using my analysis technique, I found that statistically significant proportions of ACA mRNA preferentially locate towards the rear of migrating cells, an assymetry that was also found to identically correlate with the asymmetry of ACA itself. This asymmetry was not seen in cAR1 mRNA, which tends to distribute uniformly. Further, the asymmetry in ACA was most exaggerated in cells migrating at the rear of streams, with the approach to the local aggregate center diminishing leading to more uniformly distributed molecules. This may suggest that ACA is locally translated at the back of migrating cells, a result requiring further investigation. I then construct a computational migration model of D. discoideum chemotaxis and use it to investigate how the streaming phase is effected by cell-cell adhesion as well as by the global degradation of cAMP. To classify the dynamics of the model with respect to cell density and external chemical gradient, the two relevant phase variables, I develop an order parameter based on the fraction of broken cell-cell contacts over time. This parameter successfully classifies the dynamic steady states of the model (independent motion, streaming, and aggregation), outperforming the often used "chemotactic index". I found that the elimination of degradation strongly diminishes any presence of streaming, suggesting that chemical degradation is vital to stream formation. In contrast, the addition of cell-cell adhesion expanded the streaming phase, stabilizing streams that were formed initially through signal relay.Item Information flow in an atmospheric model and data assimilation(2011) Yoon, Young-noh; Ott, Edward; Physics; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)Weather forecasting consists of two processes, model integration and analysis (data assimilation). During the model integration, the state estimate produced by the analysis evolves to the next cycle time according to the atmospheric model to become the background estimate. The analysis then produces a new state estimate by combining the background state estimate with new observations, and the cycle repeats. In an ensemble Kalman filter, the probability distribution of the state estimate is represented by an ensemble of sample states, and the covariance matrix is calculated using the ensemble of sample states. We perform numerical experiments on toy atmospheric models introduced by Lorenz in 2005 to study the information flow in an atmospheric model in conjunction with ensemble Kalman filtering for data assimilation. This dissertation consists of two parts. The first part of this dissertation is about the propagation of information and the use of localization in ensemble Kalman filtering. If we can perform data assimilation locally by considering the observations and the state variables only near each grid point, then we can reduce the number of ensemble members necessary to cover the probability distribution of the state estimate, reducing the computational cost for the data assimilation and the model integration. Several localized versions of the ensemble Kalman filter have been proposed. Although tests applying such schemes have proven them to be extremely promising, a full basic understanding of the rationale and limitations of localization is currently lacking. We address these issues and elucidate the role played by chaotic wave dynamics in the propagation of information and the resulting impact on forecasts. The second part of this dissertation is about ensemble regional data assimilation using joint states. Assuming that we have a global model and a regional model of higher accuracy defined in a subregion inside the global region, we propose a data assimilation scheme that produces the analyses for the global and the regional model simultaneously, considering forecast information from both models. We show that our new data assimilation scheme produces better results both in the subregion and the global region than the data assimilation scheme that produces the analyses for the global and the regional model separately.