Efficient and Generalizable Machine Learning Models for Predicting Complex Dynamics

Abstract

Machine learning offers effective approaches to modeling dynamical systems solely from observed data. However, without explicit structural priors (built-in assumptions about the underlying dynamics) or additional contextual inputs, even modern high-capacity models that demonstrate impressive generalization typically require large and diverse training datasets, and may still struggle to generalize to aspects of the dynamics that are poorly represented in the training data.

In this dissertation, we first show that reservoir computing—a simple, efficient, and versatile framework for data-driven modeling of dynamical systems—can generalize to unexplored regions of state space without explicit structural priors. Using multistable dynamical systems as a test setting, we demonstrate that reservoir computers trained on trajectories from a single basin of attraction can achieve out-of-domain generalization by capturing system behavior in entirely unobserved basins.

We then consider settings in which the underlying dynamics (governing equations) also differ between the training and test data, a challenging scenario for models both with and without structural priors. We introduce Meta-learning for Tailored Forecasting using Related Time Series (METAFORS), which builds and initializes a model tailored to short time-series data from a target system by leveraging a library of models trained on longer time series from potentially related systems. Without requiring contextual labels, METAFORS reliably predicts both short-term evolution and long-term statistical properties, even when the target and related systems exhibit substantially different behaviors.

Finally, we turn to the problem of learning complex brain dynamics from noisy and dynamically diverse electroencephalography (EEG) recordings. We outline how reservoir computing’s efficiency and versatility may help to address the specific challenges EEG poses to data-driven modeling of dynamical systems. Then, using a configurable whole-brain neural mass model to enable controlled experiments, we show that predicting both the short-term evolution and long-term statistics of a brain-like dynamical system using a single reservoir computer requires much more careful hyperparameter tuning than in previous successful applications. Further, injecting noise into the autonomous reservoir system during the prediction stage can enable more robust replication of long-term statistical properties while still offering useful short-term predictions.