Learning low-dimensional structures in high dimensional scientific data

Abstract

This dissertation develops mathematical foundations for data-driven algorithms in three scientific domains: molecular dynamics, magnetic resonance relaxometry, and neural network weight spaces. A unifying theme is that high-dimensional scientific data possess hidden low-dimensional structure, and that principled algorithmic design requires rigorous understanding and exploitation of this structure. Chapter 2 derives sharp pointwise consistency estimates for the convergence of graph Laplacians in the target-measure diffusion map (TMDmap) algorithm and applies them to the setting of rare events in overdamped Langevin dynamics arising in molecular dynamics (MD) applications. The derived estimates provide bounds on the convergence of solutions to partial differential equations on manifolds and show that this convergence is, in particular, faster for the committor problem arising in transition path theory. Chapter 3 further explores the context of MD simulations by addressing the problem of collective variable discovery, with particular emphasis on dimensionality reduction that preserves dynamics. In particular, drawing from the quantitative coarse-graining theory of Legoll & Lelievre, we propose an algorithm for devising collective variables (CVs) using the so-called orthogonality condition. We validate our algorithm on reproducing transition rates for the n-butane molecule and on coarse-graining the alanine dipeptide. In this algorithm, we develop several new techniques for combining manifold learning algorithms such as diffusion maps, diffusion nets, and conformal autoencoders, and show how group-invariant featurization and functional independence can be crucial for separating timescales and resolving topological artifacts in diffusion map embeddings. Chapter 4 switches to the context of magnetic resonance imaging, where high-dimensional signals are characterized by a small number of parameters but are highly sensitive to measurement noise. Using input layer regularization proposed by Rozowski et al. (2022), we combine the empirical Bayes method with deep learning to provide an input preprocessing technique that optimally informs deep learning using a classical estimator. Our hybrid method improves on both pure deep learning and pure classical estimation and introduces a novel way to combine the generalized cross-validation estimator with deep learning for myelin water fraction estimation in in vivo brain data. Finally, in Chapter 5, we study the weights of fully connected trained neural networks, which exhibit low-dimensional structure after training. We provide convergence guarantees for neural network descrambling, an explainability algorithm that extracts this low-dimensional structure, showing that the singular value decomposition of network weights emerges as an interpretable factorization for deep learning. We show how singular vectors can be used to interpret networks in a variety of settings in mathematical signal processing, including deep electron-electron resonance, magnetic resonance, and phase retrieval.