Differentiable Systems for Efficient Optimization and Learning: FROM KINEMATICS TO GEOMETRY

Abstract

Modern scientific and engineering challenges rooted in physics often require searching immense solution spaces. Sampling these high-dimensional spaces can be computationally prohibitive. I observe that many underlying dynamical processes are nearly everywhere differentiable: physical states evolve smoothly in time, and even brief discontinuities preserve overall continuity. When a system’s next state and its gradient with respect to the current state are available, those gradients act as beacons that steer optimization far more efficiently than gradient-free exploration. More broadly, gradients capture how changes in one set of parameters induce changes in system outputs, and thus provide a general mechanism for guiding optimization—even in systems that are not strictly continuous. In robotics, for example, differentiable physics allows us to train controllers in very high-dimensional action spaces and to learn policies that operate robustly in contact-rich environments, where gradients propagate through complex interactions between bodies. In 3D reconstruction, differentiable formulations enable efficient optimization over high-dimensional visual cues aggregated from many viewpoints, making it feasible to recover detailed 3D geometry from multi-view observations without resorting to exhaustive search. Consequently, researchers are investing heavily in differentiable models of real-world phenomena, especially within physical sciences and engineering, where the laws of motion and interaction naturally provide the required smoothness.

Building on these observations across natural and engineered systems, this thesis demonstrates how exploiting differentiability can systematically accelerate problem-solving in both decision making and geometric reconstruction. I first develop gradient-aware reinforcement learning methods that explicitly incorporate system-level derivatives into policy learning: one that augments Proximal Policy Optimization (PPO) with analytic gradients to constrain policy updates and speed convergence, and another that trains a continuous-time world model by randomly sampling time steps, greatly improving sample efficiency. Across these studies, a common insight emerges: exposing derivatives of a system’s dynamics turns exploration in high-dimensional control spaces into a far more efficient and directed optimization process. This perspective extends beyond continuous-time dynamics: even in systems with discrete structures or topological changes, useful gradients can be obtained by relaxing, approximating, or reparameterizing the underlying representations. Motivated by this insight, I then focus on implementing and strengthening differentiability in complex physical systems themselves, so that their internal gradients can be directly leveraged for downstream tasks. The first system is a GPU-accelerated traffic simulator with exact vehicle-kinematic derivatives, enabling rapid optimization of autonomous vehicle policies and traffic-signal timing in road networks. The second is a family of differentiable mesh formulations that make traditionally discrete triangle and tetrahedral meshes amenable to gradient-based optimization, including a probabilistic triangular-mesh representation that relaxes edge existence into continuous variables and a differentiable tetrahedral-mesh extraction pipeline for 3D object and mesh reconstruction. Although mesh connectivity and topology are inherently discrete, these methods reformulate geometric relationships so that gradients—interpreted as sensitivities between geometric parameters and observations—can be computed and exploited for efficient optimization. By weaving these case studies together, the thesis articulates a unified path toward simulators and learned agents that use physics-based gradients to reason more effectively about the real world, support scalable optimization for robotics and autonomous systems, and enable high-fidelity, differentiable 3D representations of our environment.