Numerous physics-based simulation approaches have been proposed to generate realistic and vivid deformations for 3D models. These systems include the mass-spring system, the finite element approach, the thin-shell model, and others. However, previous systems based on analytic and numerical methods tend to be computationally intensive. Achieving an ideal balance between simulation accuracy and efficiency still poses several challenges.

In this dissertation, we present novel learning-based physics simulations and collision-handling algorithms that leverage the benefits of neural networks and optimization techniques. We use neural networks to compress high-dimensional 3D deformable models and accelerate the processing time. We also employ algorithms such as reinforcement learning, active learning, and imitation learning to capture complex physical behaviors that lack closed-form analytic models.

We propose multiple novel approaches for novel learning-based physics simulation. First, we train a learning-based collision detector for 3D deformable models and utilize the detector as a surrogate constraint in an optimization-based collision handler. Our focus is on collisions between topologically disjoint triangles in triangular meshes. Traditional geometric-based search methods for collision detection are computationally expensive, with costs ranging from $O(n\log n)$ to $O(n^2)$. In comparison, our neural collision detector is $80\times$ faster. To perform stable collision prediction performance in large and unseen spaces, we employ active learning by progressively incorporating new collision data based on network inferences, reaching a collision detection accuracy of up to $98.1%$. Second, we present an approach to accelerate collision response computations by incorporating an additional repulsive force unit in the learning-based pipeline. Our experiments demonstrate that backbone networks trained with the repulsive force unit can significantly decrease the number of collisions, boosting collision-free models from $49%$ to $77%$, while maintaining real-time performance, adding only $2$ milliseconds to the inference system. Third, we present a neural volumetric deformable object simulator with collision detection and handling based on an actor-critic neural architecture. Our critic network learns to estimate collision penetrations, while our actor network learns to minimize the penalty function through a series of gradient descent steps, resulting in nearly collision-free quasistatic deformable object poses. Finally, we introduce a novel framework for randomly reposing 3D humans to arbitrary poses based on a geometric optimization regularization that incorporates control information into diffusion-based inpainting. Our geometric inpainting algorithm reduces errors by $93%$ when moving different body parts to random locations.

In practice, our learning-based physics simulation systems can generate realistic 3D models that satisfy various constraints. We have tested our method using several large-scale datasets, including AMASS for humans and TailorNet for garments. Our approach can generate plausible results, and we observe $100-300\times$ speedups over numerical or analytic methods.