UNDERSTANDING FAILURE MODES OF DEEP LEARNING MODELS

Abstract

In the past few years, deep learning models have advanced significantly, achieving remarkable results in challenging problems across multiple domains such as vision and language. Despite their proficiency, these models exhibit numerous failure modes and intriguing behaviors that cannot be mitigated solely through scaling. A comprehensive understanding of these failure modes is imperative for their safe application and utilization.

The first part of the thesis focuses on memorization in neural networks as a failure mode. We begin by asking, ``Do diffusion models create unique works of art, or are they replicating content directly from their training sets?" We discuss our proposed framework for comparing generated images with training images and detecting content replication. While it is widely believed that duplicated images in the training set are responsible for content replication at inference time, we demonstrate that the model's text conditioning plays a similarly important role. Motivated by our findings, we propose several techniques to reduce data replication at both training and inference times. We then explore the question, ``Do diffusion models memorize style?" First, we train a feature extractor to capture the style elements in a given image. Then, we use this feature extractor to study how effectively recent generative models can replicate style in their generations.

The second part of the thesis focuses on the failure mode -- reproducibility in neural networks. We present visualization techniques for neural network decision boundaries to study reproducibility across different classification architectures and training methodologies. We show that architectural changes significantly alter decision boundaries, whereas similar architectures tend to yield similar decision regions, especially in wider models. Additionally, we also investigate the double descent phenomenon using these visualizations and propose a few hypotheses on how decision regions change with model scaling.