SOLVING THE DATA PROBLEM OF INVERSE RENDERING

Abstract

Intrinsic decomposition and inverse rendering for indoor scenes remain significant challenges in computer vision, primarily due to two distinct data gaps: evaluation data and training data. To bridge the evaluation data gap, this dissertation introduces the Measured Albedo in the Wild (MAW) dataset, comprising 888 images with physical albedo measurements, alongside complementary metrics for assessing albedo intensity, chromaticity, and texture beyond the traditional Weighted Human Disagreement Rate (WHDR).

To address the training data gap in challenging indoor scenes, the thesis introduces GaNI, a novel photometric stereo inverse rendering framework designed to effectively handle global illumination effects. GaNI employs a three-stage approach that facilitates the accurate capture and reconstruction of indoor scene properties, which enables collection of near ground-truth inverse rendering training data with photometric stereo.

Building upon GaNI, this dissertation further develops GLOW—a global-illumination-aware inverse-rendering system that performs joint optimization of geometry and material properties. GLOW also introduces a dynamic radiance cache, allowing more accurate modeling of light position dependent scene radiance. These improvements combined allow GLOW to further advances the quality of reconstructed material properties and geometry.

Together, MAW, GaNI, and GLOW form a complete data-centric pipeline that bridges the evaluation and training gaps in inverse rendering, paving the way for robust photorealistic modeling for virtual and augmented reality, robotics perception, and computational photography.