EFFECTS OF DIVERSE INITIALIZATION ON BAYESIAN OPTIMIZERS

Simple item page
dc.contributor.advisorFuge, Mark Den_US
dc.contributor.authorKamrah, Eeshen_US
dc.contributor.departmentMechanical Engineeringen_US
dc.contributor.publisherDigital Repository at the University of Marylanden_US
dc.contributor.publisherUniversity of Maryland (College Park, Md.)en_US
dc.date.accessioned2023-10-13T05:32:01Z
dc.date.available2023-10-13T05:32:01Z
dc.date.issued2023en_US
dc.description.abstractDesign researchers have struggled to produce quantitative predictions for exactly why andwhen diversity might help or hinder design search efforts. This thesis addresses that problem by studying one ubiquitously used search strategy—Bayesian Optimization (BO)—on different ND test problems with modifiable convexity and difficulty. Specifically, we test how providing diverse versus non-diverse initial samples to BO affects its performance during search and introduce a fast ranked-DPP method for computing diverse sets, which we need to detect sets of highly diverse or non-diverse initial samples. We initially found, to our surprise, that diversity did not appear to affect BO, neither helping nor hurting the optimizer’s convergence. However, follow-on experiments illuminated a key trade-off. Non-diverse initial samples hastened posterior convergence for the underlying model hyper-parameters—a Model Building advantage. In contrast, diverse initial samples accelerated exploring the function itself—a Space Exploration advantage. Both advantages help BO, but in different ways, and the initial sample diversity directly modulates how BO trades those advantages. Indeed, we show that fixing the BO hyper-parameters removes the Model Building advantage, causing diverse initial samples to always outperform models trained with non-diverse samples. These findings shed light on why, at least for BO-type optimizers, the use of diversity has mixed effects and cautions against the ubiquitous use of space-filling initializations in BO. To the extent that humans use explore-exploit search strategies similar to BO, our results provide a testable conjecture for why and when diversity may affect human-subject or design team experiments. The thesis is organized as follows: Chapter 2 provides an overview of existing studies that explore the impact of different initial stimuli. In Chapter 3, we explain the methodology used in the subsequent experiments. Chapter 4 presents the results of our initial study on the diverse initialization of BO (Bayesian Optimization) applied to the wildcat wells function. In this chapter we also investigate the conditions under which less diverse initial examples perform better and expand on these findings in Chapter 5 by considering additional ND continuous functions. The final chapter discusses the limitations of our findings and proposes potential areas for future research.en_US
dc.identifierhttps://doi.org/10.13016/dspace/180q-filh
dc.identifier.urihttp://hdl.handle.net/1903/30992
dc.language.isoenen_US
dc.subject.pqcontrolledMechanical engineeringen_US
dc.subject.pqcontrolledDesignen_US
dc.subject.pqcontrolledCognitive psychologyen_US
dc.subject.pquncontrolledBayesian optimizationen_US
dc.subject.pquncontrolledDiverseen_US
dc.subject.pquncontrolledexploration advantageen_US
dc.subject.pquncontrolledinitializationen_US
dc.subject.pquncontrolledlengthscaleen_US
dc.subject.pquncontrolledmodeling advantageen_US
dc.titleEFFECTS OF DIVERSE INITIALIZATION ON BAYESIAN OPTIMIZERSen_US
dc.typeThesisen_US

Files

Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
Kamrah_umd_0117N_23754.pdf
Size:
4.64 MB
Format:
Adobe Portable Document Format
Download

Collections

UMD Theses and Dissertations
Mechanical Engineering Theses and Dissertations