Risk-Sensitive, Minimax, and Mixed Risk-Neutral/Minimax Control of Markov Decision Processes

Abstract

This paper analyzes a connection between risk-sensitive and minimaxcriteria for discrete-time, finite-state Markov Decision Processes(MDPs). We synthesize optimal policies with respect to both criteria,both for finite horizon and discounted infinite horizon problems. Ageneralized decision-making framework is introduced, leading tostationary risk-sensitive and minimax optimal policies on theinfinite horizon with discounted costs.We introduce the mixed risk-neutral/minimaxobjective, and utilize results from risk-neutral and minimax controlto derive an information state process and dynamic programmingequations for the value function. We synthesize optimal control lawsboth on the finite and infinite horizon, and establish the effectivenessof the controller as a tool to trade off risk-neutral and minimaxobjectives.