Risk-Sensitive and Minimax Control of Discrete-Time, Finite-State Markov Decision Processes

Abstract

This paper analyzes a connection between risk-sensitive and minimaxcriteria for discrete-time, finite-states Markov Decision Processes(MDPs). We synthesize optimal policies with respect to both criteria,both for finite horizon and discounted infinite horizon problems. Ageneralized decision-making framework is introduced, which includes asspecial cases a number of approaches that have been considered in theliterature. The framework allows for discounted risk-sensitive andminimax formulations leading to stationary optimal policies on theinfinite horizon. We illustrate our results with a simple machinereplacement problem.