Risk-Sensitive Optimal Control of Hidden Markov Models: Structural Results

Simple item page
dc.contributor.authorFernandez-Gaucherand, Emmanuelen_US
dc.contributor.authorMarcus, Steven I.en_US
dc.contributor.departmentISRen_US
dc.date.accessioned2007-05-23T10:02:22Z
dc.date.available2007-05-23T10:02:22Z
dc.date.issued1996en_US
dc.description.abstractWe consider a risk-sensitive optimal control problem for hidden Markov models (HMM), i.e. controlled Markov chains where state information is only available to the controller via an output (message) process. Building upon recent results by Baras, James and Elliott, we report in this paper result of an investigation on the nature and structure of risk-sensitive controllers. The question we pose is: How does risk-sensitivity manifest itself in the structure of a controller? We present the dynamic programming equations for risk-sensitive control of HMMs and show a number of structural properties of the value function (e.g., concavity and piecewise linearity) and the optimal risk-sensitive controller, and compare these to the corresponding results for the risk- neutral case. Furthermore, we show that indeed the risk-sensitive controller and its corresponding information state converge to the known solutions for the risk-neutral situation, as the risk factor goes to zero. We also study the infinite and general risk aversion cases. In addition, we present a particular case study of a popular benchmark machine replacement problem.en_US
dc.format.extent233508 bytes
dc.format.mimetypeapplication/pdf
dc.identifier.urihttp://hdl.handle.net/1903/5794
dc.language.isoen_USen_US
dc.relation.ispartofseriesISR; TR 1996-79en_US
dc.subjectoptimal controlen_US
dc.subjectstochastic systemsen_US
dc.subjectdiscrete event dynamical systemsen_US
dc.subjectSystems Integration Methodologyen_US
dc.titleRisk-Sensitive Optimal Control of Hidden Markov Models: Structural Resultsen_US
dc.typeTechnical Reporten_US

Files

Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
TR_96-79.pdf
Size:
228.04 KB
Format:
Adobe Portable Document Format
Download

Collections

Institute for Systems Research Technical Reports