Institute for Systems Research Technical Reports

Permanent URI for this collectionhttp://hdl.handle.net/1903/4376

This archive contains a collection of reports generated by the faculty and students of the Institute for Systems Research (ISR), a permanent, interdisciplinary research unit in the A. James Clark School of Engineering at the University of Maryland. ISR-based projects are conducted through partnerships with industry and government, bringing together faculty and students from multiple academic departments and colleges across the university.

Browse

Author: search.filters.author.Fernandez-Gaucherand, EmmanuelSubject: search.filters.subject.discrete event dynamical systems

Search Results

Now showing 1 - 2 of 2
  • Thumbnail Image
    Item
    Risk-Sensitive Optimal Control of Hidden Markov Models: Structural Results
    (1996) Fernandez-Gaucherand, Emmanuel; Marcus, Steven I.; ISR
    We consider a risk-sensitive optimal control problem for hidden Markov models (HMM), i.e. controlled Markov chains where state information is only available to the controller via an output (message) process. Building upon recent results by Baras, James and Elliott, we report in this paper result of an investigation on the nature and structure of risk-sensitive controllers. The question we pose is: How does risk-sensitivity manifest itself in the structure of a controller? We present the dynamic programming equations for risk-sensitive control of HMMs and show a number of structural properties of the value function (e.g., concavity and piecewise linearity) and the optimal risk-sensitive controller, and compare these to the corresponding results for the risk- neutral case. Furthermore, we show that indeed the risk-sensitive controller and its corresponding information state converge to the known solutions for the risk-neutral situation, as the risk factor goes to zero. We also study the infinite and general risk aversion cases. In addition, we present a particular case study of a popular benchmark machine replacement problem.
  • Thumbnail Image
    Item
    Risk-Sensitive Optimal Control of Hidden Markov Models: A Case Study
    (1994) Fernandez-Gaucherand, Emmanuel; Marcus, Steven I.; ISR
    We consider a risk-sensitive optimal control problem for hidden Markov models (HMM). Building upon recent results by Baras, James and Elliott, we investigate the structure of risk-sensitive controllers for HMM, via an examination of a popular benchmark problem. We obtain new results on the structure of the risk- sensitive controller by first proving concavity and piecewise linearity of the value function. Furthermore, we compare the structure of risk-sensitive and risk-neutral controllers.