Optimal Risk Sensitive Control of Semi-Markov Decision Processes

Simple item page
dc.contributor.advisorMarcus, Steven I.en_US
dc.contributor.advisorShayman, Mark A.en_US
dc.contributor.authorChawla, Jay P.en_US
dc.contributor.departmentISRen_US
dc.date.accessioned2007-05-23T10:09:43Z
dc.date.available2007-05-23T10:09:43Z
dc.date.issued2000en_US
dc.description.abstractIn this thesis, we study risk-sensitive cost minimization in semi-Markov decision processes. The main thrust of the thesis concerns the minimization of average risk sensitive costs over the infinite horizon. <p>Existing theory is expanded intwo directions: the semi-Markov case is considered, and non-irreduciblechains are considered. In particular, the analysis of the non-irreduciblecase is a significant addition to the literature, since many real-worldsystems do not exhibit irreducibility under all stationary Markov policies. Extension of existing results to the semi-Markovcase is significant because it requires the definition of a newdynamic programming equation and a technically challenging adaptation of the Perron-Frobeniuseigenvalue from the discrete time case.<p>In order to determine an optimal policy, new concepts in the classificationof Markov chains need to be introduced. This is because in thenon-irreducible case, the average risk sensitive cost objective function permits extremely unlikely events to exert a controlling influence on costs. We define equivalence classes of statescalled `strongly communicating classes' and formulate in terms of thema new characterization of the underlying structureof Markov Decision Problems and Markov chains.<p>In the risk sensitive case, the expected cost incurred prior to a stopping time with finite expected valuecan be infinite. For this reason, we introduce an assumption: reachability with finite cost. This is the fundamental assumptionrequired to achieve the major results of this thesis.<p>We explore existence conditions for an optimal policy, optimality equations,and behavior for large and small risk sensitivity parameter. (Onlynon-negative risk parameters are discussed in this thesis -- i.e. the risk averse and risk neutral cases, not the risk seeking case.) Ramificationsfor the risk neutral objective function are also analyzed.Furthermore, a simple solution technique we call `recursive computation'to find an optimal policy that isapplicable to small state spaces is described through examples.<p>The countable state space case is explored, and results that hold only for a finite state space are also presented. Other, relatedobjective functions such as sample path cost are analyzed and discussed.<p>We also explore finite time horizon semi-Markov problems, and present a general technique for solving them.We define a new objective function, the minimization of which is calledthe `deadline problem'. This is a problem in which the probability of reaching the goal state in a set period of time is maximized. We transform thedeadline problem objective function into an equivalent finite-horizonrisk sensitive objective function.en_US
dc.format.extent1134429 bytes
dc.format.mimetypeapplication/pdf
dc.identifier.urihttp://hdl.handle.net/1903/6150
dc.language.isoen_USen_US
dc.relation.ispartofseriesISR; PhD 2000-8en_US
dc.subjectqueueing networksen_US
dc.subjectoptimal controlen_US
dc.subjectdiscrete event dynamical systems DEDSen_US
dc.subjectrisk sensitiveen_US
dc.subjectsemi-Markoven_US
dc.subjectMDPen_US
dc.subjectMarkov decision processen_US
dc.subjectoptimal controlen_US
dc.subjectstochastic controlen_US
dc.subjectPerron-Frobeniusen_US
dc.subjectreducibleen_US
dc.subjectMarkoven_US
dc.subjectrealizationen_US
dc.subjectGlobal Communication Systemsen_US
dc.titleOptimal Risk Sensitive Control of Semi-Markov Decision Processesen_US
dc.typeDissertationen_US

Files

Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
PhD_2000-8.pdf
Size:
1.08 MB
Format:
Adobe Portable Document Format
Download

Collections

Institute for Systems Research Technical Reports