Stochastic Approximations for Finite-State Markov Chains.

dc.contributor.authorMa, Dye-Jyunen_US
dc.contributor.authorMakowski, Armand M.en_US
dc.contributor.authorShwartz, A.en_US
dc.contributor.departmentISRen_US
dc.date.accessioned2007-05-23T09:44:10Z
dc.date.available2007-05-23T09:44:10Z
dc.date.issued1989en_US
dc.description.abstractThis paper develops an a.s. convergence theory for a class of projected Stochastic Approximations driven by finite-state Markov chains. The conditions are mild and are given explicitly in terms of the model data, mainly the Lipschitz continuity of the one- step transition probabilities. The approach used here is a version of the ODE method as proposed by Metivier and Priouret. It combines the Kushner-Clark Lemma with properties of the Poisson equation associated with the underlying family of Markov chains. The class of algorithms studied here was motivated by implementation issues for constrained Markov decision problems, where the policies of interest often depend on quantities not readily available due either to insufflcient knowledge of the model parameters or to computational difficulties. This naturally leads to the on-line estimation (or computation) problem investigated here. Several examples from the area of queueing systems are discussed.en_US
dc.format.extent993575 bytes
dc.format.mimetypeapplication/pdf
dc.identifier.urihttp://hdl.handle.net/1903/4915
dc.language.isoen_USen_US
dc.relation.ispartofseriesISR; TR 1989-76en_US
dc.titleStochastic Approximations for Finite-State Markov Chains.en_US
dc.typeTechnical Reporten_US

Files

Original bundle
Now showing 1 - 1 of 1
Loading...
Thumbnail Image
Name:
TR_89-76.pdf
Size:
970.29 KB
Format:
Adobe Portable Document Format