On the Poisson Equation for Countable Markov Chains: Existence of Solutions and Parameter Dependence by Probabilistic Methods
| dc.contributor.author | Makowski, Armand M. | en_US |
| dc.contributor.author | Shwartz, A. | en_US |
| dc.contributor.department | ISR | en_US |
| dc.date.accessioned | 2007-05-23T09:57:23Z | |
| dc.date.available | 2007-05-23T09:57:23Z | |
| dc.date.issued | 1994 | en_US |
| dc.description.abstract | This paper considers the Poisson equation associated with time- homogeneous Markov chains on a countable state space. The discussion emphasizes probabilistic arguments and focuses on three separate issues, namely (i) the existence and uniqueness of solutions to the Poisson equation, (ii) growth estimates and bounds on these solutions and (iii) their parametric dependence. Answers to these questions are obtained under a variety of recurrence conditions.<P>Motivating applications can be found in the theory of Markov decision processes in both its adaptive and non-adaptive formulations, and in the theory of Stochastic Approximations. The results complement available results from Potential Theory for Markov chains, and are therefore of independent interest. | en_US |
| dc.format.extent | 1505151 bytes | |
| dc.format.mimetype | application/pdf | |
| dc.identifier.uri | http://hdl.handle.net/1903/5556 | |
| dc.language.iso | en_US | en_US |
| dc.relation.ispartofseries | ISR; TR 1994-87 | en_US |
| dc.subject | adaptive control | en_US |
| dc.subject | stochastic systems | en_US |
| dc.subject | markov | en_US |
| dc.subject | chains | en_US |
| dc.subject | poisson equation | en_US |
| dc.subject | smoothness of solutions | en_US |
| dc.subject | Intelligent Signal Processing | en_US |
| dc.subject | Communications Systems | en_US |
| dc.title | On the Poisson Equation for Countable Markov Chains: Existence of Solutions and Parameter Dependence by Probabilistic Methods | en_US |
| dc.type | Technical Report | en_US |
Files
Original bundle
1 - 1 of 1