Institute for Systems Research Technical Reports

Permanent URI for this collectionhttp://hdl.handle.net/1903/4376

This archive contains a collection of reports generated by the faculty and students of the Institute for Systems Research (ISR), a permanent, interdisciplinary research unit in the A. James Clark School of Engineering at the University of Maryland. ISR-based projects are conducted through partnerships with industry and government, bringing together faculty and students from multiple academic departments and colleges across the university.

Browse

Filters

19941

Settings

Author: search.filters.author.Shwartz, A.Subject: search.filters.subject.adaptive control

Search Results

Now showing 1 - 1 of 1
  • Thumbnail Image
    Item
    On the Poisson Equation for Countable Markov Chains: Existence of Solutions and Parameter Dependence by Probabilistic Methods
    (1994) Makowski, Armand M.; Shwartz, A.; ISR
    This paper considers the Poisson equation associated with time- homogeneous Markov chains on a countable state space. The discussion emphasizes probabilistic arguments and focuses on three separate issues, namely (i) the existence and uniqueness of solutions to the Poisson equation, (ii) growth estimates and bounds on these solutions and (iii) their parametric dependence. Answers to these questions are obtained under a variety of recurrence conditions.

    Motivating applications can be found in the theory of Markov decision processes in both its adaptive and non-adaptive formulations, and in the theory of Stochastic Approximations. The results complement available results from Potential Theory for Markov chains, and are therefore of independent interest.