Institute for Systems Research Technical Reports

Permanent URI for this collectionhttp://hdl.handle.net/1903/4376

This archive contains a collection of reports generated by the faculty and students of the Institute for Systems Research (ISR), a permanent, interdisciplinary research unit in the A. James Clark School of Engineering at the University of Maryland. ISR-based projects are conducted through partnerships with industry and government, bringing together faculty and students from multiple academic departments and colleges across the university.

Browse

Author: search.filters.author.Shwartz, Adam

Search Results

Now showing 1 - 2 of 2
  • Thumbnail Image
    Item
    Markov Decision Models with Weighted Discounted Criteria
    (1991) Feinberg, Eugene A.; Shwartz, Adam; ISR
    We consider a discrete time Markov Decision Process with infinite horizon. The criterion to be maximized is the sum of a number of standard discounted rewards, each with a different discount factor. Situations in which such criteria arise include modeling investments, modeling projects of different durations and systems with different time-scales, and some axiomatic formulations of multi-attribute preference theory. We show that for this criterion for some positive e there need not exist an e - optimal (randomized) stationary strategy, even when the state and action sets are finite. However, e - optimal Markov (non-randomized) strategies and optimal Markov strategies exist under weak conditions. We exhibit e - optimal Markov strategies which are stationary from some time onward. When both state and action spaces are finite, there exists an optimal Markov strategy with this property. We provide an explicit algorithm for the computation of such strategies.
  • Thumbnail Image
    Item
    A Braching Random Walk with a Barrier
    (1991) Biggins, J.D.; Lubachevsky, Boris D.; Shwartz, Adam; Weiss, Alan; ISR
    Suppose that a child is likely to be weaker than its parent, and child who is too weak will not reproduce. What is the condition for a family to survive? Let b denote the mean number of children a viable parent will have; we suppose that this is independent of strength of strength as long as strength is positive. Let F denote the distribution of the change in strength from parent to child, and define h = supq (- log U eqt dF(t))). We show that the situation is black or white: 1) If b < eh then P(family line dies) = 1, 2) If b > eh then P(family survives) > 0. Define f(x) := E(number of members in the family | initial strength x). We show that if b < eh, then there exists a positive constant C such that limx ƀ e-ax f(x) = C where a is the smaller of the (at most) two positive roots of b U est dF(t) = 1. We also find an explicit expression for f(x) when the walk is on a lattice and is skip-free to the left. This process arose in an analysis of rollback-based simulation, and these results are the foundation of that analysis.