Skip to content
University of Maryland LibrariesDigital Repository at the University of Maryland
    • Login
    View Item 
    •   DRUM
    • A. James Clark School of Engineering
    • Institute for Systems Research Technical Reports
    • View Item
    •   DRUM
    • A. James Clark School of Engineering
    • Institute for Systems Research Technical Reports
    • View Item
    JavaScript is disabled for your browser. Some features of this site may not work without it.

    On The Convergence and ODE Limit of A Two-Dimensional Stochastic Approximation

    Thumbnail
    View/Open
    TR_93-5.pdf (400.7Kb)
    No. of downloads: 483

    Date
    1993
    Author
    Ma, Dye-Jyun
    Makowski, Armand M.
    Metadata
    Show full item record
    Abstract
    We consider a two-dimensional stochastic approximations scheme of the Robbins-Monro type which naturally arises in the study of steering policies for Markov decision processes [6,7]. Making use of a decoupling change of variable, we establish almost sure convergence by ad-hoc arguments that combine standard results on one-dimensional stochastic approximations with a version of the law of large number for martingale differences. Coming full circle, this direct analysis gives clues on how to select the test function which appears in standard convergence results for multi-dimensional schemes. Furthermore, a blind application of the ODE method is not possible here as solutions to the limiting ODE cannot be defined in an elementary way, but the aforementioned change of variasble paves the way for an interpretation of the behavior of solutions to the limiting ODE.
    URI
    http://hdl.handle.net/1903/5352
    Collections
    • Institute for Systems Research Technical Reports

    DRUM is brought to you by the University of Maryland Libraries
    University of Maryland, College Park, MD 20742-7011 (301)314-1328.
    Please send us your comments.
    Web Accessibility
     

     

    Browse

    All of DRUMCommunities & CollectionsBy Issue DateAuthorsTitlesSubjectsThis CollectionBy Issue DateAuthorsTitlesSubjects

    My Account

    LoginRegister
    Pages
    About DRUMAbout Download Statistics

    DRUM is brought to you by the University of Maryland Libraries
    University of Maryland, College Park, MD 20742-7011 (301)314-1328.
    Please send us your comments.
    Web Accessibility