On The Convergence and ODE Limit of A Two-Dimensional Stochastic Approximation

Simple item page
dc.contributor.authorMa, Dye-Jyunen_US
dc.contributor.authorMakowski, Armand M.en_US
dc.contributor.departmentISRen_US
dc.date.accessioned2007-05-23T09:53:19Z
dc.date.available2007-05-23T09:53:19Z
dc.date.issued1993en_US
dc.description.abstractWe consider a two-dimensional stochastic approximations scheme of the Robbins-Monro type which naturally arises in the study of steering policies for Markov decision processes [6,7]. Making use of a decoupling change of variable, we establish almost sure convergence by ad-hoc arguments that combine standard results on one-dimensional stochastic approximations with a version of the law of large number for martingale differences. Coming full circle, this direct analysis gives clues on how to select the test function which appears in standard convergence results for multi-dimensional schemes. Furthermore, a blind application of the ODE method is not possible here as solutions to the limiting ODE cannot be defined in an elementary way, but the aforementioned change of variasble paves the way for an interpretation of the behavior of solutions to the limiting ODE.en_US
dc.format.extent410368 bytes
dc.format.mimetypeapplication/pdf
dc.identifier.urihttp://hdl.handle.net/1903/5352
dc.language.isoen_USen_US
dc.relation.ispartofseriesISR; TR 1993-5en_US
dc.subjectadaptive controlen_US
dc.subjectstochastic systemsen_US
dc.subjectCommunication en_US
dc.subjectSignal Processing Systemsen_US
dc.titleOn The Convergence and ODE Limit of A Two-Dimensional Stochastic Approximationen_US
dc.typeTechnical Reporten_US

Files

Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
TR_93-5.pdf
Size:
400.75 KB
Format:
Adobe Portable Document Format
Download

Collections

Institute for Systems Research Technical Reports