Institute for Systems Research Technical Reports

Permanent URI for this collectionhttp://hdl.handle.net/1903/4376

This archive contains a collection of reports generated by the faculty and students of the Institute for Systems Research (ISR), a permanent, interdisciplinary research unit in the A. James Clark School of Engineering at the University of Maryland. ISR-based projects are conducted through partnerships with industry and government, bringing together faculty and students from multiple academic departments and colleges across the university.

Browse

Author: search.filters.author.Bhatnagar, Shalabh

Search Results

Now showing 1 - 4 of 4
  • Thumbnail Image
    Item
    Approximate Policy Iteration for Semiconductor Fab-Level Decision Making - a Case Study
    (2000) He, Ying; Bhatnagar, Shalabh; Fu, Michael C.; Marcus, Steven I.; Marcus, Steven I.; ISR
    In this paper, we propose an approximate policy iteration (API) algorithm for asemiconductor fab-level decision making problem. This problem is formulated as adiscounted cost Markov Decision Process (MDP), and we have applied exact policy iterationto solve a simple example in prior work. However, the overwhelmingcomputational requirements of exact policy iteration prevent its application forlarger problems. Approximate policy iteration overcomes this obstacle by approximating thecost-to-go using function approximation. Numerical simulation on the same example showsthat the proposed API algorithm leads to a policy with cost close to that of the optimalpolicy.
  • Thumbnail Image
    Item
    Randomized Difference Two-Timescale Simultaneous Perturbation Stochastic Approximation Algorithms for Simulation Optimization of Hidden Markov Models
    (2000) Bhatnagar, Shalabh; Fu, Michael C.; Marcus, Steven I.; Bhatnagar, Shashank; Marcus, Steven I.; Fu, Michael C.; ISR
    We proposetwo finite difference two-timescale simultaneous perturbationstochastic approximation (SPSA)algorithmsfor simulation optimization ofhidden Markov models. Stability and convergence of both thealgorithms is proved.

    Numericalexperiments on a queueing model with high-dimensional parameter vectorsdemonstrate orders of magnitude faster convergence using thesealgorithms over related $(N+1)$-Simulation finite difference analoguesand another two-simulation finite difference algorithm that updates incycles.

  • Thumbnail Image
    Item
    Optimal Multilevel Feedback Policies for ABR Flow Control using Two Timescale SPSA
    (1999) Bhatnagar, Shalabh; Fu, Michael C.; Marcus, Steven I.; ISR
    Optimal multilevel feedback control policies for rate based flow controlin available bit rate (ABR) service in asynchronous transfer mode (ATM)networks are obtained in the presence of information and propagationdelays, using a numerically efficient two timescale simultaneousperturbation stochastic approximation (SPSA) algorithm. Convergenceanalysis of the algorithm is presented. Numerical experiments demonstratefast convergence even in the presence of significant delays and largenumber of parametrized policy levels.
  • Thumbnail Image
    Item
    Optimal Multilevel Feedback Policies for ABR Flow Control using Two Timescale SPSA
    (1999) Bhatnagar, Shalabh; Fu, Michael C.; Marcus, Steven I.; ISR
    Optimal multilevel control policies for rate based flow control in available bit rate (ABR) service in asynchronous transfer mode (ATM) networks are obtained in the presence of information and propagation delays, using a numerically efficient two timescale simultaneous perturbation stochastic approximation (SPSA) algorithm. Numerical experiments demonstrate fast convergence even in the presence of significant delays and a large number of parametrized parameter levels.