A Simple Problem of Flow Control II: Implementation of Threshold Policies via Stochastic Approximations.

Abstract

This paper considers a flow control model for discrete M|M|1 queues. The problem of implementing a given threshold policy via an adaptive policy is discussed in terms of an adaptive algorithm of the Stochastic Approximations type. Such an implementation problem for threshold policies typically arises when some of the model parameters are not exactly known. The proposed algorithm is extremely simple, easy to implement on-line, and requires no a priori knowledge of the actual values of the model parameters. Convergence of the algorithm and convergence of the long-run average cost under the adaptive policy are investigated. The obtained results apply easily to the optimal flow control studied in a companion paper [5].