Optimality Results for a Simple Flow Control Problem.

Abstract

This paper presents a problem of optimal flow control for discrete-time M|M|l queues, where the decision-maker seeks to maximize the throughput subject to a bound on the average queue size. The problem is cast as a constrained Markov decision process and solved via Lagrangian arguments. The optimal strategy is shown to be a threshold policy which asturates the constraint. The method of analysis proceeds through the discounted version of the Lagrangian problems whose value functions are shown to be integer-concave. Dynamic Programming and stochastic comparison ideas constitute the main ingredients of the solution.