Bounding Functions of Markov processes and the Shortest Queue Problem.

Abstract

In this paper we prove a general lemma which can be used to show that the expectation of a nonnegative function of the state of a time-homogeneous Markov process is uniformly bounded in time. This is reminiscent of the classical theory of nonnegative supermartingales, except that our analog of the supermartingale inequality need not hold almost surely. Consequently, the lemma is suitable for establishing the stability of systems that evolve in a stabilizing mode in most states, though from certain states they may jump to a less stable state. We use this lemma to show that "joining the shortest queue" can bound the expected sum of the squares of the differences between all pairs among N queues, even under arbitrarily heavy traffic.