Open Loop Feedback Policies for Markovian Queueing Systems

This paper recasts a recent approach to the control of Markovian systems proposed by Krishnan and Ott in the framework of open-loop feedback control policies and the value iteration algorithm. A simple recursion for the differential costs of a queue in isolation is obtained as a limit of the recursion for the discounted cost- to-go. This allows the treatment of general costs, state-dependent servers, and limited buffers. Several possible extensions of the approach are also discussed.