Admission Control Policies in a Finite Capacity Geo/Geo/1 Queue Under Partial State Observations

We consider the problem of admission control in a discrete time Markovian queue with a finite capacity, a single server, and a geometric arrival and departure processes. We prove the threshold structure of the optimal admission policy under full information on the number of customers in the system. We also consider the admission control problem under partial state information, where the decision maker is only informed whether the system is empty, full, or in some intermediate state. We formulate this problem as a Markov Decision Process with the state representing the posterior distribution of the number of customers and apply a heuristic algorithm from the literature to approximate the optimal policy. In numerical experiments we demonstrate that the pair of the mean and variance of the posterior distribution may be effectively used instead of the full distribution, to implement the optimal policy. We also explore the behavior of the profit function and the value of information with respect to several system parameters.