To Pool or Not to Pool: Queueing Design for Large-Scale Service Systems

There are two basic queue structures commonly adopted in service systems: the pooled structure, where waiting customers are organized into a single queue served by a group of servers, and the dedicated structure, where each server has her own queue. Although the pooled structure, known to minimize the servers’ idle time, is widely used in large-scale service systems, this study reveals that the dedicated structure, along with the join-the-shortest-queue routing policy, could be more advantageous for improving certain performance measures, such as the probability of a customer’s waiting time being within a delay target. The servers’ additional idleness resulting from the dedicated structure will be negligible when the system scale is large. Using a fluid model substantiated by asymptotic analysis, we provide a performance comparison between the two structures for a moderately overloaded queueing system with customer abandonment. We intend to help service system designers answer the following question: To reach a specified service-level target, which queue structure will be more cost effective? Aside from structure design, our results are of practical value for performance analysis and staffing deployment.