Distribution-Free Confidence Intervals for Conditional Probabilities and Ratios of Expectations

Many simulation experiments are concerned with the estimation of a ratio of two unknown means, the estimation of a conditional probability being an example. We propose confidence intervals for the case in which the ratio is estimated by using independent, identically distributed random pairs with bounded and ordered components. Emphasis is given to the case in which each component can be expressed as the product of a Bernoulli and a bounded random variable. The proposed intervals result from distribution-free bounds on error probabilities, are valid for every sample size, and their asymptotic width decreases at the same rate as that of confidence intervals based on the central limit theorem. We evaluate their performance by means of two experiments. The first considers the estimation of the probability that a path in a directed a network is shortest while the second considers the estimation of the distribution of the inventory level in a stationary inventory system with periodic review. The experiments show that the intervals are conservative with superior coverage for small sample sizes (\le 50).