Nonstationary stochastic gold‐mining: A time‐sequential tactical‐allocation problem

This paper presents an extension of gold‐mining problems formulated in earlier work by R. Bellman and J. Kadane. Bellman assumes there are two gold mines labeled A and B, respectively, each with a known initial amount of gold. There is one delicate gold‐mining machine which can be used to excavate one mine per day. Associated with mine A is a known constant return rate and a known constant probability of breakdown. There is also a return rate and probability of breakdown for mine B. Bellman solves the problem of finding a sequential decision procedure to maximize the expected amount of gold obtained before breakdown of the machine. Kadane extends the problem by assuming that there are several mines and that there are sequences of constants such that the jth constant for each mine represents the return rate for the jth excavation of that mine. He also assumes that the probability of breakdown during the jth excavation of a mine depends on j. We extend these results by assuming that the return rates are random variables with known joint distribution and by allowing the probability of breakdown to be a function of previous observations on the return rates. We show that under certain regularity conditions on the joint distributions of the random variables, the optimal policy is: at each stage always select a mine which has maximal conditional expected return per unit risk. This gold‐mining problem is also a formulation of the problem of time‐sequential tactical allocation of bombers to targets. Several examples illustrating these results are presented.