Optimal consumption and exploration: A case studyin piecewise‐deterministic Markov modelling

The subject of this paper is modelling and optimal control of consumption and explorationof non‐renewable resources via piecewise‐deterministic Markov processes (PDP). Ingeneral, one assumes that new reserves are found at random times T 1 , T 2 , T 3 ,..., where theinter‐arrival times are exponentially distributed. The amount of resource found at each T i is also a random variable with a given distribution. This process falls into the category ofPDP. In this paper, we clearly show how one can apply the PDP optimal control theory tothe optimal consumption and exploration of non‐renewable resources under uncertain exploration.The advantage of using a PDP model is that no matter how complicated thedynamics, utility function and distribution functions are, as long as they satisfy some mildassumptions, one can easily transform the problem into a deterministic optimal controlproblem, which can be solved iteratively with guaranteed convergence for the value function. Copyright Kluwer Academic Publishers 1999