Optimal management of a limited number of replacements under Markovian deterioration

This article examines the problem of adaptively scheduling a limited number of identical replacements of a vital component, each of which degrades according to a discrete-time Markov chain. The component is vital in the sense that its failure results in irreparable system breakdown; therefore, the objective is to maximize the total expected lifetime of the system, though other performance measures are explored under the optimal policy. The problem is formulated as a discrete-time Markov decision process and its structural properties are examined. While the problem scenario is fairly simple, the proofs of intuitive structural properties via standard dynamic programming approaches are elusive. Instead, we demonstrate the usefulness of a sample path approach in a maintenance optimization context, and provide some counterexamples to other seemingly intuitive structural results.