Technical Note—Markov Decision Processes with State-Information Lag

The Markov-decision-process formulation provides a method for selecting the optimal policy in a process where changes of state are Markovian, but assumes perfect information as to process state at each stage of the process. Where the available observations of the actual process state provide imperfect state information, the Markov-decision-process approach is applicable only if the observed state changes in a Markovian fashion. Although this is not true in the general case, it does apply in the important special case where information about the physical state becomes available after a delay of one transition or stage. This information-lag process can be analyzed as a Markov decision process. The degradation in gain, or expected return per unit time, from that of the perfect-information process provides a measure of the potential value of improving the information system.