A Sequential Decision Problem Based On The Rate Depending On A Markov Process

It is usual to grasp an activity of the public sector as a cycle of inputs, outputs and outcomes. The inputs are the resources, the outputs are the products achieved, and the outcome is the criterion to measure the results, but it is difficult to evaluate the outcome. A sequential expenditure problem on a Markov process will be considered, and a state of this process is closely related to the outcomes. This state can be changed by expending an additional amount, and it also changes according to a Markovian transition rule based on TP2. This stochastic order plays an important role in the Bayesian learning procedure for a partially observable Markov process. The dynamic programming formulation implies a recursive equation about the expected value obtainable under the optimal policy. There are some monotonic properties concerning this value and the optimal policy. Finally, we treat this problem on a partially observable Markov process with Bayesian learning after observing some properties under assumptions since the state can be changed by decisions. It is also possible to consider a monotonic property for this case.