Maximum Entropy Techniques

Finally, in this chapter the inference and decision strategies applied in stochastic optimization methods are considered in more detail: A large number of optimization problems arising in engineering, control, and economics can be described by the minimization of a certain (cost) function $$v=v(a,x)$$ v = v ( a , x ) depending on a random parameter vector $$a=a(\omega )$$ a = a ( ω ) and a decision vector $$x \in D$$ x ∈ D lying in a given set D of feasible decision, design or control variables. Hence, in order to get robust optimal decisions, i.e., optimal decisions being most insensitive with respect to variations of the random parameter vector $$a=a(\omega )$$ a = a ( ω ) , the original optimization problem is replaced by the deterministic substitute problem which consists in the minimization of the expected objective function $$\textbf{E}v=\textbf{E}v(a(\omega ),x)$$ E v = E v ( a ( ω ) , x ) subject to $$x \in D$$ x ∈ D . Since the true probability distribution $$\lambda $$ λ of $$a=a(\omega )$$ a = a ( ω ) is not exactly known in practice, one has to replace $$\lambda $$ λ by a certain estimate or guess $$\beta $$ β . Consequently, one has the following inference and decision problem: inference/estimation step Determine an estimation $$\beta $$ β of the true probability distribution $$\lambda $$ λ of $$a=a(\omega )$$ a = a ( ω ) , decision step Determine an optimal solution $$x^*$$ x ∗ of $$min \int v(a(\omega ),x) \beta (d\omega ) ~ \text{ s.t. } ~ x \in D $$ m i n ∫ v ( a ( ω ) , x ) β ( d ω ) s.t. x ∈ D . Computing approximation, estimation $$\beta $$ β of $$\lambda $$ λ , the criterion for judging an approximation $$\beta $$ β of $$\lambda $$ λ should be based on its utility for the decision-making process, i.e., one should weight the approximation error according to its influence on decision errors, and the decision errors should be weighted in turn according to the loss caused by an incorrect decision. Based on inferential ideas developed among others by Kerridge, Kullback, in this chapter generalized decision-oriented inaccuracy and divergence functions for probability distributions $$\lambda $$ λ , $$\beta $$ β are developed, taking into account that the outcome $$\beta $$ β of the inferential stage is used in a subsequent (ultimate) decision-making problem modeled by the above-mentioned stochastic optimization problem. In addition, stability properties of the inference and decision process $$\begin{aligned} \lambda \longrightarrow \beta \longrightarrow x \in D_{\epsilon }(\beta ) \end{aligned}$$ λ ⟶ β ⟶ x ∈ D ϵ ( β ) are studied, where $$D_{\epsilon }(\beta )$$ D ϵ ( β ) denotes the set of $$\epsilon -$$ ϵ - optimal decisions with respect to probability distribution $$P_{a(\cdot )}=\beta $$ P a ( · ) = β of the random parameter vector $$a=a(\omega )$$ a = a ( ω ) .