Determinism versus uncertainty: Examining the worst-case expected performance of data-driven policies

This paper explores binary decision making, a critical domain in areas such as finance and supply chain management, where decision makers must often choose between a deterministic-cost option and an uncertain-cost option. Given the limited historical data on the uncertain cost and its unknown probability distribution, this research aims to ascertain how decision makers can optimize their decisions. To this end, we evaluate the worst-case expected performance of all possible data-driven policies, including the sample average approximation policy, across four scenarios differentiated by the extent of knowledge regarding the lower and upper bounds of the first moment of the uncertain cost distribution. Our analysis, using worst-case expected absolute regret and worst-case expected relative regret metrics, consistently shows that no data-driven policy outperforms the straightforward strategy of choosing either a deterministic-cost or uncertain-cost option in these scenarios. Notably, the optimal choice between these two options depends on the specific lower and upper bounds of the first moment. Our research contributes to the literature by revealing the minimal worst-case expected performance of all possible data-driven policies for binary decision-making problems.