IDEAS home Printed from https://ideas.repec.org/a/inm/ordeca/v7y2010i4p378-403.html
   My bibliography  Save this article

Paradoxes in Learning and the Marginal Value of Information

Author

Listed:
  • Peter I. Frazier

    (School of Operations Research and Information Engineering, Cornell University, Ithaca, New York 14853)

  • Warren B. Powell

    (Department of Operations Research and Financial Engineering, Princeton University, Princeton, New Jersey 08544)

Abstract

We consider the Bayesian ranking and selection problem, in which one wishes to allocate an information collection budget as efficiently as possible to choose the best among several alternatives. In this problem, the marginal value of information is not concave, leading to algorithmic difficulties and apparent paradoxes. Among these paradoxes is that when there are many identical alternatives, it is often better to ignore some completely and focus on a smaller number than it is to spread the measurement budget equally across all the alternatives. We analyze the consequences of this nonconcavity in several classes of ranking and selection problems, showing that the value of information is “eventually concave,” i.e., concave when the number of measurements of each alternative is large enough. We also present a new fully sequential measurement strategy that addresses the challenge that nonconcavity it presents.

Suggested Citation

  • Peter I. Frazier & Warren B. Powell, 2010. "Paradoxes in Learning and the Marginal Value of Information," Decision Analysis, INFORMS, vol. 7(4), pages 378-403, December.
    • Handle: RePEc:inm:ordeca:v:7:y:2010:i:4:p:378-403
    DOI: 10.1287/deca.1100.0190
    as

    Download full text from publisher

    File URL: http://dx.doi.org/10.1287/deca.1100.0190
    Download Restriction: no
    File URL: https://libkey.io/10.1287/deca.1100.0190?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. G. Tiao & B. Afonja, 1976. "Some bayesian considerations of the choice of design for ranking, selection and estimation," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 28(1), pages 167-185, December.
    2. Stigler, George J., 2011. "Economics of Information," Ekonomicheskaya Politika / Economic Policy, Russian Presidential Academy of National Economy and Public Administration, vol. 5, pages 35-49.
    3. L. P. Fatti & A. Mehrez & M. Pachter, 1987. "Bounds and properties of the expected value of sample information for a project‐selection problem," Naval Research Logistics (NRL), John Wiley & Sons, vol. 34(1), pages 141-150, February.
    4. Michel de Lara & L. Gilotte, 2007. "A tight sufficient condition for Radner-Stiglitz nonconcavity in the value of information," Post-Print hal-00716396, HAL.
    5. Stephen E. Chick & Noah Gans, 2009. "Economic Analysis of Simulation Selection Problems," Management Science, INFORMS, vol. 55(3), pages 421-437, March.
    6. Chade, Hector & Schlee, Edward, 2002. "Another Look at the Radner-Stiglitz Nonconcavity in the Value of Information," Journal of Economic Theory, Elsevier, vol. 107(2), pages 421-452, December.
    7. Fumie Yokota & Kimberly M. Thompson, 2004. "Value of Information Literature Analysis: A Review of Applications in Health Risk Management," Medical Decision Making, , vol. 24(3), pages 287-298, June.
    8. J. Eric Bickel & James E. Smith, 2006. "Optimal Sequential Exploration: A Binary Learning Model," Decision Analysis, INFORMS, vol. 3(1), pages 16-32, March.
    9. Jason R. W. Merrick, 2009. "Bayesian Simulation and Decision Analysis: An Expository Survey," Decision Analysis, INFORMS, vol. 6(4), pages 222-238, December.
    10. Samson, D. & Wirth, A. & Rickard, J., 1989. "The value of information from multiple sources of uncertainty in decision analysis," European Journal of Operational Research, Elsevier, vol. 39(3), pages 254-260, April.
    11. Jörgen Weibull & Lars-Göran Mattsson & Mark Voorneveld, 2007. "Better May be Worse: Some Monotonicity Results and Paradoxes in Discrete Choice Under Uncertainty," Theory and Decision, Springer, vol. 63(2), pages 121-151, September.
    12. De Lara, Michel & Gilotte, Laurent, 2007. "A tight sufficient condition for Radner-Stiglitz nonconcavity in the value of information," Journal of Economic Theory, Elsevier, vol. 137(1), pages 696-708, November.
    13. J. Eric Bickel, 2008. "The Relationship Between Perfect and Imperfect Information in a Two-Action Risk-Sensitive Problem," Decision Analysis, INFORMS, vol. 5(3), pages 116-128, September.
    14. Stephen E. Chick & Jürgen Branke & Christian Schmidt, 2010. "Sequential Sampling to Myopically Maximize the Expected Value of Information," INFORMS Journal on Computing, INFORMS, vol. 22(1), pages 71-80, February.
    15. Stephen E. Chick & Koichiro Inoue, 2001. "New Two-Stage and Sequential Procedures for Selecting the Best Simulated System," Operations Research, INFORMS, vol. 49(5), pages 732-743, October.
    16. Giuseppe Moscarini & Lones Smith, 2002. "The Law of Large Demand for Information," Econometrica, Econometric Society, vol. 70(6), pages 2351-2366, November.
    17. Jeffrey Keisler, 2004. "Value of Information in Portfolio Decision Analysis," Decision Analysis, INFORMS, vol. 1(3), pages 177-189, September.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Vilkkumaa, Eeva & Liesiö, Juuso & Salo, Ahti, 2014. "Optimal strategies for selecting project portfolios using uncertain value estimates," European Journal of Operational Research, Elsevier, vol. 233(3), pages 772-783.
    2. Hoffmann, Florian & Inderst, Roman & Turlo, Sergey, 2018. "Regulating Cancellation Rights with Consumer Experimentation," EconStor Preprints 253659, ZBW - Leibniz Information Centre for Economics.
    3. Francisco Santos-Arteaga & Debora Di Caprio & Madjid Tavana, 2014. "A Self-regulating Information Acquisition Algorithm for Preventing Choice Regret in Multi-perspective Decision Making," Business & Information Systems Engineering: The International Journal of WIRTSCHAFTSINFORMATIK, Springer;Gesellschaft für Informatik e.V. (GI), vol. 6(3), pages 165-175, June.
    4. Sushil Bikhchandani & John W. Mamer, 2013. "Decreasing Marginal Value of Information Under Symmetric Loss," Decision Analysis, INFORMS, vol. 10(3), pages 245-256, September.
    5. Powell, Warren B., 2019. "A unified framework for stochastic optimization," European Journal of Operational Research, Elsevier, vol. 275(3), pages 795-821.
    6. Stephen Chick & Martin Forster & Paolo Pertile, 2017. "A Bayesian decision theoretic model of sequential experimentation with delayed response," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 79(5), pages 1439-1462, November.
    7. Stephen E. Chick & Peter Frazier, 2012. "Sequential Sampling with Economics of Selection Procedures," Management Science, INFORMS, vol. 58(3), pages 550-569, March.
    8. Ilya O. Ryzhov, 2016. "On the Convergence Rates of Expected Improvement Methods," Operations Research, INFORMS, vol. 64(6), pages 1515-1528, December.
    9. L. Robin Keller & Ali Abbas & Manel Baucells & Vicki M. Bier & David Budescu & John C. Butler & Philippe Delquié & Jason R. W. Merrick & Ahti Salo & George Wu, 2010. "From the Editors..," Decision Analysis, INFORMS, vol. 7(4), pages 327-330, December.
      • L. Robin Keller & Manel Baucells & John C. Butler & Philippe Delquié & Jason R. W. Merrick & Gregory S. Parnell & Ahti Salo, 2008. "From the Editors..," Decision Analysis, INFORMS, vol. 5(4), pages 173-176, December.
      • L. Robin Keller & Manel Baucells & Kevin F. McCardle & Gregory S. Parnell & Ahti Salo, 2007. "From the Editors..," Decision Analysis, INFORMS, vol. 4(4), pages 173-175, December.
      • L. Robin Keller & Manel Baucells & John C. Butler & Philippe Delquié & Jason R. W. Merrick & Gregory S. Parnell & Ahti Salo, 2009. "From the Editors ..," Decision Analysis, INFORMS, vol. 6(4), pages 199-201, December.
    10. L. Robin Keller & Ali Abbas & J. Eric Bickel & Vicki M. Bier & David V. Budescu & John C. Butler & Philippe Delquié & Kenneth C. Lichtendahl & Jason R. W. Merrick & Ahti Salo & George Wu, 2011. "From the Editors ---Probability Scoring Rules, Ambiguity, Multiattribute Terrorist Utility, and Sensitivity Analysis," Decision Analysis, INFORMS, vol. 8(4), pages 251-255, December.
    11. Jing Xie & Peter I. Frazier, 2013. "Sequential Bayes-Optimal Policies for Multiple Comparisons with a Known Standard," Operations Research, INFORMS, vol. 61(5), pages 1174-1189, October.
    12. Daniel Russo & Benjamin Van Roy, 2018. "Learning to Optimize via Information-Directed Sampling," Operations Research, INFORMS, vol. 66(1), pages 230-252, January.
    13. Groves, Matthew & Branke, Juergen, 2019. "Top-κ selection with pairwise comparisons," European Journal of Operational Research, Elsevier, vol. 274(2), pages 615-626.
    14. Li Chen & Adam J.Mersereau & Zhe (Frank) Wang, 2017. "Optimal Merchandise Testing with Limited Inventory," Operations Research, INFORMS, vol. 65(4), pages 968-991, August.
    15. Rakesh K. Sarin, 2013. "From the Editor —Optimal Betting, Reducing Unnecessary Mammography in Breast Cancer Diagnosis, Product Line Design, and Value of Information," Decision Analysis, INFORMS, vol. 10(3), pages 187-188, September.
    16. Kjell Hausken & Jun Zhuang, 2011. "Governments' and Terrorists' Defense and Attack in a T -Period Game," Decision Analysis, INFORMS, vol. 8(1), pages 46-70, March.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. L. Robin Keller & Ali Abbas & Manel Baucells & Vicki M. Bier & David Budescu & John C. Butler & Philippe Delquié & Jason R. W. Merrick & Ahti Salo & George Wu, 2010. "From the Editors..," Decision Analysis, INFORMS, vol. 7(4), pages 327-330, December.
      • L. Robin Keller & Manel Baucells & Kevin F. McCardle & Gregory S. Parnell & Ahti Salo, 2007. "From the Editors..," Decision Analysis, INFORMS, vol. 4(4), pages 173-175, December.
      • L. Robin Keller & Manel Baucells & John C. Butler & Philippe Delquié & Jason R. W. Merrick & Gregory S. Parnell & Ahti Salo, 2008. "From the Editors..," Decision Analysis, INFORMS, vol. 5(4), pages 173-176, December.
      • L. Robin Keller & Manel Baucells & John C. Butler & Philippe Delquié & Jason R. W. Merrick & Gregory S. Parnell & Ahti Salo, 2009. "From the Editors ..," Decision Analysis, INFORMS, vol. 6(4), pages 199-201, December.
    2. van Staden, Heletjé E. & Boute, Robert N., 2021. "The effect of multi-sensor data on condition-based maintenance policies," European Journal of Operational Research, Elsevier, vol. 290(2), pages 585-600.
    3. Mark Whitmeyer, 2022. "Making Information More Valuable," Papers 2210.04418, arXiv.org, revised Jun 2024.
    4. Mark Whitmeyer, 2024. "Can One Hear the Shape of a Decision Problem?," Papers 2403.06344, arXiv.org, revised Apr 2024.
    5. Debarun Bhattacharjya & Jo Eidsvik & Tapan Mukerji, 2013. "The Value of Information in Portfolio Problems with Dependent Projects," Decision Analysis, INFORMS, vol. 10(4), pages 341-351, December.
    6. L. Robin Keller, 2008. "From the Editor..," Decision Analysis, INFORMS, vol. 5(3), pages 113-115, September.
    7. Stephen E. Chick & Peter Frazier, 2012. "Sequential Sampling with Economics of Selection Procedures," Management Science, INFORMS, vol. 58(3), pages 550-569, March.
    8. Yijie Peng & Chun-Hung Chen & Michael C. Fu & Jian-Qiang Hu, 2016. "Dynamic Sampling Allocation and Design Selection," INFORMS Journal on Computing, INFORMS, vol. 28(2), pages 195-208, May.
    9. Mark Whitmeyer, 2024. "Call the Dentist! A (Con-)Cavity in the Value of Information," Papers 2404.01190, arXiv.org.
    10. Hans Olav Vogt Myklebust & Jo Eidsvik & Iver Bakken Sperstad & Debarun Bhattacharjya, 2020. "Value of Information Analysis for Complex Simulator Models: Application to Wind Farm Maintenance," Decision Analysis, INFORMS, vol. 17(2), pages 134-153, June.
    11. Michel de Lara & Olivier Gossner, 2020. "Payoffs-Beliefs Duality and the Value of Information," Post-Print hal-01941006, HAL.
    12. Daniel Russo, 2020. "Simple Bayesian Algorithms for Best-Arm Identification," Operations Research, INFORMS, vol. 68(6), pages 1625-1647, November.
    13. Eric M. Schwartz & Eric T. Bradlow & Peter S. Fader, 2017. "Customer Acquisition via Display Advertising Using Multi-Armed Bandit Experiments," Marketing Science, INFORMS, vol. 36(4), pages 500-522, July.
    14. Niyazi Onur Bakır & Georgia-Ann Klutke, 2014. "Buying price of event information in two-action decision problems," Environment Systems and Decisions, Springer, vol. 34(1), pages 38-48, March.
    15. Vilkkumaa, Eeva & Liesiö, Juuso & Salo, Ahti, 2014. "Optimal strategies for selecting project portfolios using uncertain value estimates," European Journal of Operational Research, Elsevier, vol. 233(3), pages 772-783.
    16. Lindbeck, Assar & Weibull, Jörgen, 2020. "Delegation of investment decisions, and optimal remuneration of agents," European Economic Review, Elsevier, vol. 129(C).
    17. Stephen E. Chick & Jürgen Branke & Christian Schmidt, 2010. "Sequential Sampling to Myopically Maximize the Expected Value of Information," INFORMS Journal on Computing, INFORMS, vol. 22(1), pages 71-80, February.
    18. Chao Qin & Daniel Russo, 2024. "Optimizing Adaptive Experiments: A Unified Approach to Regret Minimization and Best-Arm Identification," Papers 2402.10592, arXiv.org, revised Jul 2024.
    19. Keppo, Jussi & Moscarini, Giuseppe & Smith, Lones, 2008. "The demand for information: More heat than light," Journal of Economic Theory, Elsevier, vol. 138(1), pages 21-50, January.
    20. Raluca M. Ursu & Qingliang Wang & Pradeep K. Chintagunta, 2020. "Search Duration," Marketing Science, INFORMS, vol. 39(5), pages 849-871, September.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:ordeca:v:7:y:2010:i:4:p:378-403. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.