IDEAS home Printed from https://ideas.repec.org/p/ess/wpaper/id12569.html
   My bibliography  Save this paper

Learning When to Quit: An Empirical Model of Experimentation

Author

Listed:
  • Bernhard Ganglmair
  • Timothy Simcoe
  • Emanuele Tarantino

Abstract

The paper studies a dynamic model of the decision to continue or abandon a research project. Researchers improve their ideas over time and also learn whether those ideas will be adopted by the scientific community. Projects are abandoned as researchers grow more pessimistic about their chance of success. It estimates the structural parameters of this dynamic decision problem using a novel data set that contains information on both successful and abandoned projects submitted to the Internet Engineering Task Force (IETF), an organization that creates and maintains internet standards. Using the model and parameter estimates, it also simulates two counterfactual policies: a cost-subsidy and a prize-based incentive scheme. For a fixed budget, subsidies have a larger impact on research output, but prizes perform better when accounting for researchers’ opportunity costs.

Suggested Citation

  • Bernhard Ganglmair & Timothy Simcoe & Emanuele Tarantino, 2018. "Learning When to Quit: An Empirical Model of Experimentation," Working Papers id:12569, eSocialSciences.
  • Handle: RePEc:ess:wpaper:id:12569
    Note: Institutional Papers
    as

    Download full text from publisher

    File URL: http://www.esocialsciences.org/Download/repecDownload.aspx?fname=A20183615316_29.pdf&fcategory=Articles&AId=12569&fref=repec
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Jesús Fernández-Villaverde & Juan F. Rubio-Ramírez, 2007. "Estimating Macroeconomic Models: A Likelihood Approach," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 74(4), pages 1059-1087.
    2. Roberts, Kevin & Weitzman, Martin L, 1981. "Funding Criteria for Research, Development, and Exploration Projects," Econometrica, Econometric Society, vol. 49(5), pages 1261-1288, September.
    3. Nicholas Bloom & Luis Garicano & Raffaella Sadun & John Van Reenen, 2014. "The Distinct Effects of Information Technology and Communication Technology on Firm Organization," Management Science, INFORMS, vol. 60(12), pages 2859-2885, December.
    4. Ufuk Akcigit & Douglas Hanley & Nicolas Serrano-Velarde, 2021. "Back to Basics: Basic Research Spillovers, Innovation Policy, and Growth," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 88(1), pages 1-43.
    5. Timothy Simcoe, 2012. "Standard Setting Committees: Consensus Governance for Shared Technology Platforms," American Economic Review, American Economic Association, vol. 102(1), pages 305-336, February.
    6. Pierre Azoulay & Joshua S. Graff Zivin & Gustavo Manso, 2011. "Incentives and creativity: evidence from the academic life sciences," RAND Journal of Economics, RAND Corporation, vol. 42(3), pages 527-554, September.
    7. Thierry Magnac & David Thesmar, 2002. "Identifying Dynamic Discrete Decision Processes," Econometrica, Econometric Society, vol. 70(2), pages 801-816, March.
    8. Godfrey Keller & Sven Rady & Martin Cripps, 2005. "Strategic Experimentation with Exponential Bandits," Econometrica, Econometric Society, vol. 73(1), pages 39-68, January.
    9. Patrick Bolton & Christopher Harris, 1999. "Strategic Experimentation," Econometrica, Econometric Society, vol. 67(2), pages 349-374, March.
    10. Galasso, Alberto & Mitchell, Matthew & Virag, Gabor, 2016. "Market outcomes and dynamic patent buyouts," International Journal of Industrial Organization, Elsevier, vol. 48(C), pages 207-243.
    11. David Hirshleifer & Angie Low & Siew Hong Teoh, 2012. "Are Overconfident CEOs Better Innovators?," Journal of Finance, American Finance Association, vol. 67(4), pages 1457-1498, August.
    12. Tat Y. Chan & Barton H. Hamilton, 2006. "Learning, Private Information, and the Economic Evaluation of Randomized Experiments," Journal of Political Economy, University of Chicago Press, vol. 114(6), pages 997-1040, December.
    13. Hugo Hopenhayn & Gerard Llobet & Matthew Mitchell, 2006. "Rewarding Sequential Innovators: Prizes, Patents, and Buyouts," Journal of Political Economy, University of Chicago Press, vol. 114(6), pages 1041-1068, December.
    14. Alessandro Bonatti & Johannes Horner, 2011. "Collaborating," American Economic Review, American Economic Association, vol. 101(2), pages 632-663, April.
    15. Richard Ericson & Ariel Pakes, 1995. "Markov-Perfect Industry Dynamics: A Framework for Empirical Work," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 62(1), pages 53-82.
    16. Heidhues, Paul & Rady, Sven & Strack, Philipp, 2015. "Strategic experimentation with private payoffs," Journal of Economic Theory, Elsevier, vol. 159(PA), pages 531-551.
    17. Richard Gilbert & Carl Shapiro, 1990. "Optimal Patent Length and Breadth," RAND Journal of Economics, The RAND Corporation, vol. 21(1), pages 106-112, Spring.
    18. Stephan, Paula E., 2010. "The Economics of Science," Handbook of the Economics of Innovation, in: Bronwyn H. Hall & Nathan Rosenberg (ed.), Handbook of the Economics of Innovation, edition 1, volume 1, chapter 0, pages 217-273, Elsevier.
    19. Pakes, Ariel S, 1986. "Patents as Options: Some Estimates of the Value of Holding European Patent Stocks," Econometrica, Econometric Society, vol. 54(4), pages 755-784, July.
    20. Giuseppe Moscarini & Lones Smith, 2001. "The Optimal Level of Experimentation," Econometrica, Econometric Society, vol. 69(6), pages 1629-1644, November.
    21. Sabrina T. Howell, 2017. "Reducing Information Frictions in Venture Capital: The Role of New Venture Competitions," NBER Working Papers 23874, National Bureau of Economic Research, Inc.
    22. William R. Kerr & Ramana Nanda & Matthew Rhodes-Kropf, 2014. "Entrepreneurship as Experimentation," Journal of Economic Perspectives, American Economic Association, vol. 28(3), pages 25-48, Summer.
    23. Bernhard Ganglmair & Emanuele Tarantino, 2014. "Conversation with secrets," RAND Journal of Economics, RAND Corporation, vol. 45(2), pages 273-302, June.
    24. Victor Manuel Bennett & Jason Snyder, 2017. "The Empirics of Learning from Failure," Strategy Science, INFORMS, vol. 2(1), pages 1-12, March.
    25. Michael Roach & Wesley M. Cohen, 2013. "Lens or Prism? Patent Citations as a Measure of Knowledge Flows from Public Research," Management Science, INFORMS, vol. 59(2), pages 504-525, October.
    26. Weitzman, Martin L, 1979. "Optimal Search for the Best Alternative," Econometrica, Econometric Society, vol. 47(3), pages 641-654, May.
    27. Barton H. Hamilton & Jack A. Nickerson & Hideo Owan, 2003. "Team Incentives and Worker Heterogeneity: An Empirical Analysis of the Impact of Teams on Productivity and Participation," Journal of Political Economy, University of Chicago Press, vol. 111(3), pages 465-497, June.
    28. Andrew T. Ching & Tülin Erdem & Michael P. Keane, 2013. "Invited Paper ---Learning Models: An Assessment of Progress, Challenges, and New Developments," Marketing Science, INFORMS, vol. 32(6), pages 913-938, November.
    29. Tülin Erdem & Michael P. Keane, 1996. "Decision-Making Under Uncertainty: Capturing Dynamic Brand Choice Processes in Turbulent Consumer Goods Markets," Marketing Science, INFORMS, vol. 15(1), pages 1-20.
    30. Moscarini, Giuseppe & Squintani, Francesco, 2010. "Competitive experimentation with private information: The survivor's curse," Journal of Economic Theory, Elsevier, vol. 145(2), pages 639-660, March.
    31. Murray, Fiona & Stern, Scott & Campbell, Georgina & MacCormack, Alan, 2012. "Grand Innovation Prizes: A theoretical, normative, and empirical evaluation," Research Policy, Elsevier, vol. 41(10), pages 1779-1792.
    32. Gregory S. Crawford & Matthew Shum, 2005. "Uncertainty and Learning in Pharmaceutical Demand," Econometrica, Econometric Society, vol. 73(4), pages 1137-1173, July.
    33. Robert L. Bray & Decio Coviello & Andrea Ichino & Nicola Persico, 2016. "Multitasking, Multiarmed Bandits, and the Italian Judiciary," Manufacturing & Service Operations Management, INFORMS, vol. 18(4), pages 545-558, October.
    34. Ufuk Akcigit & Qingmin Liu, 2016. "The Role Of Information In Innovation And Competition," Journal of the European Economic Association, European Economic Association, vol. 14(4), pages 828-870, August.
    35. Glenn Ellison, 2002. "Evolving Standards for Academic Publishing: A q-r Theory," Journal of Political Economy, University of Chicago Press, vol. 110(5), pages 994-1034, October.
    36. Daniel P. Gross, 2017. "Performance feedback in competitive product development," RAND Journal of Economics, RAND Corporation, vol. 48(2), pages 438-466, May.
    37. Alberto Galasso & Timothy S. Simcoe, 2011. "CEO Overconfidence and Innovation," Management Science, INFORMS, vol. 57(8), pages 1469-1484, August.
    38. Andrew T. Ching & Tülin Erdem & Michael P. Keane, 2013. "Learning Models: An Assessment of Progress, Challenges and New Developments," Economics Papers 2013-W07, Economics Group, Nuffield College, University of Oxford.
    39. Berend Wierenga & Ralf van der Lans (ed.), 2017. "Handbook of Marketing Decision Models," International Series in Operations Research and Management Science, Springer, edition 2, number 978-3-319-56941-3, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Baron, Justus, 2020. "Counting standard contributions to measure the value of patent portfolios - A tale of apples and oranges," Telecommunications Policy, Elsevier, vol. 44(3).
    2. Alessandra Allocca, 2023. "“No Man is an Island”: An Empirical Study on Team Formation and Performance," Rationality and Competition Discussion Paper Series 389, CRC TRR 190 Rationality and Competition.
    3. Baron, Justus & Kanevskaia, Olia, 2023. "Wearing multiple hats—The role of working group chairs’ affiliation in standards development," Research Policy, Elsevier, vol. 52(9).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Hu, Yingyao, 2017. "The Econometrics of Unobservables -- Latent Variable and Measurement Error Models and Their Applications in Empirical Industrial Organization and Labor Economics [The Econometrics of Unobservables]," Economics Working Paper Archive 64578, The Johns Hopkins University,Department of Economics, revised 2021.
    2. Keller, Godfrey & Novák, Vladimír & Willems, Tim, 2019. "A note on optimal experimentation under risk aversion," Journal of Economic Theory, Elsevier, vol. 179(C), pages 476-487.
    3. Maloney,William F. & Zambrano,Andrés, 2021. "Learning to Learn : Experimentation, Entrepreneurial Capital, and Development," Policy Research Working Paper Series 9890, The World Bank.
    4. Jason R. Blevins & Ahmed Khwaja & Nathan Yang, 2018. "Firm Expansion, Size Spillovers, and Market Dominance in Retail Chain Dynamics," Management Science, INFORMS, vol. 64(9), pages 4070-4093.
    5. Kohei Kawaguchi, 2021. "When Will Workers Follow an Algorithm? A Field Experiment with a Retail Business," Management Science, INFORMS, vol. 67(3), pages 1670-1695, March.
    6. Guofang Huang & Hong Luo & Jing Xia, 2019. "Invest in Information or Wing It? A Model of Dynamic Pricing with Seller Learning," Management Science, INFORMS, vol. 65(12), pages 5556-5583, December.
    7. Sorensen, Morten, 2007. "Learning by Investing: Evidence from Venture Capital," SIFR Research Report Series 53, Institute for Financial Research.
    8. Sabrina T. Howell, 2017. "Reducing Information Frictions in Venture Capital: The Role of New Venture Competitions," NBER Working Papers 23874, National Bureau of Economic Research, Inc.
    9. A. Ronald Gallant & Han Hong & Ahmed Khwaja, 2018. "The Dynamic Spillovers of Entry: An Application to the Generic Drug Industry," Management Science, INFORMS, vol. 64(3), pages 1189-1211, March.
    10. Chunhua Wu & Hai Che & Tat Y. Chan & Xianghua Lu, 2015. "The Economic Value of Online Reviews," Marketing Science, INFORMS, vol. 34(5), pages 739-754, September.
    11. Hu, Yingyao & Shum, Matthew, 2012. "Nonparametric identification of dynamic models with unobserved state variables," Journal of Econometrics, Elsevier, vol. 171(1), pages 32-44.
    12. Gomes, Renato & Gottlieb, Daniel & Maestri, Lucas, 2016. "Experimentation and project selection: Screening and learning," Games and Economic Behavior, Elsevier, vol. 96(C), pages 145-169.
    13. S. Sriram & Pradeep K. Chintagunta & Puneet Manchanda, 2015. "Service Quality Variability and Termination Behavior," Management Science, INFORMS, vol. 61(11), pages 2739-2759, November.
    14. Vineet Kumar & Yacheng Sun, 2020. "Designing Pricing Strategy for Operational and Technological Transformation," Management Science, INFORMS, vol. 66(6), pages 2706-2734, June.
    15. Weng, Xi, 2015. "Dynamic pricing in the presence of individual learning," Journal of Economic Theory, Elsevier, vol. 155(C), pages 262-299.
    16. Jie Bai, 2016. "Melons as Lemons: Asymmetric Information, Consumer Learning and Seller Reputation," Natural Field Experiments 00540, The Field Experiments Website.
    17. Mark Schankerman & Florian Schuett, 2022. "Patent Screening, Innovation, and Welfare," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 89(4), pages 2101-2148.
    18. Ching, Andrew T. & Erdem, Tülin & Keane, Michael P., 2014. "A simple method to estimate the roles of learning, inventories and category consideration in consumer choice," Journal of choice modelling, Elsevier, vol. 13(C), pages 60-72.
    19. Asaf Cohen & Eilon Solan, 2013. "Bandit Problems with Lévy Processes," Mathematics of Operations Research, INFORMS, vol. 38(1), pages 92-107, February.
    20. Arthur Charpentier & Romuald Élie & Carl Remlinger, 2023. "Reinforcement Learning in Economics and Finance," Computational Economics, Springer;Society for Computational Economics, vol. 62(1), pages 425-462, June.

    More about this item

    Keywords

    eSS; learning; Internet Engineering Task Force (IETF); internet standard; parameters; counterfactual policies; cost-subsidy; prize-based incentive scheme; decision; research report; scientific community; novel data.;
    All these keywords.

    JEL classification:

    • D83 - Microeconomics - - Information, Knowledge, and Uncertainty - - - Search; Learning; Information and Knowledge; Communication; Belief; Unawareness
    • O31 - Economic Development, Innovation, Technological Change, and Growth - - Innovation; Research and Development; Technological Change; Intellectual Property Rights - - - Innovation and Invention: Processes and Incentives
    • O32 - Economic Development, Innovation, Technological Change, and Growth - - Innovation; Research and Development; Technological Change; Intellectual Property Rights - - - Management of Technological Innovation and R&D

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ess:wpaper:id:12569. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Padma Prakash (email available below). General contact details of provider: http://www.esocialsciences.org .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.