IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2004.03865.html
   My bibliography  Save this paper

Manipulation-Proof Machine Learning

Author

Listed:
  • Daniel Bjorkegren
  • Joshua E. Blumenstock
  • Samsun Knight

Abstract

An increasing number of decisions are guided by machine learning algorithms. In many settings, from consumer credit to criminal justice, those decisions are made by applying an estimator to data on an individual's observed behavior. But when consequential decisions are encoded in rules, individuals may strategically alter their behavior to achieve desired outcomes. This paper develops a new class of estimator that is stable under manipulation, even when the decision rule is fully transparent. We explicitly model the costs of manipulating different behaviors, and identify decision rules that are stable in equilibrium. Through a large field experiment in Kenya, we show that decision rules estimated with our strategy-robust method outperform those based on standard supervised learning approaches.

Suggested Citation

  • Daniel Bjorkegren & Joshua E. Blumenstock & Samsun Knight, 2020. "Manipulation-Proof Machine Learning," Papers 2004.03865, arXiv.org.
    • Handle: RePEc:arx:papers:2004.03865
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2004.03865
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Stefano DellaVigna & Devin Pope, 2018. "Predicting Experimental Results: Who Knows What?," Journal of Political Economy, University of Chicago Press, vol. 126(6), pages 2410-2456.
    2. Thomas S. Dee & Will Dobbie & Brian A. Jacob & Jonah Rockoff, 2019. "The Causes and Consequences of Test Score Manipulation: Evidence from the New York Regents Examinations," American Economic Journal: Applied Economics, American Economic Association, vol. 11(3), pages 382-423, July.
    3. Rema Hanna & Benjamin A. Olken, 2018. "Universal Basic Incomes versus Targeted Transfers: Anti-Poverty Programs in Developing Countries," Journal of Economic Perspectives, American Economic Association, vol. 32(4), pages 201-226, Fall.
    4. Michael Greenstone & Guojun He & Ruixue Jia & Tong Liu, 2022. "Can Technology Solve the Principal-Agent Problem? Evidence from China's War on Air Pollution," American Economic Review: Insights, American Economic Association, vol. 4(1), pages 54-70, March.
    5. J. A. Mirrlees, 1971. "An Exploration in the Theory of Optimum Income Taxation," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 38(2), pages 175-208.
    6. Hanna, Rema & Olken, Benjamin A., 2018. "Universal Basic Incomes vs. Targeted Transfers: Anti-Poverty Programs in Developing Countries," Working Paper Series rwp18-024, Harvard University, John F. Kennedy School of Government.
    7. Yanhao Wei & Pinar Yildirim & Christophe Van den Bulte & Chrysanthos Dellarocas, 2016. "Credit Scoring with Social Network Data," Marketing Science, INFORMS, vol. 35(2), pages 234-258, March.
    8. Joshua E. Blumenstock, 2018. "Estimating Economic Characteristics with Phone Data," AEA Papers and Proceedings, American Economic Association, vol. 108, pages 72-76, May.
    9. Holmstrom, Bengt & Milgrom, Paul, 1987. "Aggregation and Linearity in the Provision of Intertemporal Incentives," Econometrica, Econometric Society, vol. 55(2), pages 303-328, March.
    10. Banerjee, Abhijit & Hanna, Rema & Olken, Benjamin A. & Sumarto, Sudarno, 2018. "The (Lack of) Distortionary Effects of Proxy-Means Tests: Results from a Nationwide Experiment in Indonesia," Working Paper Series rwp18-041, Harvard University, John F. Kennedy School of Government.
    11. Nichols, Albert L & Zeckhauser, Richard J, 1982. "Targeting Transfers through Restrictions on Recipients," American Economic Review, American Economic Association, vol. 72(2), pages 372-377, May.
    12. Jon Kleinberg & Himabindu Lakkaraju & Jure Leskovec & Jens Ludwig & Sendhil Mullainathan, 2018. "Human Decisions and Machine Predictions," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 133(1), pages 237-293.
    13. Gabriel Carroll, 2015. "Robustness and Linear Contracts," American Economic Review, American Economic Association, vol. 105(2), pages 536-563, February.
    14. Eliaz, Kfir & Spiegler, Ran, 2022. "On incentive-compatible estimators," Games and Economic Behavior, Elsevier, vol. 132(C), pages 204-220.
    15. Lucas, Robert Jr, 1976. "Econometric policy evaluation: A critique," Carnegie-Rochester Conference Series on Public Policy, Elsevier, vol. 1(1), pages 19-46, January.
    16. Gonzalez-Lira, Andres & Mobarak, Ahmed Mushfiq, 2019. "Slippery Fish: Enforcing Regulation under Subversive Adaptation," IZA Discussion Papers 12179, Institute of Labor Economics (IZA).
    17. Vivi Alatas & Abhijit Banerjee & Rema Hanna & Benjamin A. Olken & Ririn Purnamasari & Matthew Wai-Poi, 2016. "Self-Targeting: Evidence from a Field Experiment in Indonesia," Journal of Political Economy, University of Chicago Press, vol. 124(2), pages 371-427.
    18. Paul Niehaus & Antonia Atanassova & Marianne Bertrand & Sendhil Mullainathan, 2013. "Targeting with Agents," American Economic Journal: Economic Policy, American Economic Association, vol. 5(1), pages 206-238, February.
    19. Michael Spence, 1973. "Job Market Signaling," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 87(3), pages 355-374.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Delprato, Marcos & Frola, Alessia & Antequera, Germán, 2022. "Indigenous and non-Indigenous proficiency gaps for out-of-school and in-school populations: A machine learning approach," International Journal of Educational Development, Elsevier, vol. 93(C).
    2. Elliott Ash & Sergio Galletta & Tommaso Giommoni, 2021. "A Machine Learning Approach to Analyze and Support Anti-Corruption Policy," CESifo Working Paper Series 9015, CESifo.
    3. Aiken, Emily L. & Bedoya, Guadalupe & Blumenstock, Joshua E. & Coville, Aidan, 2023. "Program targeting with machine learning and mobile phone data: Evidence from an anti-poverty intervention in Afghanistan," Journal of Development Economics, Elsevier, vol. 161(C).
    4. Roshni Sahoo & Stefan Wager, 2022. "Policy Learning with Competing Agents," Papers 2204.01884, arXiv.org, revised Apr 2024.
    5. Blumenstock, Joshua & Aiken, Emily & Bellue, Suzanne & Udry, Christopher & Karlan, Dean, 2021. "Machine Learning and Mobile Phone Data Can Improve the Targeting of Humanitarian Assistance," CEPR Discussion Papers 16385, C.E.P.R. Discussion Papers.
    6. Meena Jagadeesan & Celestine Mendler-Dunner & Moritz Hardt, 2021. "Alternative Microfoundations for Strategic Classification," Papers 2106.12705, arXiv.org.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Emily Aiken & Suzanne Bellue & Dean Karlan & Christopher R. Udry & Joshua Blumenstock, 2021. "Machine Learning and Mobile Phone Data Can Improve the Targeting of Humanitarian Assistance," NBER Working Papers 29070, National Bureau of Economic Research, Inc.
    2. Hilary W. Hoynes & Jesse Rothstein, 2019. "Universal Basic Income in the US and Advanced Countries," NBER Working Papers 25538, National Bureau of Economic Research, Inc.
    3. Haseeb, Muhammad & Vyborny, Kate, 2022. "Data, discretion and institutional capacity: Evidence from cash transfers in Pakistan," Journal of Public Economics, Elsevier, vol. 206(C).
    4. Rema Hanna & Benjamin A. Olken, 2018. "Universal Basic Incomes vs. Targeted Transfers: Anti-Poverty Programs in Developing Countries," NBER Working Papers 24939, National Bureau of Economic Research, Inc.
    5. Ali Enami & Ugo Gentilini & Patricio Larroulet & Nora Lustig & Emma Monsalve & Siyu Quan & Jamele Rigolini, 2023. "Universal Basic Income Programs: How Much Would Taxes Need to Rise? Evidence for Brazil, Chile, India, Russia, and South Africa," Journal of Development Studies, Taylor & Francis Journals, vol. 59(9), pages 1443-1463, September.
    6. Clément Carbonnier, 2023. "Welfare Economics and Neoliberalism: Interpreting the ideal type of perfect competition general equilibrium," SciencePo Working papers Main hal-04062786, HAL.
    7. Patacchini, Eleonora & Barwick, Panle Jia & Liu, Yanyan & Wu, Qi, 2019. "Information, Mobile Communication, and Referral Effects," CEPR Discussion Papers 13786, C.E.P.R. Discussion Papers.
    8. Abhijit Banerjee & Paul Niehaus & Tavneet Suri, 2019. "Universal Basic Income in the Developing World," Annual Review of Economics, Annual Reviews, vol. 11(1), pages 959-983, August.
    9. Bjorkegren, Dan & Blumenstock, Joshua & Knight, Samsun, 2022. "(Machine) Learning What Policies Value," CEPR Discussion Papers 17364, C.E.P.R. Discussion Papers.
    10. Robin Boadway, 2011. "Viewpoint: Innovations in the theory and practice of redistribution policy," Canadian Journal of Economics/Revue canadienne d'économique, John Wiley & Sons, vol. 44(4), pages 1138-1183, November.
    11. Marianne Bertrand & Bruno Crépon & Alicia Marguerie & Patrick Premand, 2021. "Do Workfare Programs Live Up to Their Promises? Experimental Evidence from Cote D’Ivoire," NBER Working Papers 28664, National Bureau of Economic Research, Inc.
    12. Fredrik Andersson, 1994. "On the screening power of incentive schemes," Review of Economic Design, Springer;Society for Economic Design, vol. 1(1), pages 251-274, December.
    13. Bottan, Nicolas & Hoffmann, Bridget & Vera-Cossio, Diego A., 2021. "Stepping up during a crisis: The unintended effects of a noncontributory pension program during the Covid-19 pandemic," Journal of Development Economics, Elsevier, vol. 150(C).
    14. Meena Jagadeesan & Celestine Mendler-Dunner & Moritz Hardt, 2021. "Alternative Microfoundations for Strategic Classification," Papers 2106.12705, arXiv.org.
    15. Raddatz, Guido, 2019. "Das bedingungslose Grundeinkommen – ein Luftschloss," Zeitthemen 02, Stiftung Marktwirtschaft / The Market Economy Foundation, Berlin.
    16. Aubrey Clark & Giovanni Reggiani, 2021. "Contracts for acquiring information," Papers 2103.03911, arXiv.org.
    17. Coelho, José, 2020. "Universal basic income and skill-biased technological change," MPRA Paper 99195, University Library of Munich, Germany, revised 20 Mar 2020.
    18. Aiken, Emily L. & Bedoya, Guadalupe & Blumenstock, Joshua E. & Coville, Aidan, 2023. "Program targeting with machine learning and mobile phone data: Evidence from an anti-poverty intervention in Afghanistan," Journal of Development Economics, Elsevier, vol. 161(C).
    19. Missbach, Leonard & Steckel, Jan Christoph & Vogt-Schilb, Adrien, 2024. "Cash transfers in the context of carbon pricing reforms in Latin America and the Caribbean," World Development, Elsevier, vol. 173(C).
    20. Oyer, Paul & Schaefer, Scott, 2011. "Personnel Economics: Hiring and Incentives," Handbook of Labor Economics, in: O. Ashenfelter & D. Card (ed.), Handbook of Labor Economics, edition 1, volume 4, chapter 20, pages 1769-1823, Elsevier.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2004.03865. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.