IDEAS home Printed from https://ideas.repec.org/p/nbr/nberwo/24352.html
   My bibliography  Save this paper

Self-Regulating Artificial General Intelligence

Author

Listed:
  • Joshua S. Gans

Abstract

This paper examines the paperclip apocalypse concern for artificial general intelligence. This arises when a superintelligent AI with a simple goal (ie., producing paperclips) accumulates power so that all resources are devoted towards that goal and are unavailable for any other use. Conditions are provided under which a paper apocalypse can arise but the model also shows that, under certain architectures for recursive self-improvement of AIs, that a paperclip AI may refrain from allowing power capabilities to be developed. The reason is that such developments pose the same control problem for the AI as they do for humans (over AIs) and hence, threaten to deprive it of resources for its primary goal.

Suggested Citation

  • Joshua S. Gans, 2018. "Self-Regulating Artificial General Intelligence," NBER Working Papers 24352, National Bureau of Economic Research, Inc.
    • Handle: RePEc:nbr:nberwo:24352
    Note: PR
    as

    Download full text from publisher

    File URL: http://www.nber.org/papers/w24352.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Rubinstein, Ariel, 1998. "Definable preferences: An example1," European Economic Review, Elsevier, vol. 42(3-5), pages 553-560, May.
    2. Sobel, Joel, 1992. "How to Count to One Thousand," Economic Journal, Royal Economic Society, vol. 102(410), pages 1-8, January.
    3. Herbert A. Simon, 1996. "The Sciences of the Artificial, 3rd Edition," MIT Press Books, The MIT Press, edition 1, volume 1, number 0262691914, December.
    4. Harold Houba & Roland Iwan Luttens & Hans-Peter Weikard, 2017. "Pareto efficiency in the jungle," Review of Economic Design, Springer;Society for Economic Design, vol. 21(3), pages 153-161, September.
    5. Oliver Hart, 2017. "Incomplete Contracts and Control," American Economic Review, American Economic Association, vol. 107(7), pages 1731-1752, July.
    6. Michele Piccione & Ariel Rubinstein, 2007. "Equilibrium in the Jungle," Economic Journal, Royal Economic Society, vol. 117(522), pages 883-896, July.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Ajay Agrawal & Joshua Gans & Avi Goldfarb, 2019. "Economic Policy for Artificial Intelligence," Innovation Policy and the Economy, University of Chicago Press, vol. 19(1), pages 139-159.
    2. Fossen, Frank M. & Sorgner, Alina, 2022. "New digital technologies and heterogeneous wage and employment dynamics in the United States: Evidence from individual-level data," Technological Forecasting and Social Change, Elsevier, vol. 175(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Massimo Egidi, 1995. "Virtuous and Adverse selection within Economic Organizations," CEEL Working Papers 9502, Cognitive and Experimental Economics Laboratory, Department of Economics, University of Trento, Italia.
    2. Bertrand Crettez, 2020. "Pareto-minimality in the jungle," Public Choice, Springer, vol. 182(3), pages 495-508, March.
    3. Massimo Egidi & Luigi Marengo, 1995. "Division of Labour and Social Coordination Modes. A simple simulation model," CEEL Working Papers 9501, Cognitive and Experimental Economics Laboratory, Department of Economics, University of Trento, Italia.
    4. Tobias Knabke & Sebastian Olbrich, 2018. "Building novel capabilities to enable business intelligence agility: results from a quantitative study," Information Systems and e-Business Management, Springer, vol. 16(3), pages 493-546, August.
    5. Sunder Shyam, 2011. "Imagined Worlds of Accounting," Accounting, Economics, and Law: A Convivium, De Gruyter, vol. 1(1), pages 1-14, January.
    6. Jung, Hanjoon Michael, 2009. "Spatial pillage game," Journal of Mathematical Economics, Elsevier, vol. 45(11), pages 701-707, December.
    7. de Miguel-Arribas, A. & Morón-Vidal, J. & Floría, L.M. & Gracia-Lázaro, C. & Hernández, L. & Moreno, Y., 2024. "Contests in two fronts," Chaos, Solitons & Fractals, Elsevier, vol. 179(C).
    8. McCown, R. L., 2002. "Changing systems for supporting farmers' decisions: problems, paradigms, and prospects," Agricultural Systems, Elsevier, vol. 74(1), pages 179-220, October.
    9. Basile, Luigi Jesus & Carbonara, Nunzia & Pellegrino, Roberta & Panniello, Umberto, 2023. "Business intelligence in the healthcare industry: The utilization of a data-driven approach to support clinical decision making," Technovation, Elsevier, vol. 120(C).
    10. Loris Gaio, 2005. "A diversity-based approach to requirements tracing in new product development," ROCK Working Papers 031, Department of Computer and Management Sciences, University of Trento, Italy, revised 13 Jun 2008.
    11. B. A. Huberman & N. S. Glance, "undated". "Diversity and Collective Action," Working Papers _001, Xerox Research Park.
    12. Ivan Balbuzanov & Maciej H. Kotowski, 2019. "Endowments, Exclusion, and Exchange," Econometrica, Econometric Society, vol. 87(5), pages 1663-1692, September.
    13. Zhewei Zhang & Youngjin Yoo & Kalle Lyytinen & Aron Lindberg, 2021. "The Unknowability of Autonomous Tools and the Liminal Experience of Their Use," Information Systems Research, INFORMS, vol. 32(4), pages 1192-1213, December.
    14. Triossi, Matteo, 2005. "Implementation with state dependent feasible sets and preferences: a renegotiation approach," UC3M Working papers. Economics we057136, Universidad Carlos III de Madrid. Departamento de Economía.
    15. Herbst, Luisa & Konrad, Kai A. & Morath, Florian, 2017. "Balance of power and the propensity of conflict," Games and Economic Behavior, Elsevier, vol. 103(C), pages 168-184.
    16. David Stadelmann & Benno Torgler, 2012. "Bounded Rationality and Voting Decisions Exploring a 160-Year Period," Working Papers 2012.70, Fondazione Eni Enrico Mattei.
    17. Luca Colombo & Paola Labrecciosa & Agnieszka Rusinowska, 2025. "A dynamic analysis of criminal networks," Post-Print hal-04850675, HAL.
    18. Francis Marleau Donais & Irène Abi-Zeid & E. Owen D. Waygood & Roxane Lavoie, 2021. "A Framework for Post-Project Evaluation of Multicriteria Decision Aiding Processes from the Stakeholders’ Perspective: Design and Application," Group Decision and Negotiation, Springer, vol. 30(5), pages 1161-1191, October.
    19. H. Christopher Frey & Sumeet R. Patil, 2002. "Identification and Review of Sensitivity Analysis Methods," Risk Analysis, John Wiley & Sons, vol. 22(3), pages 553-578, June.
    20. Kim, Yoonha, 2021. "Linking theories of incomplete contracts to empirics in IPO contracting," Finance Research Letters, Elsevier, vol. 41(C).

    More about this item

    JEL classification:

    • C72 - Mathematical and Quantitative Methods - - Game Theory and Bargaining Theory - - - Noncooperative Games
    • D02 - Microeconomics - - General - - - Institutions: Design, Formation, Operations, and Impact

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nbr:nberwo:24352. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: the person in charge (email available below). General contact details of provider: https://edirc.repec.org/data/nberrus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.