IDEAS home Printed from https://ideas.repec.org/a/eee/mateco/v100y2022ics0304406821001695.html
   My bibliography  Save this article

Reference points and learning

Author

Listed:
  • Beggs, Alan

Abstract

This paper studies learning when agents evaluate outcomes in comparison to reference points, which may be adjusted in light of experience. It shows that certain models of reinforcement learning, motivated by those popular in machine learning and neuroscience, lead to classes of recursive preferences.

Suggested Citation

  • Beggs, Alan, 2022. "Reference points and learning," Journal of Mathematical Economics, Elsevier, vol. 100(C).
    • Handle: RePEc:eee:mateco:v:100:y:2022:i:c:s0304406821001695
    DOI: 10.1016/j.jmateco.2021.102621
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0304406821001695
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.jmateco.2021.102621?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Colin Camerer & Linda Babcock & George Loewenstein & Richard Thaler, 1997. "Labor Supply of New York City Cabdrivers: One Day at a Time," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 112(2), pages 407-441.
    2. Johannes Abeler & Armin Falk & Lorenz Goette & David Huffman, 2011. "Reference Points and Effort Provision," American Economic Review, American Economic Association, vol. 101(2), pages 470-492, April.
    3. Yaari, Menahem E, 1987. "The Dual Theory of Choice under Risk," Econometrica, Econometric Society, vol. 55(1), pages 95-115, January.
    4. Botond Kőszegi & Matthew Rabin, 2006. "A Model of Reference-Dependent Preferences," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 121(4), pages 1133-1165.
    5. Larry G. Epstein & Stanley E. Zin, 2013. "Substitution, risk aversion and the temporal behavior of consumption and asset returns: A theoretical framework," World Scientific Book Chapters, in: Leonard C MacLean & William T Ziemba (ed.), HANDBOOK OF THE FUNDAMENTALS OF FINANCIAL DECISION MAKING Part I, chapter 12, pages 207-239, World Scientific Publishing Co. Pte. Ltd..
    6. David K. Backus & Bryan R. Routledge & Stanley E. Zin, 2005. "Exotic Preferences for Macroeconomists," NBER Chapters, in: NBER Macroeconomics Annual 2004, Volume 19, pages 319-414, National Bureau of Economic Research, Inc.
    7. Kimball, Miles S, 1990. "Precautionary Saving in the Small and in the Large," Econometrica, Econometric Society, vol. 58(1), pages 53-73, January.
    8. Johnsen, Thore H & Donaldson, John B, 1985. "The Structure of Intertemporal Preferences under Uncertainty and Time Consistent Plans," Econometrica, Econometric Society, vol. 53(6), pages 1451-1458, November.
    9. Ma, D.-J. & Makowski, A.M. & Shwartz, A., 1990. "Stochastic approximations for finite-state Markov chains," Stochastic Processes and their Applications, Elsevier, vol. 35(1), pages 27-45, June.
    10. Gul, Faruk, 1991. "A Theory of Disappointment Aversion," Econometrica, Econometric Society, vol. 59(3), pages 667-686, May.
    11. Daniel Kahneman & Amos Tversky, 2013. "Prospect Theory: An Analysis of Decision Under Risk," World Scientific Book Chapters, in: Leonard C MacLean & William T Ziemba (ed.), HANDBOOK OF THE FUNDAMENTALS OF FINANCIAL DECISION MAKING Part I, chapter 6, pages 99-127, World Scientific Publishing Co. Pte. Ltd..
    12. Beggs, A.W., 2005. "On the convergence of reinforcement learning," Journal of Economic Theory, Elsevier, vol. 122(1), pages 1-36, May.
    13. Botond Koszegi & Matthew Rabin, 2007. "Reference-Dependent Risk Attitudes," American Economic Review, American Economic Association, vol. 97(4), pages 1047-1073, September.
    14. David Gill & Victoria Prowse, 2012. "A Structural Analysis of Disappointment Aversion in a Real Effort Competition," American Economic Review, American Economic Association, vol. 102(1), pages 469-503, February.
    15. Rulon D. Pope & Jean-Paul Chavas, 1985. "Producer Surplus and Risk," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 100(Supplemen), pages 853-869.
    16. Marinacci, Massimo & Montrucchio, Luigi, 2010. "Unique solutions for stochastic recursive utilities," Journal of Economic Theory, Elsevier, vol. 145(5), pages 1776-1804, September.
    17. Todd Sarver, 2012. "Optimal Reference Points and Anticipation," Discussion Papers 1566, Northwestern University, Center for Mathematical Studies in Economics and Management Science.
    18. Hopkins, Ed & Posch, Martin, 2005. "Attainability of boundary points under reinforcement learning," Games and Economic Behavior, Elsevier, vol. 53(1), pages 110-125, October.
    19. Edward Hill & Marco Bardoscia & Arthur Turrell, 2021. "Solving Heterogeneous General Equilibrium Economic Models with Deep Reinforcement Learning," Papers 2103.16977, arXiv.org.
    20. Botond Koszegi & Matthew Rabin, 2009. "Reference-Dependent Consumption Plans," American Economic Review, American Economic Association, vol. 99(3), pages 909-936, June.
    21. Erev, Ido & Roth, Alvin E, 1998. "Predicting How People Play Games: Reinforcement Learning in Experimental Games with Unique, Mixed Strategy Equilibria," American Economic Review, American Economic Association, vol. 88(4), pages 848-881, September.
    22. Camelia M. Kuhnen, 2015. "Asymmetric Learning from Financial Information," Journal of Finance, American Finance Association, vol. 70(5), pages 2029-2062, October.
    23. In-Koo Cho & Anna Rubinchik, 2017. "Contemplation vs. intuition: a reinforcement learning perspective," EURO Journal on Decision Processes, Springer;EURO - The Association of European Operational Research Societies, vol. 5(1), pages 141-167, November.
    24. Andrew Caplin & Mark Dean & Paul W. Glimcher & Robb B. Rutledge, 2010. "Measuring Beliefs and Rewards: A Neuroeconomic Approach," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 125(3), pages 923-960.
    25. Kreps, David M & Porteus, Evan L, 1978. "Temporal Resolution of Uncertainty and Dynamic Choice Theory," Econometrica, Econometric Society, vol. 46(1), pages 185-200, January.
    26. Dekel, Eddie, 1986. "An axiomatic characterization of preferences under uncertainty: Weakening the independence axiom," Journal of Economic Theory, Elsevier, vol. 40(2), pages 304-318, December.
    27. Newey, Whitney K & Powell, James L, 1987. "Asymmetric Least Squares Estimation and Testing," Econometrica, Econometric Society, vol. 55(4), pages 819-847, July.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Peyman Khezr & Shabbir Ahmad, 2018. "Anchoring in the Housing Market: Evidence from Sydney," Discussion Papers Series 596, School of Economics, University of Queensland, Australia.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Florian Zimmermann, 2015. "Clumped or Piecewise? Evidence on Preferences for Information," Management Science, INFORMS, vol. 61(4), pages 740-753, April.
    2. Björn Bartling & Leif Brandes & Daniel Schunk, 2015. "Expectations as Reference Points: Field Evidence from Professional Soccer," Management Science, INFORMS, vol. 61(11), pages 2646-2661, November.
    3. Karle, Heiko & Schumacher, Heiner & Vølund, Rune, 2023. "Consumer loss aversion and scale-dependent psychological switching costs," Games and Economic Behavior, Elsevier, vol. 138(C), pages 214-237.
    4. Jong-Hee Hahn & Jinwoo Kim & Sang-Hyun Kim & Jihong Lee, 2018. "Price discrimination with loss averse consumers," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 65(3), pages 681-728, May.
    5. Björn Bartling & Leif Brandes & Daniel Schunk, 2012. "Expectations as reference points: field evidence from experienced subjects in a competitive, high-stakes environment," ECON - Working Papers 073, Department of Economics - University of Zurich.
    6. Hsiaw, Alice, 2018. "Goal bracketing and self-control," Games and Economic Behavior, Elsevier, vol. 111(C), pages 100-121.
    7. Heiko Karle & Heiner Schumacher & Rune Vølund, 2020. "Consumer search and the uncertainty effect," Working Papers of Department of Economics, Leuven 657766, KU Leuven, Faculty of Economics and Business (FEB), Department of Economics, Leuven.
    8. Zhihua Li & Songfa Zhong, 2023. "Reference Dependence in Intertemporal Preference," Management Science, INFORMS, vol. 69(1), pages 475-490, January.
    9. Antonio Rosato, 2016. "Selling substitute goods to loss-averse consumers: limited availability, bargains, and rip-offs," RAND Journal of Economics, RAND Corporation, vol. 47(3), pages 709-733, August.
    10. Kohei Daido & Takeshi Murooka, 2016. "Team Incentives and Reference‐Dependent Preferences," Journal of Economics & Management Strategy, Wiley Blackwell, vol. 25(4), pages 958-989, December.
    11. Meisner, Vincent & von Wangenheim, Jonas, 2019. "School Choice and Loss Aversion," Rationality and Competition Discussion Paper Series 208, CRC TRR 190 Rationality and Competition.
    12. Adhvaryu, Achyuta & Nyshadham, Anant & Xu, Huayu, 2023. "Hostel takeover: Living conditions, reference dependence, and the well-being of migrant workers," Journal of Public Economics, Elsevier, vol. 226(C).
    13. Wenner, Lukas M., 2015. "Expected prices as reference points—Theory and experiments," European Economic Review, Elsevier, vol. 75(C), pages 60-79.
    14. Macera, Rosario, 2018. "Intertemporal incentives under loss aversion," Journal of Economic Theory, Elsevier, vol. 178(C), pages 551-594.
    15. Dato, Simon & Grunewald, Andreas & Müller, Daniel & Strack, Philipp, 2017. "Expectation-based loss aversion and strategic interaction," Games and Economic Behavior, Elsevier, vol. 104(C), pages 681-705.
    16. Heiko Karle & Dirk Engelmann & Martin Peitz, 2022. "Student performance and loss aversion," Scandinavian Journal of Economics, Wiley Blackwell, vol. 124(2), pages 420-456, April.
    17. Luca De Gennaro Aquino & Xuedong He & Moris Simon Strub & Yuting Yang, 2024. "Reference-dependent asset pricing with a stochastic consumption-dividend ratio," Papers 2401.12856, arXiv.org.
    18. Heffetz, Ori, 2021. "Are reference points merely lagged beliefs over probabilities?," Journal of Economic Behavior & Organization, Elsevier, vol. 181(C), pages 252-269.
    19. Benjamin Balzer & Antonio Rosato, 2021. "Expectations-Based Loss Aversion in Auctions with Interdependent Values: Extensive vs. Intensive Risk," Management Science, INFORMS, vol. 67(2), pages 1056-1074, February.
    20. Smith, Alec, 2019. "Lagged beliefs and reference-dependent utility," Journal of Economic Behavior & Organization, Elsevier, vol. 167(C), pages 331-340.

    More about this item

    Keywords

    Reference points; Reinforcement learning; Recursive preferences;
    All these keywords.

    JEL classification:

    • D83 - Microeconomics - - Information, Knowledge, and Uncertainty - - - Search; Learning; Information and Knowledge; Communication; Belief; Unawareness
    • D87 - Microeconomics - - Information, Knowledge, and Uncertainty - - - Neuroeconomics

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:mateco:v:100:y:2022:i:c:s0304406821001695. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/jmateco .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.