IDEAS home Printed from https://ideas.repec.org/p/zbw/bubdps/512021.html
   My bibliography  Save this paper

Optimal monetary policy using reinforcement learning

Author

Listed:
  • Hinterlang, Natascha
  • Tänzer, Alina

Abstract

This paper introduces a reinforcement learning based approach to compute optimal interest rate reaction functions in terms of fulfilling inflation and output gap targets. The method is generally flexible enough to incorporate restrictions like the zero lower bound, nonlinear economy structures or asymmetric preferences. We use quarterly U.S. data from1987:Q3-2007:Q2 to estimate (nonlinear) model transition equations, train optimal policies and perform counterfactual analyses to evaluate them, assuming that the transition equations remain unchanged. All of our resulting policy rules outperform other common rules as well as the actual federal funds rate. Given a neural network representation of the economy, our optimized nonlinear policy rules reduce the central bank's loss by over43 %. A DSGE model comparison exercise further indicates robustness of the optimized rules.

Suggested Citation

  • Hinterlang, Natascha & Tänzer, Alina, 2021. "Optimal monetary policy using reinforcement learning," Discussion Papers 51/2021, Deutsche Bundesbank.
  • Handle: RePEc:zbw:bubdps:512021
    as

    Download full text from publisher

    File URL: https://www.econstor.eu/bitstream/10419/248736/1/1784835994.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Adam, Klaus & Billi, Roberto M., 2006. "Optimal Monetary Policy under Commitment with a Zero Bound on Nominal Interest Rates," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 38(7), pages 1877-1905, October.
    2. Dolado Juan & Pedrero Ramón María-Dolores & Ruge-Murcia Francisco J., 2004. "Nonlinear Monetary Policy Rules: Some New Evidence for the U.S," Studies in Nonlinear Dynamics & Econometrics, De Gruyter, vol. 8(3), pages 1-34, September.
    3. Orphanides, Athanasios & Wieland, Volker, 2000. "Inflation zone targeting," European Economic Review, Elsevier, vol. 44(7), pages 1351-1387, June.
    4. John B. Taylor & Volker Wieland, 2012. "Surprising Comparative Properties of Monetary Models: Results from a New Model Database," The Review of Economics and Statistics, MIT Press, vol. 94(3), pages 800-816, August.
    5. Lars Peter Hansen & Thomas J Sargent, 2014. "Robust Control and Model Uncertainty," World Scientific Book Chapters, in: UNCERTAINTY WITHIN ECONOMIC MODELS, chapter 5, pages 145-154, World Scientific Publishing Co. Pte. Ltd..
    6. Laurence Ball & Sandeep Mazumder, 2019. "A Phillips Curve with Anchored Expectations and Short‐Term Unemployment," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 51(1), pages 111-137, February.
    7. Wieland, Volker, 2000. "Monetary policy, parameter uncertainty and optimal learning," Journal of Monetary Economics, Elsevier, vol. 46(1), pages 199-228, August.
    8. Svensson, Lars E. O., 1997. "Inflation forecast targeting: Implementing and monitoring inflation targets," European Economic Review, Elsevier, vol. 41(6), pages 1111-1146, June.
    9. Stephan Zheng & Alexander Trott & Sunil Srinivasa & Nikhil Naik & Melvin Gruesbeck & David C. Parkes & Richard Socher, 2020. "The AI Economist: Improving Equality and Productivity with AI-Driven Tax Policies," Papers 2004.13332, arXiv.org.
    10. Ioan Carabenciov & Charles Freedman & Mr. Roberto Garcia-Saltos & Mr. Douglas Laxton & Mr. Ondrej Kamenik & Mr. Petar Manchev, 2013. "GPM6: The Global Projection Model with 6 Regions," IMF Working Papers 2013/087, International Monetary Fund.
    11. Andrew Levin & Volker Wieland & John C. Williams, 2003. "The Performance of Forecast-Based Monetary Policy Rules Under Model Uncertainty," American Economic Review, American Economic Association, vol. 93(3), pages 622-645, June.
    12. Olivier Coibion & Yuriy Gorodnichenko, 2015. "Is the Phillips Curve Alive and Well after All? Inflation Expectations and the Missing Disinflation," American Economic Journal: Macroeconomics, American Economic Association, vol. 7(1), pages 197-232, January.
    13. Cúrdia, Vasco & Woodford, Michael, 2016. "Credit Frictions and Optimal Monetary Policy," Journal of Monetary Economics, Elsevier, vol. 84(C), pages 30-65.
    14. Marco Del Negro & Marc P. Giannoni & Frank Schorfheide, 2015. "Inflation in the Great Recession and New Keynesian Models," American Economic Journal: Macroeconomics, American Economic Association, vol. 7(1), pages 168-196, January.
    15. Lars E.O. Svensson, 2020. "Monetary Policy Strategies for the Federal Reserve," International Journal of Central Banking, International Journal of Central Banking, vol. 16(1), pages 133-193, February.
    16. Ben S. Bernanke & Julio J. Rotemberg (ed.), 1997. "NBER Macroeconomics Annual 1997," MIT Press Books, The MIT Press, edition 1, volume 1, number 026252242x, December.
    17. Athanasios Orphanides, 2001. "Monetary Policy Rules Based on Real-Time Data," American Economic Review, American Economic Association, vol. 91(4), pages 964-985, September.
    18. Cogan, John F. & Cwik, Tobias & Taylor, John B. & Wieland, Volker, 2010. "New Keynesian versus old Keynesian government spending multipliers," Journal of Economic Dynamics and Control, Elsevier, vol. 34(3), pages 281-295, March.
    19. Jesús Fernández-Villaverde & Pablo Guerrón-Quintana & Keith Kuester & Juan Rubio-Ramírez, 2015. "Fiscal Volatility Shocks and Economic Activity," American Economic Review, American Economic Association, vol. 105(11), pages 3352-3384, November.
    20. Marvin Goodfriend, 2004. "Inflation Targeting in the United States?," NBER Chapters, in: The Inflation-Targeting Debate, pages 311-337, National Bureau of Economic Research, Inc.
    21. Tambakis Demosthenes N., 2009. "Optimal Monetary Policy with a Convex Phillips Curve," The B.E. Journal of Macroeconomics, De Gruyter, vol. 9(1), pages 1-25, June.
    22. Bernanke, Ben S. & Gertler, Mark & Gilchrist, Simon, 1999. "The financial accelerator in a quantitative business cycle framework," Handbook of Macroeconomics, in: J. B. Taylor & M. Woodford (ed.), Handbook of Macroeconomics, edition 1, volume 1, chapter 21, pages 1341-1393, Elsevier.
    23. Wieland, V. & Afanasyeva, E. & Kuete, M. & Yoo, J., 2016. "New Methods for Macro-Financial Model Comparison and Policy Analysis," Handbook of Macroeconomics, in: J. B. Taylor & Harald Uhlig (ed.), Handbook of Macroeconomics, edition 1, volume 2, chapter 0, pages 1241-1319, Elsevier.
    24. Glenn Rudebusch & Lars E.O. Svensson, 1999. "Policy Rules for Inflation Targeting," NBER Chapters, in: Monetary Policy Rules, pages 203-262, National Bureau of Economic Research, Inc.
    25. Michael Woodford, 2001. "The Taylor Rule and Optimal Monetary Policy," American Economic Review, American Economic Association, vol. 91(2), pages 232-237, May.
    26. Matteo Iacoviello & Stefano Neri, 2010. "Housing Market Spillovers: Evidence from an Estimated DSGE Model," American Economic Journal: Macroeconomics, American Economic Association, vol. 2(2), pages 125-164, April.
    27. Dolado, Juan J. & Maria-Dolores, Ramon & Naveira, Manuel, 2005. "Are monetary-policy reaction functions asymmetric?: The role of nonlinearity in the Phillips curve," European Economic Review, Elsevier, vol. 49(2), pages 485-503, February.
    28. J. Tetlow, Robert & von zur Muehlen, Peter, 2001. "Robust monetary policy with misspecified models: Does model uncertainty always call for attenuated policy?," Journal of Economic Dynamics and Control, Elsevier, vol. 25(6-7), pages 911-949, June.
    29. Taylor, John B. & Williams, John C., 2010. "Simple and Robust Rules for Monetary Policy," Handbook of Monetary Economics, in: Benjamin M. Friedman & Michael Woodford (ed.), Handbook of Monetary Economics, edition 1, volume 3, chapter 15, pages 829-859, Elsevier.
    30. Volodymyr Mnih & Koray Kavukcuoglu & David Silver & Andrei A. Rusu & Joel Veness & Marc G. Bellemare & Alex Graves & Martin Riedmiller & Andreas K. Fidjeland & Georg Ostrovski & Stig Petersen & Charle, 2015. "Human-level control through deep reinforcement learning," Nature, Nature, vol. 518(7540), pages 529-533, February.
    31. Schaling, Eric, 2004. "The Nonlinear Phillips Curve and Inflation Forecast Targeting: Symmetric versus Asymmetric Monetary Policy Rules," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 36(3), pages 361-386, June.
    32. Orphanides, Athanasios, 2003. "Monetary policy evaluation with noisy information," Journal of Monetary Economics, Elsevier, vol. 50(3), pages 605-631, April.
    33. Lawrence J. Christiano & Roberto Motto & Massimo Rostagno, 2014. "Risk Shocks," American Economic Review, American Economic Association, vol. 104(1), pages 27-65, January.
    34. Wieland, Volker & Cwik, Tobias & Müller, Gernot J. & Schmidt, Sebastian & Wolters, Maik, 2012. "A new comparative approach to macroeconomic modeling and policy analysis," Journal of Economic Behavior & Organization, Elsevier, vol. 83(3), pages 523-541.
    35. Julio J. Rotemberg & Michael Woodford, 1997. "An Optimization-Based Econometric Framework for the Evaluation of Monetary Policy," NBER Chapters, in: NBER Macroeconomics Annual 1997, Volume 12, pages 297-361, National Bureau of Economic Research, Inc.
    36. Nikolsko-Rzhevskyy, Alex & Papell, David H. & Prodan, Ruxandra, 2014. "Deviations from rules-based policy and their effects," Journal of Economic Dynamics and Control, Elsevier, vol. 49(C), pages 4-17.
    37. Mingli Chen & Andreas Joseph & Michael Kumhof & Xinlei Pan & Xuan Zhou, 2021. "Deep Reinforcement Learning in a Monetary Model," Papers 2104.09368, arXiv.org, revised Jan 2023.
    38. Giorgio E. Primiceri, 2005. "Time Varying Structural Vector Autoregressions and Monetary Policy," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 72(3), pages 821-852.
    39. Lucas, Robert Jr, 1976. "Econometric policy evaluation: A critique," Carnegie-Rochester Conference Series on Public Policy, Elsevier, vol. 1(1), pages 19-46, January.
    40. Gertler, Mark & Karadi, Peter, 2011. "A model of unconventional monetary policy," Journal of Monetary Economics, Elsevier, vol. 58(1), pages 17-34, January.
    41. Mark W. Watson, 2014. "Inflation Persistence, the NAIRU, and the Great Recession," American Economic Review, American Economic Association, vol. 104(5), pages 31-36, May.
    42. John B Taylor, 2007. "The Explanatory Power of Monetary Policy Rules," Business Economics, Palgrave Macmillan;National Association for Business Economics, vol. 42(4), pages 8-15, October.
    43. Pablo S. Castro & Ajit Desai & Han Du & Rodney Garratt & Francisco Rivadeneyra, 2021. "Estimating Policy Functions in Payments Systems Using Reinforcement Learning," Staff Working Papers 21-7, Bank of Canada.
    44. Taylor, John B., 1993. "Discretion versus policy rules in practice," Carnegie-Rochester Conference Series on Public Policy, Elsevier, vol. 39(1), pages 195-214, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Jesús Fernández-Villaverde & Galo Nuño & Jesse Perla, 2024. "Taming the curse of dimensionality: quantitative economics with deep learning," Working Papers 2444, Banco de España.
    2. Kshama Dwarakanath & Jialin Dong & Svitlana Vyetrenko, 2024. "Tax Credits and Household Behavior: The Roles of Myopic Decision-Making and Liquidity in a Simulated Economy," Papers 2408.10391, arXiv.org, revised Oct 2024.
    3. Kshama Dwarakanath & Svitlana Vyetrenko & Tucker Balch, 2024. "Empirical Equilibria in Agent-based Economic systems with Learning agents," Papers 2408.12038, arXiv.org.
    4. Simone Brusatin & Tommaso Padoan & Andrea Coletta & Domenico Delli Gatti & Aldo Glielmo, 2024. "Simulating the Economic Impact of Rationality through Reinforcement Learning and Agent-Based Modelling," Papers 2405.02161, arXiv.org, revised Oct 2024.
    5. Ajit Desai, 2023. "Machine Learning for Economics Research: When What and How?," Papers 2304.00086, arXiv.org, revised Apr 2023.
    6. Kshama Dwarakanath & Svitlana Vyetrenko & Peyman Tavallali & Tucker Balch, 2024. "ABIDES-Economist: Agent-Based Simulation of Economic Systems with Learning Agents," Papers 2402.09563, arXiv.org.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Wieland, Volker & Binder, Michael & Lieberknecht, Philipp & Quintana, Jorge, 2017. "Model Uncertainty in Macroeconomics: On the Implications of Financial Frictions," CEPR Discussion Papers 12013, C.E.P.R. Discussion Papers.
    2. Wieland, Volker & Wolters, Maik, 2013. "Forecasting and Policy Making," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 239-325, Elsevier.
    3. Wieland, V. & Afanasyeva, E. & Kuete, M. & Yoo, J., 2016. "New Methods for Macro-Financial Model Comparison and Policy Analysis," Handbook of Macroeconomics, in: J. B. Taylor & Harald Uhlig (ed.), Handbook of Macroeconomics, edition 1, volume 2, chapter 0, pages 1241-1319, Elsevier.
    4. Schmidt, Sebastian & Wieland, Volker, 2013. "The New Keynesian Approach to Dynamic General Equilibrium Modeling: Models, Methods and Macroeconomic Policy Evaluation," Handbook of Computable General Equilibrium Modeling, in: Peter B. Dixon & Dale Jorgenson (ed.), Handbook of Computable General Equilibrium Modeling, edition 1, volume 1, chapter 0, pages 1439-1512, Elsevier.
    5. Lieberknecht, Philipp, 2018. "Financial Frictions, the Phillips Curve and Monetary Policy," MPRA Paper 89429, University Library of Munich, Germany.
    6. Wieland, Volker & Cwik, Tobias & Müller, Gernot J. & Schmidt, Sebastian & Wolters, Maik, 2012. "A new comparative approach to macroeconomic modeling and policy analysis," Journal of Economic Behavior & Organization, Elsevier, vol. 83(3), pages 523-541.
    7. Lindé, J. & Smets, F. & Wouters, R., 2016. "Challenges for Central Banks’ Macro Models," Handbook of Macroeconomics, in: J. B. Taylor & Harald Uhlig (ed.), Handbook of Macroeconomics, edition 1, volume 2, chapter 0, pages 2185-2262, Elsevier.
    8. Leitemo, Kai & Lonning, Ingunn, 2006. "Simple Monetary Policymaking without the Output Gap," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 38(6), pages 1619-1640, September.
    9. Lansing, Kevin J. & Trehan, Bharat, 2003. "Forward-looking behavior and optimal discretionary monetary policy," Economics Letters, Elsevier, vol. 81(2), pages 249-256, November.
    10. Taylor, John B. & Williams, John C., 2010. "Simple and Robust Rules for Monetary Policy," Handbook of Monetary Economics, in: Benjamin M. Friedman & Michael Woodford (ed.), Handbook of Monetary Economics, edition 1, volume 3, chapter 15, pages 829-859, Elsevier.
    11. Matteo Cacciatore & Dmitry Matveev & Rodrigo Sekkel, 2022. "Uncertainty and Monetary Policy Experimentation: Empirical Challenges and Insights from Academic Literature," Discussion Papers 2022-9, Bank of Canada.
    12. Lars E. O. Svensson, 2003. "What Is Wrong with Taylor Rules? Using Judgment in Monetary Policy through Targeting Rules," Journal of Economic Literature, American Economic Association, vol. 41(2), pages 426-477, June.
    13. Lars E. O. Svensson, 2003. "What Is Wrong with Taylor Rules? Using Judgment in Monetary Policy through Targeting Rules," Journal of Economic Literature, American Economic Association, vol. 41(2), pages 426-477, June.
    14. Michael Paetz, 2007. "Robust Control and Persistence in the New Keynesian Economy," Quantitative Macroeconomics Working Papers 20711, Hamburg University, Department of Economics.
    15. John B. Taylor & Volker Wieland, 2012. "Surprising Comparative Properties of Monetary Models: Results from a New Model Database," The Review of Economics and Statistics, MIT Press, vol. 94(3), pages 800-816, August.
    16. Lindé, Jesper & Smets, Frank & Wouters, Rafael, 2016. "Challenges for Central Banks´ Macro Models," Working Paper Series 323, Sveriges Riksbank (Central Bank of Sweden).
    17. Levin, Andrew T. & Williams, John C., 2003. "Robust monetary policy with competing reference models," Journal of Monetary Economics, Elsevier, vol. 50(5), pages 945-975, July.
    18. Aoki, Kosuke, 2003. "On the optimal monetary policy response to noisy indicators," Journal of Monetary Economics, Elsevier, vol. 50(3), pages 501-523, April.
    19. Volker Wieland, "undated". "Monetary Policy and Uncertainty about the Natural Unemployment Rate," Computing in Economics and Finance 1997 11, Society for Computational Economics.
    20. Gern, Klaus-Jürgen & Hauber, Philipp & Jannsen, Nils & Kooths, Stefan & Potjagailo, Galina & Wolters, Maik H., 2015. "Weltkonjunktur im Herbst 2015 - Schwäche in den Schwellenländern bremst Weltkonjunktur [Weakness in emerging markets weighs on global growth]," Kieler Konjunkturberichte 9, Kiel Institute for the World Economy (IfW Kiel).

    More about this item

    Keywords

    Optimal Monetary Policy; Reinforcement Learning; Artificial Neural Network; Machine Learning; Reaction Function;
    All these keywords.

    JEL classification:

    • C45 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods: Special Topics - - - Neural Networks and Related Topics
    • C61 - Mathematical and Quantitative Methods - - Mathematical Methods; Programming Models; Mathematical and Simulation Modeling - - - Optimization Techniques; Programming Models; Dynamic Analysis
    • E52 - Macroeconomics and Monetary Economics - - Monetary Policy, Central Banking, and the Supply of Money and Credit - - - Monetary Policy
    • E58 - Macroeconomics and Monetary Economics - - Monetary Policy, Central Banking, and the Supply of Money and Credit - - - Central Banks and Their Policies

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:zbw:bubdps:512021. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: ZBW - Leibniz Information Centre for Economics (email available below). General contact details of provider: https://edirc.repec.org/data/dbbgvde.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.