IDEAS home Printed from https://ideas.repec.org/p/usi/wpaper/788.html
   My bibliography  Save this paper

How active is active learning: value function method vs an approximation method

Author

Listed:
  • Hans M. Amman
  • Marco Paolo Tucci

Abstract

In a previous paper Amman and Tucci (2018) compare the two dominant approaches for solving models with optimal experimentation (also called active learning), i.e. the value function and the approximation method. By using the same model and dataset as in Beck and Wieland (2002), theyfind that the approximation method produces solutions close to those generated by the value function approach and identify some elements of the model specifications which affect the difference between the two solutions. They conclude that differences are small when the effects of learning are limited. However the dataset used in the experiment describes a situation where the controller is dealing with a nonstationary process and there is no penalty on the control. The goal of this paper is to see if their conclusions hold in the more commonly studied case of a controller facing a stationary process and a positive penalty on the control.

Suggested Citation

  • Hans M. Amman & Marco Paolo Tucci, 2018. "How active is active learning: value function method vs an approximation method," Department of Economics University of Siena 788, Department of Economics, University of Siena.
  • Handle: RePEc:usi:wpaper:788
    as

    Download full text from publisher

    File URL: http://repec.deps.unisi.it/quaderni/788.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Philippe Aghion & Patrick Bolton & Christopher Harris & Bruno Jullien, 1991. "Optimal Learning by Experimentation," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 58(4), pages 621-654.
    2. Wieland, Volker, 2000. "Monetary policy, parameter uncertainty and optimal learning," Journal of Monetary Economics, Elsevier, vol. 46(1), pages 199-228, August.
    3. Andrew Levin & Volker Wieland & John C. Williams, 2003. "The Performance of Forecast-Based Monetary Policy Rules Under Model Uncertainty," American Economic Review, American Economic Association, vol. 93(3), pages 622-645, June.
    4. Hans M. Amman & Marco P. Tucci, 2017. "The DUAL Approach in an Infinite Horizon Model," Department of Economics University of Siena 766, Department of Economics, University of Siena.
    5. Coenen, Gunter & Levin, Andrew & Wieland, Volker, 2005. "Data uncertainty and the role of money as an information variable for monetary policy," European Economic Review, Elsevier, vol. 49(4), pages 975-1006, May.
    6. Tucci, Marco P. & Kendrick, David A. & Amman, Hans M., 2010. "The parameter set in an adaptive control Monte Carlo experiment: Some considerations," Journal of Economic Dynamics and Control, Elsevier, vol. 34(9), pages 1531-1549, September.
    7. Patrick Bolton & Christopher Harris, 1999. "Strategic Experimentation," Econometrica, Econometric Society, vol. 67(2), pages 349-374, March.
    8. Kiefer, Nicholas M., 1989. "A value function arising in the economics of information," Journal of Economic Dynamics and Control, Elsevier, vol. 13(2), pages 201-223, April.
    9. Wieland, Volker, 2000. "Learning by doing and the value of optimal experimentation," Journal of Economic Dynamics and Control, Elsevier, vol. 24(4), pages 501-534, April.
    10. Amman, Hans, 1996. "Numerical methods for linear-quadratic models," Handbook of Computational Economics, in: H. M. Amman & D. A. Kendrick & J. Rust (ed.), Handbook of Computational Economics, edition 1, volume 1, chapter 13, pages 587-618, Elsevier.
    11. Beck, Gunter W. & Wieland, Volker, 2002. "Learning and control in a changing economic environment," Journal of Economic Dynamics and Control, Elsevier, vol. 26(9-10), pages 1359-1377, August.
    12. Francisco J. Buera & Alexander Monge‐Naranjo & Giorgio E. Primiceri, 2011. "Learning the Wealth of Nations," Econometrica, Econometric Society, vol. 79(1), pages 1-45, January.
    13. Ivan Savin & Dmitri Blueschke, 2016. "Lost in Translation: Explicitly Solving Nonlinear Stochastic Optimal Control Problems Using the Median Objective Value," Computational Economics, Springer;Society for Computational Economics, vol. 48(2), pages 317-338, August.
    14. Giuseppe Moscarini & Lones Smith, 2001. "The Optimal Level of Experimentation," Econometrica, Econometric Society, vol. 69(6), pages 1629-1644, November.
    15. MacRae, Elizabeth Chase, 1975. "An Adaptive Learning Rule for Multiperiod Decision Problems," Econometrica, Econometric Society, vol. 43(5-6), pages 893-906, Sept.-Nov.
    16. Elizabeth Chase MacRae, 1972. "Linear Decision with Experimentation," NBER Chapters, in: Annals of Economic and Social Measurement, Volume 1, number 4, pages 437-447, National Bureau of Economic Research, Inc.
    17. H. M. Amman & D. A. Kendrick & J. Rust (ed.), 1996. "Handbook of Computational Economics," Handbook of Computational Economics, Elsevier, edition 1, volume 1, number 1.
    18. Easley, David & Kiefer, Nicholas M, 1988. "Controlling a Stochastic Process with Unknown Parameters," Econometrica, Econometric Society, vol. 56(5), pages 1045-1064, September.
    19. Timothy C. Salmon, 2001. "An Evaluation of Econometric Models of Adaptive Learning," Econometrica, Econometric Society, vol. 69(6), pages 1597-1628, November.
    20. Cosimano, Thomas F., 2008. "Optimal experimentation and the perturbation method in the neighborhood of the augmented linear regulator problem," Journal of Economic Dynamics and Control, Elsevier, vol. 32(6), pages 1857-1894, June.
    21. Prescott, Edward C, 1972. "The Multi-Period Control Problem Under Uncertainty," Econometrica, Econometric Society, vol. 40(6), pages 1043-1058, November.
    22. Kiefer, Nicholas M & Nyarko, Yaw, 1989. "Optimal Control of an Unknown Linear Process with Learning," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 30(3), pages 571-586, August.
    23. Amman, Hans M. & Kendrick, David A. & Tucci, Marco P., 2020. "Approximating The Value Function For Optimal Experimentation," Macroeconomic Dynamics, Cambridge University Press, vol. 24(5), pages 1073-1086, July.
    24. Taylor, John B, 1974. "Asymptotic Properties of Multiperiod Control Rules in the Linear Regression Model," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 15(2), pages 472-484, June.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. D. Blueschke & I. Savin & V. Blueschke-Nikolaeva, 2020. "An Evolutionary Approach to Passive Learning in Optimal Control Problems," Computational Economics, Springer;Society for Computational Economics, vol. 56(3), pages 659-673, October.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Hans M. Amman & Marco P. Tucci, 2020. "How Active is Active Learning: Value Function Method Versus an Approximation Method," Computational Economics, Springer;Society for Computational Economics, vol. 56(3), pages 675-693, October.
    2. Amman, Hans M. & Kendrick, David A. & Tucci, Marco P., 2020. "Approximating The Value Function For Optimal Experimentation," Macroeconomic Dynamics, Cambridge University Press, vol. 24(5), pages 1073-1086, July.
    3. H.M. Amman & D.A. Kendrick, 2012. "Conjectures on the policy function in the presence of optimal experimentation," Working Papers 12-09, Utrecht School of Economics.
    4. Kendrick, David A., 2005. "Stochastic control for economic models: past, present and the paths ahead," Journal of Economic Dynamics and Control, Elsevier, vol. 29(1-2), pages 3-30, January.
    5. Cosimano, Thomas F., 2008. "Optimal experimentation and the perturbation method in the neighborhood of the augmented linear regulator problem," Journal of Economic Dynamics and Control, Elsevier, vol. 32(6), pages 1857-1894, June.
    6. D.A. Kendrick & H.M. Amman & M.P. Tucci, 2008. "Learning About Learning in Dynamic Economic Models," Working Papers 08-20, Utrecht School of Economics.
    7. Hans Amman & David Kendrick, 2014. "Comparison of policy functions from the optimal learning and adaptive control frameworks," Computational Management Science, Springer, vol. 11(3), pages 221-235, July.
    8. repec:use:tkiwps:2020 is not listed on IDEAS
    9. Beck, Gunter W. & Wieland, Volker, 2002. "Learning and control in a changing economic environment," Journal of Economic Dynamics and Control, Elsevier, vol. 26(9-10), pages 1359-1377, August.
    10. David Kendrick & Hans Amman, 2006. "A Classification System for Economic Stochastic Control Models," Computational Economics, Springer;Society for Computational Economics, vol. 27(4), pages 453-481, June.
    11. Tim Willems, 2017. "Actively Learning by Pricing: A Model of an Experimenting Seller," Economic Journal, Royal Economic Society, vol. 127(604), pages 2216-2239, September.
    12. In Chang Hwang, 2016. "Active learning and optimal climate policy," EcoMod2016 9611, EcoMod.
    13. Peter John Robinson & W. J. Wouter Botzen & Fujin Zhou, 2021. "An experimental study of charity hazard: The effect of risky and ambiguous government compensation on flood insurance demand," Journal of Risk and Uncertainty, Springer, vol. 63(3), pages 275-318, December.
    14. Volker Wieland, "undated". "Monetary Policy and Uncertainty about the Natural Unemployment Rate," Computing in Economics and Finance 1997 11, Society for Computational Economics.
    15. Cunha-e-Sa, Maria A. & Santos, Vasco, 2008. "Experimentation with accumulation," Journal of Economic Dynamics and Control, Elsevier, vol. 32(2), pages 470-496, February.
    16. Wieland, Volker, 2000. "Learning by doing and the value of optimal experimentation," Journal of Economic Dynamics and Control, Elsevier, vol. 24(4), pages 501-534, April.
    17. Maria Antonieta Cunha-e-Sa & Vasco Santos, 2007. "Experimentation with accumulation," Nova SBE Working Paper Series wp503, Universidade Nova de Lisboa, Nova School of Business and Economics.
    18. Koulovatianos, Christos & Mirman, Leonard J. & Santugini, Marc, 2009. "Optimal growth and uncertainty: Learning," Journal of Economic Theory, Elsevier, vol. 144(1), pages 280-295, January.
    19. Wieland, Volker, 2000. "Monetary policy, parameter uncertainty and optimal learning," Journal of Monetary Economics, Elsevier, vol. 46(1), pages 199-228, August.
    20. Tesfaselassie, M.F. & Schaling, E., 2010. "Managing disinflation under uncertainty," Journal of Economic Dynamics and Control, Elsevier, vol. 34(12), pages 2568-2577, December.
    21. Bullard, James & Suda, Jacek, 2016. "The stability of macroeconomic systems with Bayesian learners," Journal of Economic Dynamics and Control, Elsevier, vol. 62(C), pages 1-16.

    More about this item

    Keywords

    Optimal experimentation; value function; approximation method; adaptive control; active learning; time-varying parameters; numerical experiments.;
    All these keywords.

    JEL classification:

    • C63 - Mathematical and Quantitative Methods - - Mathematical Methods; Programming Models; Mathematical and Simulation Modeling - - - Computational Techniques
    • E61 - Macroeconomics and Monetary Economics - - Macroeconomic Policy, Macroeconomic Aspects of Public Finance, and General Outlook - - - Policy Objectives; Policy Designs and Consistency; Policy Coordination
    • E62 - Macroeconomics and Monetary Economics - - Macroeconomic Policy, Macroeconomic Aspects of Public Finance, and General Outlook - - - Fiscal Policy; Modern Monetary Theory

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:usi:wpaper:788. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Fabrizio Becatti (email available below). General contact details of provider: https://edirc.repec.org/data/desieit.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.