IDEAS home Printed from https://ideas.repec.org/a/eee/dyncon/v32y2008i6p1857-1894.html
   My bibliography  Save this article

Optimal experimentation and the perturbation method in the neighborhood of the augmented linear regulator problem

Author

Listed:
  • Cosimano, Thomas F.

Abstract

The perturbation method is used to approximate optimal experimentation problems. The approximation is in the neighborhood of the linear regulator (LR) problem. The first order perturbation of the optimal decision under experimentation is a combination of the LR solution and a term that captures the impact of the uncertainty on the agent's value function. An algorithm is developed in a companion paper to quickly implement this procedure on the computer. As a result, the impact of optimal experimentation on an agent's decisions can be quantified and estimated for a large class of problems encountered in economics.

Suggested Citation

  • Cosimano, Thomas F., 2008. "Optimal experimentation and the perturbation method in the neighborhood of the augmented linear regulator problem," Journal of Economic Dynamics and Control, Elsevier, vol. 32(6), pages 1857-1894, June.
    • Handle: RePEc:eee:dyncon:v:32:y:2008:i:6:p:1857-1894
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0165-1889(07)00174-1
    Download Restriction: Full text for ScienceDirect subscribers only
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Easley, David & Kiefer, Nicholas M, 1988. "Controlling a Stochastic Process with Unknown Parameters," Econometrica, Econometric Society, vol. 56(5), pages 1045-1064, September.
    2. Godfrey Keller & Sven Rady, 1999. "Optimal Experimentation in a Changing Environment," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 66(3), pages 475-507.
    3. Philippe Aghion & Patrick Bolton & Christopher Harris & Bruno Jullien, 1991. "Optimal Learning by Experimentation," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 58(4), pages 621-654.
    4. Wieland, Volker, 2000. "Monetary policy, parameter uncertainty and optimal learning," Journal of Monetary Economics, Elsevier, vol. 46(1), pages 199-228, August.
    5. Chen, Yu & Cosimano, Thomas F. & Himonas, Alex A., 2008. "Analytic solving of asset pricing models: The by force of habit case," Journal of Economic Dynamics and Control, Elsevier, vol. 32(11), pages 3631-3660, November.
    6. Ronald J. Balvers & Thomas F. Cosimano, 1994. "Inflation Variability and Gradualist Monetary Policy," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 61(4), pages 721-738.
    7. Varian, Hal R, 1980. "A Model of Sales," American Economic Review, American Economic Association, vol. 70(4), pages 651-659, September.
    8. Marcet, Albert & Sargent, Thomas J., 1989. "Convergence of least squares learning mechanisms in self-referential linear stochastic models," Journal of Economic Theory, Elsevier, vol. 48(2), pages 337-368, August.
    9. Kenneth L. Judd, 1998. "Numerical Methods in Economics," MIT Press Books, The MIT Press, edition 1, volume 1, number 0262100711, April.
    10. Orphanides, Athanasios & Williams, John C., 2005. "Expectations, learning and monetary policy," Journal of Economic Dynamics and Control, Elsevier, vol. 29(11), pages 1807-1808, November.
    11. Anderson, Evan W. & McGrattan, Ellen R. & Hansen, Lars Peter & Sargent, Thomas J., 1996. "Mechanics of forming and estimating dynamic linear economies," Handbook of Computational Economics, in: H. M. Amman & D. A. Kendrick & J. Rust (ed.), Handbook of Computational Economics, edition 1, volume 1, chapter 4, pages 171-252, Elsevier.
    12. Paul Klemperer, 1995. "Competition when Consumers have Switching Costs: An Overview with Applications to Industrial Organization, Macroeconomics, and International Trade," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 62(4), pages 515-539.
    13. Patrick Bolton & Christopher Harris, 1999. "Strategic Experimentation," Econometrica, Econometric Society, vol. 67(2), pages 349-374, March.
    14. Volker Wieland, "undated". "Monetary Policy and Uncertainty about the Natural Unemployment Rate," Computing in Economics and Finance 1997 11, Society for Computational Economics.
    15. Trefler, Daniel, 1993. "The Ignorant Monopolist: Optimal Learning with Endogenous Information," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 34(3), pages 565-581, August.
    16. Brock, William A. & Durlauf, Steven N., 2005. "Local robustness analysis: Theory and application," Journal of Economic Dynamics and Control, Elsevier, vol. 29(11), pages 2067-2092, November.
    17. Prescott, Edward C, 1972. "The Multi-Period Control Problem Under Uncertainty," Econometrica, Econometric Society, vol. 40(6), pages 1043-1058, November.
    18. Gaspar, Jess & L. Judd, Kenneth, 1997. "Solving Large-Scale Rational-Expectations Models," Macroeconomic Dynamics, Cambridge University Press, vol. 1(1), pages 45-75, January.
    19. Kiefer, Nicholas M & Nyarko, Yaw, 1989. "Optimal Control of an Unknown Linear Process with Learning," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 30(3), pages 571-586, August.
    20. Balvers, Ronald J. & Cosimano, Thomas F., 1993. "Periodic learning about a hidden state variable," Journal of Economic Dynamics and Control, Elsevier, vol. 17(5-6), pages 805-827.
    21. William A. Brock & Steven N. Durlauf & Kenneth D. West, 2003. "Policy Evaluation in Uncertain Economic Environments," Brookings Papers on Economic Activity, Economic Studies Program, The Brookings Institution, vol. 34(1), pages 235-322.
    22. Wieland, Volker, 2000. "Learning by doing and the value of optimal experimentation," Journal of Economic Dynamics and Control, Elsevier, vol. 24(4), pages 501-534, April.
    23. Kendrick, David A., 2005. "Stochastic control for economic models: past, present and the paths ahead," Journal of Economic Dynamics and Control, Elsevier, vol. 29(1-2), pages 3-30, January.
    24. McGrattan, Ellen R., 1994. "A note on computing competitive equilibria in linear models," Journal of Economic Dynamics and Control, Elsevier, vol. 18(1), pages 149-160, January.
    25. Lars Peter Hansen & Thomas J. Sargent, 2001. "Acknowledging Misspecification in Macroeconomic Theory," Review of Economic Dynamics, Elsevier for the Society for Economic Dynamics, vol. 4(3), pages 519-535, July.
    26. Beck, Gunter W. & Wieland, Volker, 2002. "Learning and control in a changing economic environment," Journal of Economic Dynamics and Control, Elsevier, vol. 26(9-10), pages 1359-1377, August.
    27. Santos, Manuel S., 1994. "Smooth dynamics and computation in models of economic growth," Journal of Economic Dynamics and Control, Elsevier, vol. 18(3-4), pages 879-895.
    28. Wieland Volker, 2006. "Monetary Policy and Uncertainty about the Natural Unemployment Rate: Brainard-Style Conservatism versus Experimental Activism," The B.E. Journal of Macroeconomics, De Gruyter, vol. 6(1), pages 1-34, March.
    29. Giuseppe Moscarini & Lones Smith, 2001. "The Optimal Level of Experimentation," Econometrica, Econometric Society, vol. 69(6), pages 1629-1644, November.
    30. Hansen, Lars Peter & Sargent, Thomas J., 2003. "Robust control of forward-looking models," Journal of Monetary Economics, Elsevier, vol. 50(3), pages 581-604, April.
    31. MacRae, Elizabeth Chase, 1975. "An Adaptive Learning Rule for Multiperiod Decision Problems," Econometrica, Econometric Society, vol. 43(5-6), pages 893-906, Sept.-Nov.
    32. Elizabeth Chase MacRae, 1972. "Linear Decision with Experimentation," NBER Chapters, in: Annals of Economic and Social Measurement, Volume 1, number 4, pages 437-447, National Bureau of Economic Research, Inc.
    33. Balvers, Ronald J & Cosimano, Thomas F, 1990. "Actively Learning about Demand and the Dynamics of Price Adjustment," Economic Journal, Royal Economic Society, vol. 100(402), pages 882-898, September.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Hans M. Amman & Marco Paolo Tucci, 2018. "How active is active learning: value function method vs an approximation method," Department of Economics University of Siena 788, Department of Economics, University of Siena.
    2. Tucci, Marco P. & Kendrick, David A. & Amman, Hans M., 2010. "The parameter set in an adaptive control Monte Carlo experiment: Some considerations," Journal of Economic Dynamics and Control, Elsevier, vol. 34(9), pages 1531-1549, September.
    3. Hans Amman & David Kendrick, 2014. "Comparison of policy functions from the optimal learning and adaptive control frameworks," Computational Management Science, Springer, vol. 11(3), pages 221-235, July.
    4. Amman, Hans M. & Kendrick, David A. & Tucci, Marco P., 2020. "Approximating The Value Function For Optimal Experimentation," Macroeconomic Dynamics, Cambridge University Press, vol. 24(5), pages 1073-1086, July.
    5. Brechet, Thierry & HRITONENKO, Natali & YATSENKO, Yuri, 2010. "Adaptation and mitigation in long-term climate policies," LIDAM Discussion Papers CORE 2010065, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
    6. Hans M. Amman & Marco P. Tucci, 2020. "How Active is Active Learning: Value Function Method Versus an Approximation Method," Computational Economics, Springer;Society for Computational Economics, vol. 56(3), pages 675-693, October.
    7. H.M. Amman & D.A. Kendrick, 2012. "Conjectures on the policy function in the presence of optimal experimentation," Working Papers 12-09, Utrecht School of Economics.
    8. Thierry Bréchet & Natali Hritonenko & Yuri Yatsenko, 2013. "Adaptation and Mitigation in Long-term Climate Policy," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 55(2), pages 217-243, June.
    9. Dia, Enzo, 2013. "How do banks respond to shocks? A dynamic model of deposit-taking institutions," Journal of Banking & Finance, Elsevier, vol. 37(9), pages 3623-3638.
    10. Tim Willems, 2017. "Actively Learning by Pricing: A Model of an Experimenting Seller," Economic Journal, Royal Economic Society, vol. 127(604), pages 2216-2239, September.
    11. In Chang Hwang, 2016. "Active learning and optimal climate policy," EcoMod2016 9611, EcoMod.
    12. Ivan Savin & Dmitri Blueschke, 2013. "Solving nonlinear stochastic optimal control problems using evolutionary heuristic optimization," Jena Economics Research Papers 2013-051, Friedrich-Schiller-University Jena.
    13. Ivan Savin & Dmitri Blueschke, 2016. "Lost in Translation: Explicitly Solving Nonlinear Stochastic Optimal Control Problems Using the Median Objective Value," Computational Economics, Springer;Society for Computational Economics, vol. 48(2), pages 317-338, August.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Tim Willems, 2017. "Actively Learning by Pricing: A Model of an Experimenting Seller," Economic Journal, Royal Economic Society, vol. 127(604), pages 2216-2239, September.
    2. Hans M. Amman & Marco Paolo Tucci, 2018. "How active is active learning: value function method vs an approximation method," Department of Economics University of Siena 788, Department of Economics, University of Siena.
    3. Amman, Hans M. & Kendrick, David A. & Tucci, Marco P., 2020. "Approximating The Value Function For Optimal Experimentation," Macroeconomic Dynamics, Cambridge University Press, vol. 24(5), pages 1073-1086, July.
    4. Hans M. Amman & Marco P. Tucci, 2020. "How Active is Active Learning: Value Function Method Versus an Approximation Method," Computational Economics, Springer;Society for Computational Economics, vol. 56(3), pages 675-693, October.
    5. Beck, Gunter W. & Wieland, Volker, 2002. "Learning and control in a changing economic environment," Journal of Economic Dynamics and Control, Elsevier, vol. 26(9-10), pages 1359-1377, August.
    6. Wieland, Volker, 2000. "Learning by doing and the value of optimal experimentation," Journal of Economic Dynamics and Control, Elsevier, vol. 24(4), pages 501-534, April.
    7. Koulovatianos, Christos & Mirman, Leonard J. & Santugini, Marc, 2009. "Optimal growth and uncertainty: Learning," Journal of Economic Theory, Elsevier, vol. 144(1), pages 280-295, January.
    8. Cunha-e-Sa, Maria A. & Santos, Vasco, 2008. "Experimentation with accumulation," Journal of Economic Dynamics and Control, Elsevier, vol. 32(2), pages 470-496, February.
    9. Kendrick, David A., 2005. "Stochastic control for economic models: past, present and the paths ahead," Journal of Economic Dynamics and Control, Elsevier, vol. 29(1-2), pages 3-30, January.
    10. H.M. Amman & D.A. Kendrick, 2012. "Conjectures on the policy function in the presence of optimal experimentation," Working Papers 12-09, Utrecht School of Economics.
    11. David Kendrick & Hans Amman, 2006. "A Classification System for Economic Stochastic Control Models," Computational Economics, Springer;Society for Computational Economics, vol. 27(4), pages 453-481, June.
    12. Maria Antonieta Cunha-e-Sa & Vasco Santos, 2007. "Experimentation with accumulation," Nova SBE Working Paper Series wp503, Universidade Nova de Lisboa, Nova School of Business and Economics.
    13. D.A. Kendrick & H.M. Amman & M.P. Tucci, 2008. "Learning About Learning in Dynamic Economic Models," Working Papers 08-20, Utrecht School of Economics.
    14. Wieland, Volker, 2000. "Monetary policy, parameter uncertainty and optimal learning," Journal of Monetary Economics, Elsevier, vol. 46(1), pages 199-228, August.
    15. Volker Wieland, "undated". "Monetary Policy and Uncertainty about the Natural Unemployment Rate," Computing in Economics and Finance 1997 11, Society for Computational Economics.
    16. repec:use:tkiwps:2020 is not listed on IDEAS
    17. Leonard J. Mirman & Kevin Reffett & Marc Santugini, 2016. "On learning and growth," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 61(4), pages 641-684, April.
    18. In Chang Hwang, 2016. "Active learning and optimal climate policy," EcoMod2016 9611, EcoMod.
    19. Mewael F. Tesfaselassie & Eric Schaling & Sylvester Eijffinger, 2011. "Learning about the Term Structure and Optimal Rules for Inflation Targeting," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 43(8), pages 1685-1706, December.
    20. Tesfaselassie, M.F. & Schaling, E., 2010. "Managing disinflation under uncertainty," Journal of Economic Dynamics and Control, Elsevier, vol. 34(12), pages 2568-2577, December.
    21. Bullard, James & Suda, Jacek, 2016. "The stability of macroeconomic systems with Bayesian learners," Journal of Economic Dynamics and Control, Elsevier, vol. 62(C), pages 1-16.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:dyncon:v:32:y:2008:i:6:p:1857-1894. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/jedc .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.