IDEAS home Printed from https://ideas.repec.org/p/hal/journl/hal-00732676.html
   My bibliography  Save this paper

The parameter set in an adaptive control Monte Carlo experiment: Some considerations

Author

Listed:
  • Marco P. Tucci

    (Dipartimento di Economia Politica - UNISI - Università degli Studi di Siena = University of Siena, Department of Economics - UTHealth - The University of Texas Health Science Center at Houston, Universiteit Utrecht / Utrecht University [Utrecht])

  • David A. Kendrick

    (Dipartimento di Economia Politica - UNISI - Università degli Studi di Siena = University of Siena, Department of Economics - UTHealth - The University of Texas Health Science Center at Houston, Universiteit Utrecht / Utrecht University [Utrecht])

  • Hans M. Amman

    (Dipartimento di Economia Politica - UNISI - Università degli Studi di Siena = University of Siena, Department of Economics - UTHealth - The University of Texas Health Science Center at Houston, Universiteit Utrecht / Utrecht University [Utrecht])

Abstract

Comparisons of various methods for solving stochastic control economic models can be done with Monte Carlo methods. These methods have been applied to simple one-state, one-control quadratic-linear tracking models; however, large outliers may occur in a substantial number of the Monte Carlo runs when certain parameter sets are used in these models. Building on the work of Mizrach (1991) and Amman and Kendrick (1994, 1995), this paper tracks the source of these outliers to two sources: (1) the use of a zero or the penalty weights on the control variables and (2) the generation of near-zero initial estimate of the control parameter in the systems equations by the Monte Carlo routine. This result leads to an understanding of why both the unsophisticated Optimal Feedback (Certainty Equivalence) and the sophisticated Dual methods do poorly in some Monte Carlo comparisons relative to the moderately sophisticated Expected Optimal Feedback method.

Suggested Citation

  • Marco P. Tucci & David A. Kendrick & Hans M. Amman, 2010. "The parameter set in an adaptive control Monte Carlo experiment: Some considerations," Post-Print hal-00732676, HAL.
    • Handle: RePEc:hal:journl:hal-00732676
    DOI: 10.1016/j.jedc.2010.06.014
    Note: View the original document on HAL open archive server: https://hal.science/hal-00732676
    as

    Download full text from publisher

    File URL: https://hal.science/hal-00732676/document
    Download Restriction: no
    File URL: https://libkey.io/10.1016/j.jedc.2010.06.014?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Mizrach, Bruce, 1991. "Nonconvexities in a stochastic control problem with learning," Journal of Economic Dynamics and Control, Elsevier, vol. 15(3), pages 515-538, July.
    2. Kendrick, David, 1978. "Non-convexities from probing in adaptive control problems," Economics Letters, Elsevier, vol. 1(4), pages 347-351.
    3. Marco Tucci & David Kendrick & Hans Amman, 2013. "Expected Optimal Feedback with Time-Varying Parameters," Computational Economics, Springer;Society for Computational Economics, vol. 42(3), pages 351-371, October.
    4. Beck, Gunter W. & Wieland, Volker, 2002. "Learning and control in a changing economic environment," Journal of Economic Dynamics and Control, Elsevier, vol. 26(9-10), pages 1359-1377, August.
    5. David Kendrick & Hans Amman, 2006. "A Classification System for Economic Stochastic Control Models," Computational Economics, Springer;Society for Computational Economics, vol. 27(4), pages 453-481, June.
    6. Tucci, Marco P, 1998. "The Nonconvexities Problem in Adaptive Control Models: A Simple Computational Solution," Computational Economics, Springer;Society for Computational Economics, vol. 12(3), pages 203-222, December.
    7. MacRae, Elizabeth Chase, 1975. "An Adaptive Learning Rule for Multiperiod Decision Problems," Econometrica, Econometric Society, vol. 43(5-6), pages 893-906, Sept.-Nov.
    8. Elizabeth Chase MacRae, 1972. "Linear Decision with Experimentation," NBER Chapters, in: Annals of Economic and Social Measurement, Volume 1, number 4, pages 437-447, National Bureau of Economic Research, Inc.
    9. Tucci, Marco P., 1997. "Adaptive control in the presence of time-varying parameters," Journal of Economic Dynamics and Control, Elsevier, vol. 22(1), pages 39-47, November.
    10. Amman, Hans M & Kendrick, David A, 1995. "Nonconvexities in Stochastic Control Models," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 36(2), pages 455-475, May.
    11. Alfred L. Norman & M. R. Norman & Carl Palash, 1979. "Multiple relative maxima in optimal macroeconomic policy: an illustration," Special Studies Papers 134, Board of Governors of the Federal Reserve System (U.S.).
    12. Cosimano, Thomas F., 2008. "Optimal experimentation and the perturbation method in the neighborhood of the augmented linear regulator problem," Journal of Economic Dynamics and Control, Elsevier, vol. 32(6), pages 1857-1894, June.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Peter John Robinson & W. J. Wouter Botzen & Fujin Zhou, 2021. "An experimental study of charity hazard: The effect of risky and ambiguous government compensation on flood insurance demand," Journal of Risk and Uncertainty, Springer, vol. 63(3), pages 275-318, December.
    2. D.A. Kendrick & H.M. Amman & M.P. Tucci, 2008. "Learning About Learning in Dynamic Economic Models," Working Papers 08-20, Utrecht School of Economics.
    3. Hans M. Amman & Marco P. Tucci, 2020. "How Active is Active Learning: Value Function Method Versus an Approximation Method," Computational Economics, Springer;Society for Computational Economics, vol. 56(3), pages 675-693, October.
    4. Kwang Mong Sim, 2023. "An Incentive-Compatible and Computationally Efficient Fog Bargaining Mechanism," Computational Economics, Springer;Society for Computational Economics, vol. 62(4), pages 1883-1918, December.
    5. D. Blueschke & V. Blueschke-Nikolaeva & R. Neck, 2013. "Stochastic Control of Linear and Nonlinear Econometric Models: Some Computational Aspects," Computational Economics, Springer;Society for Computational Economics, vol. 42(1), pages 107-118, June.
    6. Hans M. Amman & Marco Paolo Tucci, 2018. "How active is active learning: value function method vs an approximation method," Department of Economics University of Siena 788, Department of Economics, University of Siena.
    7. Hans Amman & David Kendrick, 2014. "Comparison of policy functions from the optimal learning and adaptive control frameworks," Computational Management Science, Springer, vol. 11(3), pages 221-235, July.
    8. Amman, Hans M. & Kendrick, David A. & Tucci, Marco P., 2020. "Approximating The Value Function For Optimal Experimentation," Macroeconomic Dynamics, Cambridge University Press, vol. 24(5), pages 1073-1086, July.
    9. H.M. Amman & D.A. Kendrick, 2012. "Conjectures on the policy function in the presence of optimal experimentation," Working Papers 12-09, Utrecht School of Economics.
    10. D. Blueschke & I. Savin & V. Blueschke-Nikolaeva, 2020. "An Evolutionary Approach to Passive Learning in Optimal Control Problems," Computational Economics, Springer;Society for Computational Economics, vol. 56(3), pages 659-673, October.
    11. Ivan Savin & Dmitri Blueschke, 2013. "Solving nonlinear stochastic optimal control problems using evolutionary heuristic optimization," Jena Economics Research Papers 2013-051, Friedrich-Schiller-University Jena.
    12. Ivan Savin & Dmitri Blueschke, 2016. "Lost in Translation: Explicitly Solving Nonlinear Stochastic Optimal Control Problems Using the Median Objective Value," Computational Economics, Springer;Society for Computational Economics, vol. 48(2), pages 317-338, August.
    13. Hans M. Amman & Marco P. Tucci, 2017. "The DUAL Approach in an Infinite Horizon Model," Department of Economics University of Siena 766, Department of Economics, University of Siena.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. D.A. Kendrick & H.M. Amman & M.P. Tucci, 2008. "Learning About Learning in Dynamic Economic Models," Working Papers 08-20, Utrecht School of Economics.
    2. repec:use:tkiwps:2020 is not listed on IDEAS
    3. Kendrick, David A., 2005. "Stochastic control for economic models: past, present and the paths ahead," Journal of Economic Dynamics and Control, Elsevier, vol. 29(1-2), pages 3-30, January.
    4. H.M. Amman & D.A. Kendrick, 2012. "Conjectures on the policy function in the presence of optimal experimentation," Working Papers 12-09, Utrecht School of Economics.
    5. Tucci, Marco P., 2002. "A note on global optimization in adaptive control, econometrics and macroeconomics," Journal of Economic Dynamics and Control, Elsevier, vol. 26(9-10), pages 1739-1764, August.
    6. Wieland, Volker, 2000. "Learning by doing and the value of optimal experimentation," Journal of Economic Dynamics and Control, Elsevier, vol. 24(4), pages 501-534, April.
    7. David Kendrick & Hans Amman, 2006. "A Classification System for Economic Stochastic Control Models," Computational Economics, Springer;Society for Computational Economics, vol. 27(4), pages 453-481, June.
    8. Amman, Hans M. & Kendrick, David A. & Tucci, Marco P., 2020. "Approximating The Value Function For Optimal Experimentation," Macroeconomic Dynamics, Cambridge University Press, vol. 24(5), pages 1073-1086, July.
    9. Bond, Craig A., 2008. "On the Potential Use of Adaptive Control Methods for Improving Adaptive Natural Resource Management," Working Papers 108721, Colorado State University, Department of Agricultural and Resource Economics.
    10. Hans M. Amman & Marco Paolo Tucci, 2018. "How active is active learning: value function method vs an approximation method," Department of Economics University of Siena 788, Department of Economics, University of Siena.
    11. Beck, Gunter W. & Wieland, Volker, 2002. "Learning and control in a changing economic environment," Journal of Economic Dynamics and Control, Elsevier, vol. 26(9-10), pages 1359-1377, August.
    12. Marco Tucci & David Kendrick & Hans Amman, 2013. "Expected Optimal Feedback with Time-Varying Parameters," Computational Economics, Springer;Society for Computational Economics, vol. 42(3), pages 351-371, October.
    13. Tim Willems, 2017. "Actively Learning by Pricing: A Model of an Experimenting Seller," Economic Journal, Royal Economic Society, vol. 127(604), pages 2216-2239, September.
    14. Hans M. Amman & Marco P. Tucci, 2020. "How Active is Active Learning: Value Function Method Versus an Approximation Method," Computational Economics, Springer;Society for Computational Economics, vol. 56(3), pages 675-693, October.
    15. Marco Tucci, 2006. "Understanding the Difference Between Robust Control and Optimal Control in a Linear Discrete-Time System with Time-Varying Parameters," Computational Economics, Springer;Society for Computational Economics, vol. 27(4), pages 533-558, June.
    16. Hans Amman & David Kendrick, 2014. "Comparison of policy functions from the optimal learning and adaptive control frameworks," Computational Management Science, Springer, vol. 11(3), pages 221-235, July.
    17. V. Blueschke-Nikolaeva & D. Blueschke & R. Neck, 2020. "OPTCON3: An Active Learning Control Algorithm for Nonlinear Quadratic Stochastic Problems," Computational Economics, Springer;Society for Computational Economics, vol. 56(1), pages 145-162, June.
    18. D. Blueschke & V. Blueschke-Nikolaeva & R. Neck, 2013. "Stochastic Control of Linear and Nonlinear Econometric Models: Some Computational Aspects," Computational Economics, Springer;Society for Computational Economics, vol. 42(1), pages 107-118, June.
    19. Cosimano, Thomas F., 2008. "Optimal experimentation and the perturbation method in the neighborhood of the augmented linear regulator problem," Journal of Economic Dynamics and Control, Elsevier, vol. 32(6), pages 1857-1894, June.
    20. In Chang Hwang, 2016. "Active learning and optimal climate policy," EcoMod2016 9611, EcoMod.
    21. Peter John Robinson & W. J. Wouter Botzen & Fujin Zhou, 2021. "An experimental study of charity hazard: The effect of risky and ambiguous government compensation on flood insurance demand," Journal of Risk and Uncertainty, Springer, vol. 63(3), pages 275-318, December.

    More about this item

    Keywords

    Active learning; Dual control; Optimal experimentation; Stochastic optimization; Time-varying parameters; Numerical experiments;
    All these keywords.

    JEL classification:

    • C63 - Mathematical and Quantitative Methods - - Mathematical Methods; Programming Models; Mathematical and Simulation Modeling - - - Computational Techniques
    • E61 - Macroeconomics and Monetary Economics - - Macroeconomic Policy, Macroeconomic Aspects of Public Finance, and General Outlook - - - Policy Objectives; Policy Designs and Consistency; Policy Coordination

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:hal:journl:hal-00732676. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: CCSD (email available below). General contact details of provider: https://hal.archives-ouvertes.fr/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.