IDEAS home Printed from https://ideas.repec.org/a/wly/navres/v56y2009i3p239-249.html
   My bibliography  Save this article

What you should know about approximate dynamic programming

Author

Listed:
  • Warren B. Powell

Abstract

Approximate dynamic programming (ADP) is a broad umbrella for a modeling and algorithmic strategy for solving problems that are sometimes large and complex, and are usually (but not always) stochastic. It is most often presented as a method for overcoming the classic curse of dimensionality that is well‐known to plague the use of Bellman's equation. For many problems, there are actually up to three curses of dimensionality. But the richer message of approximate dynamic programming is learning what to learn, and how to learn it, to make better decisions over time. This article provides a brief review of approximate dynamic programming, without intending to be a complete tutorial. Instead, our goal is to provide a broader perspective of ADP and how it should be approached from the perspective of different problem classes. © 2009 Wiley Periodicals, Inc. Naval Research Logistics 2009

Suggested Citation

  • Warren B. Powell, 2009. "What you should know about approximate dynamic programming," Naval Research Logistics (NRL), John Wiley & Sons, vol. 56(3), pages 239-249, April.
    • Handle: RePEc:wly:navres:v:56:y:2009:i:3:p:239-249
    DOI: 10.1002/nav.20347
    as

    Download full text from publisher

    File URL: https://doi.org/10.1002/nav.20347
    Download Restriction: no
    File URL: https://libkey.io/10.1002/nav.20347?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Michael C. Fu, 2002. "Feature Article: Optimization for simulation: Theory vs. Practice," INFORMS Journal on Computing, INFORMS, vol. 14(3), pages 192-215, August.
    2. Kenneth L. Judd, 1998. "Numerical Methods in Economics," MIT Press Books, The MIT Press, edition 1, volume 1, number 0262100711, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Alexandre Carbonneau & Fr'ed'eric Godin, 2021. "Deep Equal Risk Pricing of Financial Derivatives with Multiple Hedging Instruments," Papers 2102.12694, arXiv.org.
    2. Rempel, M. & Cai, J., 2021. "A review of approximate dynamic programming applications within military operations research," Operations Research Perspectives, Elsevier, vol. 8(C).
    3. Cervellera, Cristiano, 2023. "Optimized ensemble value function approximation for dynamic programming," European Journal of Operational Research, Elsevier, vol. 309(2), pages 719-730.
    4. Alexandra M. Newman & Martin Weiss, 2013. "A Survey of Linear and Mixed-Integer Optimization Tutorials," INFORMS Transactions on Education, INFORMS, vol. 14(1), pages 26-38, September.
    5. Xiao, Baichun & Yang, Wei, 2021. "A Bayesian learning model for estimating unknown demand parameter in revenue management," European Journal of Operational Research, Elsevier, vol. 293(1), pages 248-262.
    6. Daniel Egan & Qilun Zhu & Robert Prucka, 2023. "A Review of Reinforcement Learning-Based Powertrain Controllers: Effects of Agent Selection for Mixed-Continuity Control and Reward Formulation," Energies, MDPI, vol. 16(8), pages 1-31, April.
    7. Marynissen, Joren & Demeulemeester, Erik, 2019. "Literature review on multi-appointment scheduling problems in hospitals," European Journal of Operational Research, Elsevier, vol. 272(2), pages 407-419.
    8. Nikola Mardešić & Tomislav Erdelić & Tonči Carić & Marko Đurasević, 2023. "Review of Stochastic Dynamic Vehicle Routing in the Evolving Urban Logistics Environment," Mathematics, MDPI, vol. 12(1), pages 1-44, December.
    9. Rossi, Roberto & Tomasella, Maurizio & Martin-Barragan, Belen & Embley, Tim & Walsh, Christopher & Langston, Matthew, 2019. "The Dynamic Bowser Routing Problem," European Journal of Operational Research, Elsevier, vol. 275(1), pages 108-126.
    10. Carbonneau, Alexandre, 2021. "Deep hedging of long-term financial derivatives," Insurance: Mathematics and Economics, Elsevier, vol. 99(C), pages 327-340.
    11. Alexandre Carbonneau, 2020. "Deep Hedging of Long-Term Financial Derivatives," Papers 2007.15128, arXiv.org.
    12. Lauer, Christopher J. & Montgomery, Claire A. & Dietterich, Thomas G., 2017. "Spatial interactions and optimal forest management on a fire-threatened landscape," Forest Policy and Economics, Elsevier, vol. 83(C), pages 107-120.
    13. Satic, U. & Jacko, P. & Kirkbride, C., 2024. "A simulation-based approximate dynamic programming approach to dynamic and stochastic resource-constrained multi-project scheduling problem," European Journal of Operational Research, Elsevier, vol. 315(2), pages 454-469.
    14. Gökalp, E. & Gülpınar, N. & Doan, X.V., 2023. "Dynamic surgery management under uncertainty," European Journal of Operational Research, Elsevier, vol. 309(2), pages 832-844.
    15. Mojtaba Heydar & Małgorzata M. O’Reilly & Erin Trainer & Mark Fackrell & Peter G. Taylor & Ali Tirdad, 2022. "A stochastic model for the patient-bed assignment problem with random arrivals and departures," Annals of Operations Research, Springer, vol. 315(2), pages 813-845, August.
    16. Christopher Dance & Alexei Gaivoronski, 2012. "Stochastic optimization for real time service capacity allocation under random service demand," Annals of Operations Research, Springer, vol. 193(1), pages 221-253, March.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Powell, Warren B., 2019. "A unified framework for stochastic optimization," European Journal of Operational Research, Elsevier, vol. 275(3), pages 795-821.
    2. Francisco Gallego & Andrés Hernando, 2009. "School Choice in Chile: Looking at the Demand Side," Documentos de Trabajo 356, Instituto de Economia. Pontificia Universidad Católica de Chile..
    3. Noordhoek, Marije & Dullaert, Wout & Lai, David S.W. & de Leeuw, Sander, 2018. "A simulation–optimization approach for a service-constrained multi-echelon distribution network," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 114(C), pages 292-311.
    4. Salerno, Gillian & Beard, Rodney & McDonald, Stuart, 2007. "Rent Seeking Behavior and Optimal Taxation of Pollution in Shallow Lakes," MPRA Paper 11225, University Library of Munich, Germany, revised 22 Oct 2008.
    5. Maria Casanova-Rivas, 2008. "Dynamic Complementarities: A Computational and Empirical Analysis of Couples' Retirement Decisions," 2008 Meeting Papers 1073, Society for Economic Dynamics.
    6. Andreas Lanz & Gregor Reich & Ole Wilms, 2022. "Adaptive grids for the estimation of dynamic models," Quantitative Marketing and Economics (QME), Springer, vol. 20(2), pages 179-238, June.
    7. Karantounias, Anastasios G., 2023. "Doubts about the model and optimal policy," Journal of Economic Theory, Elsevier, vol. 210(C).
    8. Pelin Ilbas, 2006. "Optimal Monetary Policy rules for the Euro area in a DSGE framework," Working Papers of Department of Economics, Leuven ces0613, KU Leuven, Faculty of Economics and Business (FEB), Department of Economics, Leuven.
    9. Atanas Christev, 2006. "Learning Hyperinflations," Computing in Economics and Finance 2006 475, Society for Computational Economics.
    10. Borovička, Jaroslav & Hansen, Lars Peter, 2014. "Examining macroeconomic models through the lens of asset pricing," Journal of Econometrics, Elsevier, vol. 183(1), pages 67-90.
    11. Nikolaj Malchow-Møller & Michael Svarer, 2003. "Estimation of the multinomial logit model with random effects," Applied Economics Letters, Taylor & Francis Journals, vol. 10(7), pages 389-392.
    12. Röhrs, Sigrid & Winter, Christoph, 2017. "Reducing government debt in the presence of inequality," Journal of Economic Dynamics and Control, Elsevier, vol. 82(C), pages 1-20.
    13. Gomme, Paul & Klein, Paul, 2011. "Second-order approximation of dynamic models without the use of tensors," Journal of Economic Dynamics and Control, Elsevier, vol. 35(4), pages 604-615, April.
    14. Wieland, Volker, 2000. "Monetary policy, parameter uncertainty and optimal learning," Journal of Monetary Economics, Elsevier, vol. 46(1), pages 199-228, August.
    15. Adam, Klaus & Billi, Roberto M., 2006. "Optimal Monetary Policy under Commitment with a Zero Bound on Nominal Interest Rates," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 38(7), pages 1877-1905, October.
    16. Levine, Paul & Pearlman, Joseph & Pierse, Richard, 2008. "Linear-quadratic approximation, external habit and targeting rules," Journal of Economic Dynamics and Control, Elsevier, vol. 32(10), pages 3315-3349, October.
    17. Manuel Arellano & Stéphane Bonhomme, 2017. "Quantile Selection Models With an Application to Understanding Changes in Wage Inequality," Econometrica, Econometric Society, vol. 85, pages 1-28, January.
    18. Hendrik Thiel & Stephan L. Thomsen, 2015. "Individual Poverty Paths and the Stability of Control-Perception," SOEPpapers on Multidisciplinary Panel Data Research 794, DIW Berlin, The German Socio-Economic Panel (SOEP).
    19. Hugo Benítez-Silva & Eva Cárceles-Poveda & Selçuk Eren, 2011. "Effects of Legal and Unauthorized Immigration on the U.S. Social Security System," Working Papers wp250, University of Michigan, Michigan Retirement Research Center.
    20. Miao, Jianjun & Wang, Neng, 2007. "Investment, consumption, and hedging under incomplete markets," Journal of Financial Economics, Elsevier, vol. 86(3), pages 608-642, December.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wly:navres:v:56:y:2009:i:3:p:239-249. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://doi.org/10.1002/(ISSN)1520-6750 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.