An Approximate Dynamic Programming Algorithm for Monotone Value Functions
Author
Abstract
Suggested Citation
DOI: 10.1287/opre.2015.1425
Download full text from publisher
References listed on IDEAS
- Rust, John, 1987. "Optimal Replacement of GMC Bus Engines: An Empirical Model of Harold Zurcher," Econometrica, Econometric Society, vol. 55(5), pages 999-1033, September.
- J. O. Ramsay, 1998. "Estimating smooth monotone functions," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 60(2), pages 365-375.
- Feldstein, Martin S & Rothschild, Michael, 1974. "Towards an Economic Theory of Replacement Investment," Econometrica, Econometric Society, vol. 42(3), pages 393-423, May.
- J. J. McCall, 1970. "Economics of Information and Job Search," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 84(1), pages 113-126.
- Juliana M. Nascimento & Warren B. Powell, 2009. "An Optimal Approximate Dynamic Programming Algorithm for the Lagged Asset Acquisition Problem," Mathematics of Operations Research, INFORMS, vol. 34(1), pages 210-237, February.
- Greg Kaplan & Giovanni L. Violante, 2014.
"A Model of the Consumption Response to Fiscal Stimulus Payments,"
Econometrica, Econometric Society, vol. 82(4), pages 1199-1239, July.
- Greg Kaplan & Giovanni L. Violante, 2011. "A Model of the Consumption Response to Fiscal Stimulus Payments," NBER Working Papers 17338, National Bureau of Economic Research, Inc.
- Gianluca Violante & Greg Kaplan, 2011. "A Model of the Consumption Response to Fiscal Stimulus Payments," 2011 Meeting Papers 243, Society for Economic Dynamics.
- Violante, Giovanni & Kaplan, Greg, 2011. "A Model of the Consumption Response to Fiscal Stimulus Payments," CEPR Discussion Papers 8562, C.E.P.R. Discussion Papers.
- Alfred Müller, 1997. "How Does the Value Function of a Markov Decision Process Depend on the Transition Probabilities?," Mathematics of Operations Research, INFORMS, vol. 22(4), pages 872-885, November.
- Warren Powell & Andrzej Ruszczyński & Huseyin Topaloglu, 2004. "Learning Algorithms for Separable Approximations of Discrete Stochastic Optimization Problems," Mathematics of Operations Research, INFORMS, vol. 29(4), pages 814-836, November.
- Nicola Secomandi, 2010. "Optimal Commodity Trading with a Capacitated Storage Asset," Management Science, INFORMS, vol. 56(3), pages 449-467, March.
- Daniel R. Jiang & Warren B. Powell, 2015. "Optimal Hour-Ahead Bidding in the Real-Time Electricity Market with Battery Storage Using Approximate Dynamic Programming," INFORMS Journal on Computing, INFORMS, vol. 27(3), pages 525-543, August.
- Rene Carmona & Michael Ludkovski, 2010. "Valuation of energy storage: an optimal switching approach," Quantitative Finance, Taylor & Francis Journals, vol. 10(4), pages 359-374.
- Papadaki, Katerina P. & Powell, Warren B., 2002. "Exploiting structure in adaptive dynamic programming algorithms for a stochastic batch service problem," European Journal of Operational Research, Elsevier, vol. 142(1), pages 108-127, October.
- Mason, J.E. & Denton, B.T. & Shah, N.D. & Smith, S.A., 2014. "Optimizing the simultaneous management of blood pressure and cholesterol for type 2 diabetes patients," European Journal of Operational Research, Elsevier, vol. 233(3), pages 727-738.
- James E. Smith & Kevin F. McCardle, 2002. "Structural Properties of Stochastic Dynamic Programs," Operations Research, INFORMS, vol. 50(5), pages 796-809, October.
- Jae Ho Kim & Warren B. Powell, 2011. "Optimal Energy Commitments with Storage and Intermittent Supply," Operations Research, INFORMS, vol. 59(6), pages 1347-1360, December.
- Jennifer E. Mason & Darin A. England & Brian T. Denton & Steven A. Smith & Murat Kurt & Nilay D. Shah, 2012. "Optimizing Statin Treatment Decisions for Diabetes Patients in the Presence of Uncertain Future Adherence," Medical Decision Making, , vol. 32(1), pages 154-166, January.
- Juliana Nascimento & Warren Powell, 2010. "Dynamic Programming Models and Algorithms for the Mutual Fund Cash Balance Problem," Management Science, INFORMS, vol. 56(5), pages 801-815, May.
- John R. Birge, 1985. "Decomposition and Partitioning Methods for Multistage Stochastic Linear Programs," Operations Research, INFORMS, vol. 33(5), pages 989-1007, October.
Citations
Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
Cited by:
- Fokkema, Jan Eise & uit het Broek, Michiel A.J. & Schrotenboer, Albert H. & Land, Martin J. & Van Foreest, Nicky D., 2022. "Seasonal hydrogen storage decisions under constrained electricity distribution capacity," Renewable Energy, Elsevier, vol. 195(C), pages 76-91.
- Daniel R. Jiang & Warren B. Powell, 2015. "Optimal Hour-Ahead Bidding in the Real-Time Electricity Market with Battery Storage Using Approximate Dynamic Programming," INFORMS Journal on Computing, INFORMS, vol. 27(3), pages 525-543, August.
- Andrew J. Collins & Patrick Hester & Barry Ezell & John Horst, 2016. "An improvement selection methodology for key performance indicators," Environment Systems and Decisions, Springer, vol. 36(2), pages 196-208, June.
- Yin, Jiateng & Tang, Tao & Yang, Lixing & Gao, Ziyou & Ran, Bin, 2016. "Energy-efficient metro train rescheduling with uncertain time-variant passenger demands: An approximate dynamic programming approach," Transportation Research Part B: Methodological, Elsevier, vol. 91(C), pages 178-210.
- Achref Bachouch & C^ome Hur'e & Nicolas Langren'e & Huyen Pham, 2018. "Deep neural networks algorithms for stochastic control problems on finite horizon: numerical applications," Papers 1812.05916, arXiv.org, revised Jan 2020.
- Sebastian Becker & Patrick Cheridito & Arnulf Jentzen & Timo Welti, 2019. "Solving high-dimensional optimal stopping problems using deep learning," Papers 1908.01602, arXiv.org, revised Aug 2021.
- Achref Bachouch & Côme Huré & Nicolas Langrené & Huyen Pham, 2020. "Deep neural networks algorithms for stochastic control problems on finite horizon: numerical applications," Post-Print hal-01949221, HAL.
- Daniel R. Jiang & Warren B. Powell, 2018. "Risk-Averse Approximate Dynamic Programming with Quantile-Based Risk Measures," Mathematics of Operations Research, INFORMS, vol. 43(2), pages 554-579, May.
- Chen, Yao & Liu, Yang & Bai, Yun & Mao, Baohua, 2024. "Real-time dispatch management of shared autonomous vehicles with on-demand and pre-booked requests," Transportation Research Part A: Policy and Practice, Elsevier, vol. 181(C).
- Antoine Jacquier & Hao Liu, 2017. "Optimal liquidation in a Level-I limit order book for large tick stocks," Papers 1701.01327, arXiv.org, revised Nov 2017.
- Achref Bachouch & Côme Huré & Nicolas Langrené & Huyen Pham, 2019. "Deep neural networks algorithms for stochastic control problems on finite horizon: numerical applications," Working Papers hal-01949221, HAL.
- Weitzel, Timm & Glock, Christoph H., 2018. "Energy management for stationary electric energy storage systems: A systematic literature review," European Journal of Operational Research, Elsevier, vol. 264(2), pages 582-606.
- Achref Bachouch & Côme Huré & Nicolas Langrené & Huyên Pham, 2022. "Deep Neural Networks Algorithms for Stochastic Control Problems on Finite Horizon: Numerical Applications," Methodology and Computing in Applied Probability, Springer, vol. 24(1), pages 143-178, March.
- Al-Kanj, Lina & Nascimento, Juliana & Powell, Warren B., 2020. "Approximate dynamic programming for planning a ride-hailing system using autonomous fleets of electric vehicles," European Journal of Operational Research, Elsevier, vol. 284(3), pages 1088-1106.
- Andersen, Jesper Fink & Andersen, Anders Reenberg & Kulahci, Murat & Nielsen, Bo Friis, 2022. "A numerical study of Markov decision process algorithms for multi-component replacement problems," European Journal of Operational Research, Elsevier, vol. 299(3), pages 898-909.
- Ulmer, Marlin W. & Thomas, Barrett W., 2020. "Meso-parametric value function approximation for dynamic customer acceptances in delivery routing," European Journal of Operational Research, Elsevier, vol. 285(1), pages 183-195.
Most related items
These are the items that most often cite the same works as this one and are cited by the same works as this one.- Daniel R. Jiang & Warren B. Powell, 2015. "Optimal Hour-Ahead Bidding in the Real-Time Electricity Market with Battery Storage Using Approximate Dynamic Programming," INFORMS Journal on Computing, INFORMS, vol. 27(3), pages 525-543, August.
- Anna Maria Gambaro & Nicola Secomandi, 2021. "A Discussion of Non‐Gaussian Price Processes for Energy and Commodity Operations," Production and Operations Management, Production and Operations Management Society, vol. 30(1), pages 47-67, January.
- Secomandi, Nicola & Seppi, Duane J., 2014. "Real Options and Merchant Operations of Energy and Other Commodities," Foundations and Trends(R) in Technology, Information and Operations Management, now publishers, vol. 6(3-4), pages 161-331, July.
- Saif Benjaafar & Daniel Jiang & Xiang Li & Xiaobo Li, 2022. "Dynamic Inventory Repositioning in On-Demand Rental Networks," Management Science, INFORMS, vol. 68(11), pages 7861-7878, November.
- Yangfang (Helen) Zhou & Alan Scheller‐Wolf & Nicola Secomandi & Stephen Smith, 2019. "Managing Wind‐Based Electricity Generation in the Presence of Storage and Transmission Capacity," Production and Operations Management, Production and Operations Management Society, vol. 28(4), pages 970-989, April.
- Ilya O. Ryzhov & Martijn R. K. Mes & Warren B. Powell & Gerald van den Berg, 2019. "Bayesian Exploration for Approximate Dynamic Programming," Operations Research, INFORMS, vol. 67(1), pages 198-214, January.
- Jochen Gönsch & Michael Hassler, 2016. "Sell or store? An ADP approach to marketing renewable energy," OR Spectrum: Quantitative Approaches in Management, Springer;Gesellschaft für Operations Research e.V., vol. 38(3), pages 633-660, July.
- Ekaterina Abramova & Derek Bunn, 2021. "Optimal Daily Trading of Battery Operations Using Arbitrage Spreads," Energies, MDPI, vol. 14(16), pages 1-23, August.
- Benedikt Finnah, 2022. "Optimal bidding functions for renewable energies in sequential electricity markets," OR Spectrum: Quantitative Approaches in Management, Springer;Gesellschaft für Operations Research e.V., vol. 44(1), pages 1-27, March.
- Sandeep Rath & Kumar Rajaram, 2022. "Staff Planning for Hospitals with Implicit Cost Estimation and Stochastic Optimization," Production and Operations Management, Production and Operations Management Society, vol. 31(3), pages 1271-1289, March.
- Weitzel, Timm & Glock, Christoph H., 2018. "Energy management for stationary electric energy storage systems: A systematic literature review," European Journal of Operational Research, Elsevier, vol. 264(2), pages 582-606.
- Bastian Felix, 2012. "Gas Storage Valuation: A Comparative Simulation Study," EWL Working Papers 1201, University of Duisburg-Essen, Chair for Management Science and Energy Economics, revised Apr 2014.
- Qingyin Ma & John Stachurski, 2019. "Dynamic Optimal Choice When Rewards are Unbounded Below," Papers 1911.13025, arXiv.org.
- Manuel Arellano & Stéphane Bonhomme, 2017.
"Nonlinear Panel Data Methods for Dynamic Heterogeneous Agent Models,"
Annual Review of Economics, Annual Reviews, vol. 9(1), pages 471-496, September.
- Manuel Arellano & Stéphane Bonhomme, 2016. "Nonlinear panel data methods for dynamic heterogeneous agent models," CeMMAP working papers 51/16, Institute for Fiscal Studies.
- Manuel Arellano & Stéphane Bonhomme, 2017. "Nonlinear Panel Data Methods for Dynamic Heterogeneous Agent Models," Working Papers wp2017_1703, CEMFI.
- Manuel Arellano & Stéphane Bonhomme, 2016. "Nonlinear Panel Data Methods for Dynamic Heterogeneous Agent Models," Working Papers wp2016_1607, CEMFI.
- Manuel Arellano & Stéphane Bonhomme, 2016. "Nonlinear panel data methods for dynamic heterogeneous agent models," CeMMAP working papers CWP51/16, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
- Felix, Bastian Joachim & Weber, Christoph, 2012. "Gas storage valuation applying numerically constructed recombining trees," European Journal of Operational Research, Elsevier, vol. 216(1), pages 178-187.
- Lai Wei & Yongpei Guan, 2014. "Optimal Control of Plug-In Hybrid Electric Vehicles with Market Impact and Risk Attitude," Transportation Science, INFORMS, vol. 48(4), pages 467-482, November.
- Löhndorf, Nils & Wozabal, David, 2021. "Gas storage valuation in incomplete markets," European Journal of Operational Research, Elsevier, vol. 288(1), pages 318-330.
- Richard Blundell & Ran Gu & Søren Leth-Petersen & Hamish Low & Costas Meghir, 2019.
"Durables and Lemons: Private Information and the Market for Cars,"
NBER Working Papers
26281, National Bureau of Economic Research, Inc.
- Hamish Low & Richard Blundell & Ran Gu & Soren Leth-Petersen & Costas Meghir, 2019. "Durables and Lemons: Private Information and the Market for Cars," Economics Series Working Papers 890, University of Oxford, Department of Economics.
- Richard Blundell & Ran Gu & Soren Leth-Petersen & Hamish Low & Costas Meghir, 2019. "Durables and Lemons: Private Information and the Market for Cars," Cowles Foundation Discussion Papers 2197R, Cowles Foundation for Research in Economics, Yale University.
- Richard Blundell & Ran Gu & Soeren Leth-Petersen & Hamish Low & Costas Meghir, 2019. "Durables and Lemons: Private Information and the Market for Cars," CEBI working paper series 19-07, University of Copenhagen. Department of Economics. The Center for Economic Behavior and Inequality (CEBI).
- Richard Blundell & Ran Gu & Soren Leth-Petersen & Hamish Low & Costas Meghir, 2019. "Durables and Lemons: Private Information and the Market for Cars," Economics Papers 2019-W10, Economics Group, Nuffield College, University of Oxford.
- Richard Blundell & Ran Gu & Soren Leth-Petersen & Hamish Low & Costas Meghir, 2019. "Durables and Lemons: Private Information and the Market for Cars," Cowles Foundation Discussion Papers 2197, Cowles Foundation for Research in Economics, Yale University.
- W. Ackooij & X. Warin, 2020. "On conditional cuts for stochastic dual dynamic programming," EURO Journal on Computational Optimization, Springer;EURO - The Association of European Operational Research Societies, vol. 8(2), pages 173-199, June.
- Guoming Lai & Mulan X. Wang & Sunder Kekre & Alan Scheller-Wolf & Nicola Secomandi, 2011. "Valuation of Storage at a Liquefied Natural Gas Terminal," Operations Research, INFORMS, vol. 59(3), pages 602-616, June.
More about this item
Keywords
approximate dynamic programming; monotonicity; optimal stopping; energy storage; glycemic control;All these keywords.
Statistics
Access and download statisticsCorrections
All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:oropre:v:63:y:2015:i:6:p:1489-1511. See general information about how to correct material in RePEc.
If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.
If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .
If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .
Please note that corrections may take a couple of weeks to filter through the various RePEc services.