IDEAS home Printed from https://ideas.repec.org/a/eee/ejores/v241y2015i1p85-98.html
   My bibliography  Save this article

Approximate dynamic programming for stochastic linear control problems on compact state spaces

Author

Listed:
  • Woerner, Stefan
  • Laumanns, Marco
  • Zenklusen, Rico
  • Fertis, Apostolos

Abstract

This paper addresses Markov Decision Processes over compact state and action spaces. We investigate the special case of linear dynamics and piecewise-linear and convex immediate costs for the average cost criterion. This model is very general and covers many interesting examples, for instance in inventory management. Due to the curse of dimensionality, the problem is intractable and optimal policies usually cannot be computed, not even for instances of moderate size.

Suggested Citation

  • Woerner, Stefan & Laumanns, Marco & Zenklusen, Rico & Fertis, Apostolos, 2015. "Approximate dynamic programming for stochastic linear control problems on compact state spaces," European Journal of Operational Research, Elsevier, vol. 241(1), pages 85-98.
    • Handle: RePEc:eee:ejores:v:241:y:2015:i:1:p:85-98
    DOI: 10.1016/j.ejor.2014.08.003
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0377221714006195
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.ejor.2014.08.003?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Robert S. Kaplan, 1970. "A Dynamic Inventory Model with Stochastic Lead Times," Management Science, INFORMS, vol. 16(7), pages 491-507, March.
    2. Donald L. Iglehart, 1963. "Optimality of (s, S) Policies in the Infinite Horizon Dynamic Inventory Problem," Management Science, INFORMS, vol. 9(2), pages 259-267, January.
    3. Yoichiro Fukuda, 1964. "Optimal Policies for the Inventory Problem with Negotiable Leadtime," Management Science, INFORMS, vol. 10(4), pages 690-708, July.
    4. Daniela Pucci de Farias & Benjamin Van Roy, 2004. "On Constraint Sampling in the Linear Programming Approach to Approximate Dynamic Programming," Mathematics of Operations Research, INFORMS, vol. 29(3), pages 462-478, August.
    5. Senthil Veeraraghavan & Alan Scheller-Wolf, 2008. "Now or Later: A Simple Policy for Effective Dual Sourcing in Capacitated Systems," Operations Research, INFORMS, vol. 56(4), pages 850-864, August.
    6. Shapiro, Alexander, 2011. "Analysis of stochastic dual dynamic programming method," European Journal of Operational Research, Elsevier, vol. 209(1), pages 63-72, February.
    7. Arthur F. Veinott, Jr. & Harvey M. Wagner, 1965. "Computing Optimal (s, S) Inventory Policies," Management Science, INFORMS, vol. 11(5), pages 525-552, March.
    8. Minner, Stefan, 2003. "Multiple-supplier inventory models in supply chain management: A review," International Journal of Production Economics, Elsevier, vol. 81(1), pages 265-279, January.
    9. Richard Ehrhardt, 1984. "( s , S ) Policies for a Dynamic Inventory Model with Stochastic Lead Times," Operations Research, INFORMS, vol. 32(1), pages 121-132, February.
    10. D. P. de Farias & B. Van Roy, 2003. "The Linear Programming Approach to Approximate Dynamic Programming," Operations Research, INFORMS, vol. 51(6), pages 850-865, December.
    11. Janssen, Fred & de Kok, Ton, 1999. "A two-supplier inventory model," International Journal of Production Economics, Elsevier, vol. 59(1-3), pages 395-403, March.
    12. Klosterhalfen, Steffen & Kiesmüller, Gudrun & Minner, Stefan, 2011. "A comparison of the constant-order and dual-index policy for dual sourcing," International Journal of Production Economics, Elsevier, vol. 133(1), pages 302-311, September.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Svoboda, Josef & Minner, Stefan & Yao, Man, 2021. "Typology and literature review on multiple supplier inventory control models," European Journal of Operational Research, Elsevier, vol. 293(1), pages 1-23.
    2. Ewald, Christian Oliver & Nolan, Charles, 2024. "On the adaptation of the Lagrange formalism to continuous time stochastic optimal control: A Lagrange-Chow redux," Journal of Economic Dynamics and Control, Elsevier, vol. 162(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Arts, Joachim & Kiesmüller, Gudrun P., 2013. "Analysis of a two-echelon inventory system with two supply modes," European Journal of Operational Research, Elsevier, vol. 225(2), pages 263-272.
    2. Svoboda, Josef & Minner, Stefan & Yao, Man, 2021. "Typology and literature review on multiple supplier inventory control models," European Journal of Operational Research, Elsevier, vol. 293(1), pages 1-23.
    3. Younes Hamdouch & Youssef Boulaksil & Kilani Ghoudi, 2023. "Dual sourcing inventory management with nonconsecutive lead times from a supply chain perspective: a numerical study," OR Spectrum: Quantitative Approaches in Management, Springer;Gesellschaft für Operations Research e.V., vol. 45(3), pages 1013-1041, September.
    4. Boulaksil, Youssef & Hamdouch, Younes & Ghoudi, Kilani & Fransoo, Jan C., 2021. "Comparing policies for the stochastic multi-period dual sourcing problem from a supply chain perspective," Other publications TiSEM 242598b9-99de-4a3c-a1a9-4, Tilburg University, School of Economics and Management.
    5. Boulaksil, Youssef & Hamdouch, Younes & Ghoudi, Kilani & Fransoo, Jan C., 2021. "Comparing policies for the stochastic multi-period dual sourcing problem from a supply chain perspective," International Journal of Production Economics, Elsevier, vol. 232(C).
    6. Linwei Xin & David A. Goldberg, 2018. "Asymptotic Optimality of Tailored Base-Surge Policies in Dual-Sourcing Inventory Systems," Management Science, INFORMS, vol. 64(1), pages 437-452, January.
    7. Cheaitou, Ali & van Delft, Christian, 2013. "Finite horizon stochastic inventory problem with dual sourcing: Near myopic and heuristics bounds," International Journal of Production Economics, Elsevier, vol. 143(2), pages 371-378.
    8. Huang, Boray & Wu, Andy, 2017. "Reduce shortage with self-reservation policy for a manufacturer paying both fixed and variable stockout expenditure," European Journal of Operational Research, Elsevier, vol. 262(3), pages 944-953.
    9. Ben-Ammar, Oussama & Bettayeb, Belgacem & Dolgui, Alexandre, 2019. "Optimization of multi-period supply planning under stochastic lead times and a dynamic demand," International Journal of Production Economics, Elsevier, vol. 218(C), pages 106-117.
    10. Jakšič, M. & Fransoo, J.C., 2018. "Dual sourcing in the age of near-shoring: Trading off stochastic capacity limitations and long lead times," European Journal of Operational Research, Elsevier, vol. 267(1), pages 150-161.
    11. Tong Wang & Xiaoyue Yan & Chaolin Yang, 2021. "Managing a Hybrid RDC‐DC Inventory System," Production and Operations Management, Production and Operations Management Society, vol. 30(10), pages 3679-3697, October.
    12. Hekimoğlu, Mustafa & Scheller-Wolf, Alan, 2023. "Dual sourcing models with stock-out dependent substitution," European Journal of Operational Research, Elsevier, vol. 311(2), pages 472-485.
    13. Sean X. Zhou & Chaolin Yang, 2016. "Continuous-Review ( R, nQ ) Policies for Inventory Systems with Dual Delivery Modes," Operations Research, INFORMS, vol. 64(6), pages 1302-1319, December.
    14. Klosterhalfen, Steffen & Kiesmüller, Gudrun & Minner, Stefan, 2011. "A comparison of the constant-order and dual-index policy for dual sourcing," International Journal of Production Economics, Elsevier, vol. 133(1), pages 302-311, September.
    15. Alessio Trivella & Danial Mohseni-Taheri & Selvaprabu Nadarajah, 2023. "Meeting Corporate Renewable Power Targets," Management Science, INFORMS, vol. 69(1), pages 491-512, January.
    16. Ganesh Janakiraman & Sridhar Seshadri & Anshul Sheopuri, 2015. "Analysis of Tailored Base-Surge Policies in Dual Sourcing Inventory Systems," Management Science, INFORMS, vol. 61(7), pages 1547-1561, July.
    17. Ghoudi, Kilani & Hamdouch, Younes & Boulaksil, Youssef & Hamdan, Sadeque, 2024. "Supply chain coordination in a dual sourcing system under the Tailored Base-Surge policy," European Journal of Operational Research, Elsevier, vol. 317(2), pages 533-549.
    18. Alain Bensoussan & Lama Moussawi-Haidar & Metin Çakanyıldırım, 2010. "Inventory control with an order-time constraint: optimality, uniqueness and significance," Annals of Operations Research, Springer, vol. 181(1), pages 603-640, December.
    19. Tang, Christopher S., 2006. "Perspectives in supply chain risk management," International Journal of Production Economics, Elsevier, vol. 103(2), pages 451-488, October.
    20. Scheller-Wolf, Alan & Tayur, Sridhar, 2009. "Risk sharing in supply chains using order bands--Analytical results and managerial insights," International Journal of Production Economics, Elsevier, vol. 121(2), pages 715-727, October.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:ejores:v:241:y:2015:i:1:p:85-98. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/eor .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.