IDEAS home Printed from https://ideas.repec.org/a/eee/proeco/v274y2024ics0925527324001695.html
   My bibliography  Save this article

The use of reinforcement learning for material flow control: An assessment by simulation

Author

Listed:
  • He, Zhiliang
  • Thürer, Matthias
  • Zhou, Wanling

Abstract

One of the main objectives of Material Flow Control (MFC) is to ensure delivery performance. Traditional MFC realizes this through independent decisions at two levels: order release and production authorization on the shop floor. This hierarchical decision-making can be improved by integration because these decisions are interconnected. This study introduces a new reinforcement learning method that combines, and jointly optimizes various MFC decisions. It enhances the delivery performance of an agent by enabling it to interact with the environment and to learn the parameters of the decision model. Results from a make-to-order pure job shop simulation model demonstrate that the new approach outperforms exiting MFC methods in most cases. This extends existing literature on MFC, which remains entrenched in traditional decision methods, and existing literature on reinforcement learning in the context of production planning and control, which remains largely focused on production scheduling. It has important implications for the future design of production planning and control systems and practice, specifically in contexts where data is readily available or a digital shadow can be obtained.

Suggested Citation

  • He, Zhiliang & Thürer, Matthias & Zhou, Wanling, 2024. "The use of reinforcement learning for material flow control: An assessment by simulation," International Journal of Production Economics, Elsevier, vol. 274(C).
  • Handle: RePEc:eee:proeco:v:274:y:2024:i:c:s0925527324001695
    DOI: 10.1016/j.ijpe.2024.109312
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0925527324001695
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.ijpe.2024.109312?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Ivanov, Dmitry, 2023. "Intelligent digital twin (iDT) for supply chain stress-testing, resilience, and viability," International Journal of Production Economics, Elsevier, vol. 263(C).
    2. Land, Martin J. & Stevenson, Mark & Thürer, Matthias & Gaalman, Gerard J.C., 2015. "Job shop control: In search of the key to delivery improvements," International Journal of Production Economics, Elsevier, vol. 168(C), pages 257-266.
    3. Manuel Schneckenreither & Stefan Haeussler & Christoph Gerhold, 2021. "Order release planning with predictive lead times: a machine learning approach," International Journal of Production Research, Taylor & Francis Journals, vol. 59(11), pages 3285-3303, June.
    4. Takahashi, Katsuhiko & Myreshka & Hirotani, Daisuke, 2005. "Comparing CONWIP, synchronized CONWIP, and Kanban in complex supply chains," International Journal of Production Economics, Elsevier, vol. 93(1), pages 25-40, January.
    5. Land, Martin, 2006. "Parameters and sensitivity in workload control," International Journal of Production Economics, Elsevier, vol. 104(2), pages 625-638, December.
    6. Stevenson, Mark & Hendry, Linda C., 2006. "Aggregate load-oriented workload control: A review and a re-classification of a key approach," International Journal of Production Economics, Elsevier, vol. 104(2), pages 676-693, December.
    7. Land, Martin & Gaalman, Gerard, 1996. "Workload control concepts in job shops A critical assessment," International Journal of Production Economics, Elsevier, vol. 46(1), pages 535-548, December.
    8. Yi Liang & Zan Sun & Tianheng Song & Qiang Chou & Wei Fan & Jianping Fan & Yong Rui & Qiping Zhou & Jessie Bai & Chun Yang & Peng Bai, 2022. "Lenovo Schedules Laptop Manufacturing Using Deep Reinforcement Learning," Interfaces, INFORMS, vol. 52(1), pages 56-68, January.
    9. Thürer, Matthias & Stevenson, Mark, 2021. "Improving superfluous load avoidance release (SLAR): A new load-based SLAR mechanism," International Journal of Production Economics, Elsevier, vol. 231(C).
    10. Oosterman, Bas & Land, Martin & Gaalman, Gerard, 2000. "The influence of shop characteristics on workload control," International Journal of Production Economics, Elsevier, vol. 68(1), pages 107-119, October.
    11. Kasper, T.A. Arno & Land, Martin J. & Teunter, Ruud H., 2023. "Non-hierarchical work-in-progress control in manufacturing," International Journal of Production Economics, Elsevier, vol. 257(C).
    12. Gustavo Bagni & Moacir Godinho Filho & Matthias Thürer & Mark Stevenson, 2021. "Systematic Review and Discussion of Production Control Systems That Emerged between 1999 and 2018," Post-Print hal-04289713, HAL.
    13. Fernando José Gómez Paredes & Moacir Godinho Filho & Matthias Thürer & Nuno O. Fernandes & Charbel José Chiappeta Jabbour, 2022. "Factors for choosing production control systems in make-to-order shops: a systematic literature review," Journal of Intelligent Manufacturing, Springer, vol. 33(3), pages 639-674, March.
    14. Pranesh Saisridhar & Matthias Thürer & Balram Avittathur, 2024. "Assessing supply chain responsiveness, resilience and robustness (Triple-R) by computer simulation: a systematic review of the literature," International Journal of Production Research, Taylor & Francis Journals, vol. 62(4), pages 1458-1488, February.
    15. Volodymyr Mnih & Koray Kavukcuoglu & David Silver & Andrei A. Rusu & Joel Veness & Marc G. Bellemare & Alex Graves & Martin Riedmiller & Andreas K. Fidjeland & Georg Ostrovski & Stig Petersen & Charle, 2015. "Human-level control through deep reinforcement learning," Nature, Nature, vol. 518(7540), pages 529-533, February.
    16. Fredendall, Lawrence D. & Ojha, Divesh & Wayne Patterson, J., 2010. "Concerning the theory of workload control," European Journal of Operational Research, Elsevier, vol. 201(1), pages 99-111, February.
    17. Ana Esteso & David Peidro & Josefa Mula & Manuel Díaz-Madroñero, 2023. "Reinforcement learning applied to production planning and control," International Journal of Production Research, Taylor & Francis Journals, vol. 61(16), pages 5772-5789, August.
    18. Land, Martin J. & Gaalman, Gerard J. C., 1998. "The performance of workload control concepts in job shops: Improving the release method," International Journal of Production Economics, Elsevier, vol. 56(1), pages 347-364, September.
    19. Haeussler, S. & Stampfer, C. & Missbauer, H., 2020. "Comparison of two optimization based order release models with fixed and variable lead times," International Journal of Production Economics, Elsevier, vol. 227(C).
    20. Marcel Panzer & Benedict Bender, 2022. "Deep reinforcement learning in production systems: a systematic literature review," International Journal of Production Research, Taylor & Francis Journals, vol. 60(13), pages 4316-4341, July.
    21. Stefan Haeussler & Pia Netzer, 2020. "Comparison between rule- and optimization-based workload control concepts: a simulation optimization approach," International Journal of Production Research, Taylor & Francis Journals, vol. 58(12), pages 3724-3743, June.
    22. Stefan Haeussler & Philipp Neuner & Matthias Thürer, 2023. "Balancing earliness and tardiness within workload control order release: an assessment by simulation," Flexible Services and Manufacturing Journal, Springer, vol. 35(2), pages 487-508, June.
    23. Marcel Panzer & Benedict Bender & Norbert Gronau, 2024. "A deep reinforcement learning based hyper-heuristic for modular production control," International Journal of Production Research, Taylor & Francis Journals, vol. 62(8), pages 2747-2768, April.
    24. Pettersen, Jan-Arne & Segerstedt, Anders, 2009. "Restricted work-in-process: A study of differences between Kanban and CONWIP," International Journal of Production Economics, Elsevier, vol. 118(1), pages 199-207, March.
    25. M. Thenarasu & K. Rameshkumar & M. Di Mascolo & S.P. Anbuudayasankar, 2024. "Multi-criteria scheduling of realistic flexible job shop: a novel approach for integrating simulation modelling and multi-criteria decision making," International Journal of Production Research, Taylor & Francis Journals, vol. 62(1-2), pages 336-358, January.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Stefan Haeussler & Philipp Neuner & Matthias Thürer, 2023. "Balancing earliness and tardiness within workload control order release: an assessment by simulation," Flexible Services and Manufacturing Journal, Springer, vol. 35(2), pages 487-508, June.
    2. Kasper, T.A. Arno & Land, Martin J. & Teunter, Ruud H., 2023. "Towards System State Dispatching in High‐Variety Manufacturing," Omega, Elsevier, vol. 114(C).
    3. Kasper, T.A. Arno & Land, Martin J. & Teunter, Ruud H., 2023. "Non-hierarchical work-in-progress control in manufacturing," International Journal of Production Economics, Elsevier, vol. 257(C).
    4. Matthias Thürer & Mark Stevenson, 2016. "Workload control in job shops with re-entrant flows: an assessment by simulation," International Journal of Production Research, Taylor & Francis Journals, vol. 54(17), pages 5136-5150, September.
    5. Fernandes, Nuno O. & Thürer, Matthias & Silva, Cristóvão & Carmo-Silva, Sílvio, 2017. "Improving workload control order release: Incorporating a starvation avoidance trigger into continuous release," International Journal of Production Economics, Elsevier, vol. 194(C), pages 181-189.
    6. Thürer, Matthias & Stevenson, Mark & Land, Martin J., 2016. "On the integration of input and output control: Workload Control order release," International Journal of Production Economics, Elsevier, vol. 174(C), pages 43-53.
    7. Matthias Thürer & Mark Stevenson, 2021. "Material handling and order release control in high-variety make-to-order shops: an assessment by simulation," Operations Management Research, Springer, vol. 14(3), pages 494-506, December.
    8. Thürer, Matthias & Stevenson, Mark & Silva, Cristovao & Qu, Ting, 2017. "Drum-buffer-rope and workload control in High-variety flow and job shops with bottlenecks: An assessment by simulation," International Journal of Production Economics, Elsevier, vol. 188(C), pages 116-127.
    9. Sagawa, Juliana Keiko & Oliveira, Allan Freitas & Mušič, Gašper & Land, Martin J. & Maluf, Arthur Sarro, 2023. "Smart workload input-output control of production systems: A proof of concept," European Journal of Operational Research, Elsevier, vol. 309(1), pages 286-305.
    10. Onay, Anita & Stampfer, Christina & Missbauer, Hubert, 2023. "A behavioral perspective on workload control concepts: The influence of order release on operators’ reaction behavior," International Journal of Production Economics, Elsevier, vol. 264(C).
    11. Kaustav Kundu & Martin J. Land & Alberto Portioli-Staudacher & Jos A. C. Bokhorst, 2021. "Order review and release in make-to-order flow shops: analysis and design of new methods," Flexible Services and Manufacturing Journal, Springer, vol. 33(3), pages 750-782, September.
    12. Thürer, Matthias & Stevenson, Mark, 2021. "Improving superfluous load avoidance release (SLAR): A new load-based SLAR mechanism," International Journal of Production Economics, Elsevier, vol. 231(C).
    13. Matthias Thürer & Nuno O. Fernandes & Mark Stevenson, 2020. "Material Flow Control in High‐Variety Make‐to‐Order Shops: Combining COBACABANA and POLCA," Production and Operations Management, Production and Operations Management Society, vol. 29(9), pages 2138-2152, September.
    14. Davide Mezzogori & Giovanni Romagnoli & Francesco Zammori, 2021. "Defining accurate delivery dates in make to order job-shops managed by workload control," Flexible Services and Manufacturing Journal, Springer, vol. 33(4), pages 956-991, December.
    15. Land, Martin J., 2009. "Cobacabana (control of balance by card-based navigation): A card-based system for job shop control," International Journal of Production Economics, Elsevier, vol. 117(1), pages 97-103, January.
    16. Federica Costa & Alberto Portioli-Staudacher, 2021. "Labor flexibility integration in workload control in Industry 4.0 era," Operations Management Research, Springer, vol. 14(3), pages 420-433, December.
    17. Matthias Thürer & Mark Stevenson & Ting Qu, 2016. "Job sequencing and selection within workload control order release: an assessment by simulation," International Journal of Production Research, Taylor & Francis Journals, vol. 54(4), pages 1061-1075, February.
    18. Matthias Thürer & Mark Stevenson, 2016. "Card-based delivery date promising in pure flow shops with order release control," International Journal of Production Research, Taylor & Francis Journals, vol. 54(22), pages 6798-6811, November.
    19. Jianjun Liu & Martin J. Land & Jos A. C. Bokhorst & Qingxin Chen, 2023. "Improving coordination in assembly job shops: redesigning order release and dispatching," Flexible Services and Manufacturing Journal, Springer, vol. 35(3), pages 669-697, September.
    20. Yarong Chen & Hongming Zhou & Peiyu Huang & FuhDer Chou & Shenquan Huang, 2022. "A refined order release method for achieving robustness of non-repetitive dynamic manufacturing system performance," Annals of Operations Research, Springer, vol. 311(1), pages 65-79, April.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:proeco:v:274:y:2024:i:c:s0925527324001695. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/ijpe .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.