IDEAS home Printed from https://ideas.repec.org/a/gam/jsusta/v15y2023i10p8262-d1150542.html
   My bibliography  Save this article

A Multi-Agent Reinforcement Learning Approach to the Dynamic Job Shop Scheduling Problem

Author

Listed:
  • Ali Fırat İnal

    (Department of Industrial Engineering, Kirikkale University, Kirikkale 71450, Turkey)

  • Çağrı Sel

    (Department of Industrial Engineering, Karabük University, Karabük 78050, Turkey)

  • Adnan Aktepe

    (Department of Industrial Engineering, Kirikkale University, Kirikkale 71450, Turkey)

  • Ahmet Kürşad Türker

    (Department of Industrial Engineering, Kirikkale University, Kirikkale 71450, Turkey)

  • Süleyman Ersöz

    (Department of Industrial Engineering, Kirikkale University, Kirikkale 71450, Turkey)

Abstract

In a production environment, scheduling decides job and machine allocations and the operation sequence. In a job shop production system, the wide variety of jobs, complex routes, and real-life events becomes challenging for scheduling activities. New, unexpected events disrupt the production schedule and require dynamic scheduling updates to the production schedule on an event-based basis. To solve the dynamic scheduling problem, we propose a multi-agent system with reinforcement learning aimed at the minimization of tardiness and flow time to improve the dynamic scheduling techniques. The performance of the proposed multi-agent system is compared with the first-in–first-out, shortest processing time, and earliest due date dispatching rules in terms of the minimization of tardy jobs, mean tardiness, maximum tardiness, mean earliness, maximum earliness, mean flow time, maximum flow time, work in process, and makespan. Five scenarios are generated with different arrival intervals of the jobs to the job shop production system. The results of the experiments, performed for the 3 × 3, 5 × 5, and 10 × 10 problem sizes, show that our multi-agent system overperforms compared to the dispatching rules as the workload of the job shop increases. Under a heavy workload, the proposed multi-agent system gives the best results for five performance criteria, which are the proportion of tardy jobs, mean tardiness, maximum tardiness, mean flow time, and maximum flow time.

Suggested Citation

  • Ali Fırat İnal & Çağrı Sel & Adnan Aktepe & Ahmet Kürşad Türker & Süleyman Ersöz, 2023. "A Multi-Agent Reinforcement Learning Approach to the Dynamic Job Shop Scheduling Problem," Sustainability, MDPI, vol. 15(10), pages 1-24, May.
  • Handle: RePEc:gam:jsusta:v:15:y:2023:i:10:p:8262-:d:1150542
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2071-1050/15/10/8262/pdf
    Download Restriction: no

    File URL: https://www.mdpi.com/2071-1050/15/10/8262/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Baykasoglu, Adil & Gocken, Mustafa & Unutmaz, Zeynep D., 2008. "New approaches to due date assignment in job shops," European Journal of Operational Research, Elsevier, vol. 187(1), pages 31-45, May.
    2. Thierry Moyaux & Yinling Liu & Guillaume Bouleux & Vincent Cheutet, 2023. "An Agent-Based Architecture of the Digital Twin for an Emergency Department," Sustainability, MDPI, vol. 15(4), pages 1-13, February.
    3. Jain, A. S. & Meeran, S., 1999. "Deterministic job-shop scheduling: Past, present and future," European Journal of Operational Research, Elsevier, vol. 113(2), pages 390-434, March.
    4. Adil Baykasoğlu & Fatma S. Karaslan, 2017. "Solving comprehensive dynamic job shop scheduling problem by using a GRASP-based approach," International Journal of Production Research, Taylor & Francis Journals, vol. 55(11), pages 3308-3325, June.
    5. Holthaus, Oliver & Rajendran, Chandrasekharan, 1997. "Efficient dispatching rules for scheduling in a job shop," International Journal of Production Economics, Elsevier, vol. 48(1), pages 87-105, January.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yu Pu & Fang Li & Shahin Rahimifard, 2024. "Multi-Agent Reinforcement Learning for Job Shop Scheduling in Dynamic Environments," Sustainability, MDPI, vol. 16(8), pages 1-26, April.
    2. Thiesse, Frédéric & Fleisch, Elgar, 2008. "On the value of location information to lot scheduling in complex manufacturing processes," International Journal of Production Economics, Elsevier, vol. 112(2), pages 532-547, April.
    3. Wolosewicz, Cathy & Dauzère-Pérès, Stéphane & Aggoune, Riad, 2015. "A Lagrangian heuristic for an integrated lot-sizing and fixed scheduling problem," European Journal of Operational Research, Elsevier, vol. 244(1), pages 3-12.
    4. Tao Ren & Yan Zhang & Shuenn-Ren Cheng & Chin-Chia Wu & Meng Zhang & Bo-yu Chang & Xin-yue Wang & Peng Zhao, 2020. "Effective Heuristic Algorithms Solving the Jobshop Scheduling Problem with Release Dates," Mathematics, MDPI, vol. 8(8), pages 1-25, July.
    5. Rossi, Andrea, 2014. "Flexible job shop scheduling with sequence-dependent setup and transportation times by ant colony with reinforced pheromone relationships," International Journal of Production Economics, Elsevier, vol. 153(C), pages 253-267.
    6. Sels, Veronique & Craeymeersch, Kjeld & Vanhoucke, Mario, 2011. "A hybrid single and dual population search procedure for the job shop scheduling problem," European Journal of Operational Research, Elsevier, vol. 215(3), pages 512-523, December.
    7. Jacomine Grobler & Andries Engelbrecht & Schalk Kok & Sarma Yadavalli, 2010. "Metaheuristics for the multi-objective FJSP with sequence-dependent set-up times, auxiliary resources and machine down time," Annals of Operations Research, Springer, vol. 180(1), pages 165-196, November.
    8. Ganesan, Viswanath Kumar & Sivakumar, Appa Iyer, 2006. "Scheduling in static jobshops for minimizing mean flowtime subject to minimum total deviation of job completion times," International Journal of Production Economics, Elsevier, vol. 103(2), pages 633-647, October.
    9. Roux, O. & Duvivier, D. & Dhaevers, V. & Meskens, N. & Artiba, A., 2008. "Multicriteria approach to rank scheduling strategies," International Journal of Production Economics, Elsevier, vol. 112(1), pages 192-201, March.
    10. Bürgy, Reinhard & Bülbül, Kerem, 2018. "The job shop scheduling problem with convex costs," European Journal of Operational Research, Elsevier, vol. 268(1), pages 82-100.
    11. Tanja Mlinar & Philippe Chevalier, 2016. "Pooling heterogeneous products for manufacturing environments," 4OR, Springer, vol. 14(2), pages 173-200, June.
    12. Azaron, Amir & Katagiri, Hideki & Kato, Kosuke & Sakawa, Masatoshi, 2006. "Longest path analysis in networks of queues: Dynamic scheduling problems," European Journal of Operational Research, Elsevier, vol. 174(1), pages 132-149, October.
    13. G I Zobolas & C D Tarantilis & G Ioannou, 2009. "A hybrid evolutionary algorithm for the job shop scheduling problem," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 60(2), pages 221-235, February.
    14. Gómez Sánchez, Mariam & Lalla-Ruiz, Eduardo & Fernández Gil, Alejandro & Castro, Carlos & Voß, Stefan, 2023. "Resource-constrained multi-project scheduling problem: A survey," European Journal of Operational Research, Elsevier, vol. 309(3), pages 958-976.
    15. Gabriel Mauricio Zambrano-Rey & Eliana María González-Neira & Gabriel Fernando Forero-Ortiz & María José Ocampo-Monsalve & Andrea Rivera-Torres, 2024. "Minimizing the expected maximum lateness for a job shop subject to stochastic machine breakdowns," Annals of Operations Research, Springer, vol. 338(1), pages 801-833, July.
    16. Giuseppe Lancia & Franca Rinaldi & Paolo Serafini, 2011. "A time-indexed LP-based approach for min-sum job-shop problems," Annals of Operations Research, Springer, vol. 186(1), pages 175-198, June.
    17. Zhang, Rui & Song, Shiji & Wu, Cheng, 2013. "A hybrid artificial bee colony algorithm for the job shop scheduling problem," International Journal of Production Economics, Elsevier, vol. 141(1), pages 167-178.
    18. D. Duvivier & O. Roux & V. Dhaevers & N. Meskens & A. Artiba, 2007. "Multicriteria optimisation and simulation: an industrial application," Annals of Operations Research, Springer, vol. 156(1), pages 45-60, December.
    19. Rajendran, Chandrasekharan & Ziegler, Hans, 2001. "A performance analysis of dispatching rules and a heuristic in static flowshops with missing operations of jobs," European Journal of Operational Research, Elsevier, vol. 131(3), pages 622-634, June.
    20. Wolfram Wiesemann & Daniel Kuhn & Berç Rustem, 2012. "Multi-resource allocation in stochastic project scheduling," Annals of Operations Research, Springer, vol. 193(1), pages 193-220, March.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jsusta:v:15:y:2023:i:10:p:8262-:d:1150542. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.