IDEAS home Printed from https://ideas.repec.org/a/gam/jsusta/v14y2022i9p5177-d801702.html
   My bibliography  Save this article

Dynamic Scheduling Method for Job-Shop Manufacturing Systems by Deep Reinforcement Learning with Proximal Policy Optimization

Author

Listed:
  • Ming Zhang

    (College of Engineering and Physical Sciences, Aston University, Birmingham B4 7ET, UK)

  • Yang Lu

    (School of Science, Technology and Health, York St John University, York YO31 7EX, UK)

  • Youxi Hu

    (College of Engineering and Physical Sciences, Aston University, Birmingham B4 7ET, UK)

  • Nasser Amaitik

    (College of Engineering and Physical Sciences, Aston University, Birmingham B4 7ET, UK)

  • Yuchun Xu

    (College of Engineering and Physical Sciences, Aston University, Birmingham B4 7ET, UK)

Abstract

With the rapid development of Industrial 4.0, the modern manufacturing system has been experiencing profoundly digital transformation. The development of new technologies helps to improve the efficiency of production and the quality of products. However, for the increasingly complex production systems, operational decision making encounters more challenges in terms of having sustainable manufacturing to satisfy customers and markets’ rapidly changing demands. Nowadays, rule-based heuristic approaches are widely used for scheduling management in production systems, which, however, significantly depends on the expert domain knowledge. In this way, the efficiency of decision making could not be guaranteed nor meet the dynamic scheduling requirement in the job-shop manufacturing environment. In this study, we propose using deep reinforcement learning (DRL) methods to tackle the dynamic scheduling problem in the job-shop manufacturing system with unexpected machine failure. The proximal policy optimization (PPO) algorithm was used in the DRL framework to accelerate the learning process and improve performance. The proposed method was testified within a real-world dynamic production environment, and it performs better compared with the state-of-the-art methods.

Suggested Citation

  • Ming Zhang & Yang Lu & Youxi Hu & Nasser Amaitik & Yuchun Xu, 2022. "Dynamic Scheduling Method for Job-Shop Manufacturing Systems by Deep Reinforcement Learning with Proximal Policy Optimization," Sustainability, MDPI, vol. 14(9), pages 1-16, April.
  • Handle: RePEc:gam:jsusta:v:14:y:2022:i:9:p:5177-:d:801702
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2071-1050/14/9/5177/pdf
    Download Restriction: no

    File URL: https://www.mdpi.com/2071-1050/14/9/5177/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Chuang Wang & Pingyu Jiang, 2018. "Manifold learning based rescheduling decision mechanism for recessive disturbances in RFID-driven job shops," Journal of Intelligent Manufacturing, Springer, vol. 29(7), pages 1485-1500, October.
    2. Andreas Kuhnle & Jan-Philipp Kaiser & Felix Theiß & Nicole Stricker & Gisela Lanza, 2021. "Designing an adaptive production control system using reinforcement learning," Journal of Intelligent Manufacturing, Springer, vol. 32(3), pages 855-876, March.
    3. Peter J. M. van Laarhoven & Emile H. L. Aarts & Jan Karel Lenstra, 1992. "Job Shop Scheduling by Simulated Annealing," Operations Research, INFORMS, vol. 40(1), pages 113-125, February.
    4. Alan S. Manne, 1960. "On the Job-Shop Scheduling Problem," Operations Research, INFORMS, vol. 8(2), pages 219-223, April.
    5. M. R. Garey & D. S. Johnson & Ravi Sethi, 1976. "The Complexity of Flowshop and Jobshop Scheduling," Mathematics of Operations Research, INFORMS, vol. 1(2), pages 117-129, May.
    6. Prateek Saxena & Panagiotis Stavropoulos & John Kechagias & Konstantinos Salonitis, 2020. "Sustainability Assessment for Manufacturing Operations," Energies, MDPI, vol. 13(11), pages 1-19, May.
    7. Haoxiang Wang & Bhaba R. Sarker & Jing Li & Jian Li, 2021. "Adaptive scheduling for assembly job shop with uncertain assembly times based on dual Q-learning," International Journal of Production Research, Taylor & Francis Journals, vol. 59(19), pages 5867-5883, October.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Walter Cardoso Satyro & Jose Celso Contador & Sonia Francisca de Paula Monken & Anderson Ferreira de Lima & Gilberto Gomes Soares Junior & Jansen Anderson Gomes & João Victor Silva Neves & José Robert, 2023. "Industry 4.0 Implementation Projects: The Cleaner Production Strategy—A Literature Review," Sustainability, MDPI, vol. 15(3), pages 1-18, January.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Da Col, Giacomo & Teppan, Erich C., 2022. "Industrial-size job shop scheduling with constraint programming," Operations Research Perspectives, Elsevier, vol. 9(C).
    2. Chong Peng & Guanglin Wu & T Warren Liao & Hedong Wang, 2019. "Research on multi-agent genetic algorithm based on tabu search for the job shop scheduling problem," PLOS ONE, Public Library of Science, vol. 14(9), pages 1-19, September.
    3. Edzard Weber & Anselm Tiefenbacher & Norbert Gronau, 2019. "Need for Standardization and Systematization of Test Data for Job-Shop Scheduling," Data, MDPI, vol. 4(1), pages 1-21, February.
    4. Selcuk Goren & Ihsan Sabuncuoglu & Utku Koc, 2012. "Optimization of schedule stability and efficiency under processing time variability and random machine breakdowns in a job shop environment," Naval Research Logistics (NRL), John Wiley & Sons, vol. 59(1), pages 26-38, February.
    5. Hong-Sen Yan & Wen-Chao Li, 2017. "A multi-objective scheduling algorithm with self-evolutionary feature for job-shop-like knowledgeable manufacturing cell," Journal of Intelligent Manufacturing, Springer, vol. 28(2), pages 337-351, February.
    6. Kan Fang & Nelson Uhan & Fu Zhao & John Sutherland, 2013. "Flow shop scheduling with peak power consumption constraints," Annals of Operations Research, Springer, vol. 206(1), pages 115-145, July.
    7. Jonas Harbering & Abhiram Ranade & Marie Schmidt & Oliver Sinnen, 2019. "Complexity, bounds and dynamic programming algorithms for single track train scheduling," Annals of Operations Research, Springer, vol. 273(1), pages 479-500, February.
    8. T. C. E. Cheng & Bo Peng & Zhipeng Lü, 2016. "A hybrid evolutionary algorithm to solve the job shop scheduling problem," Annals of Operations Research, Springer, vol. 242(2), pages 223-237, July.
    9. Jian Zhang & Guofu Ding & Yisheng Zou & Shengfeng Qin & Jianlin Fu, 2019. "Review of job shop scheduling research and its new perspectives under Industry 4.0," Journal of Intelligent Manufacturing, Springer, vol. 30(4), pages 1809-1830, April.
    10. Berterottière, Lucas & Dauzère-Pérès, Stéphane & Yugma, Claude, 2024. "Flexible job-shop scheduling with transportation resources," European Journal of Operational Research, Elsevier, vol. 312(3), pages 890-909.
    11. Blazewicz, Jacek & Domschke, Wolfgang & Pesch, Erwin, 1996. "The job shop scheduling problem: Conventional and new solution techniques," European Journal of Operational Research, Elsevier, vol. 93(1), pages 1-33, August.
    12. Yaliang Wang & Xinyu Fan & Chendi Ni & Kanghong Gao & Shousong Jin, 2023. "Collaborative optimization of workshop layout and scheduling," Journal of Scheduling, Springer, vol. 26(1), pages 43-59, February.
    13. Mehravaran, Yasaman & Logendran, Rasaratnam, 2012. "Non-permutation flowshop scheduling in a supply chain with sequence-dependent setup times," International Journal of Production Economics, Elsevier, vol. 135(2), pages 953-963.
    14. Lunardi, Willian T. & Birgin, Ernesto G. & Ronconi, Débora P. & Voos, Holger, 2021. "Metaheuristics for the online printing shop scheduling problem," European Journal of Operational Research, Elsevier, vol. 293(2), pages 419-441.
    15. Zhengcai Cao & Lijie Zhou & Biao Hu & Chengran Lin, 2019. "An Adaptive Scheduling Algorithm for Dynamic Jobs for Dealing with the Flexible Job Shop Scheduling Problem," Business & Information Systems Engineering: The International Journal of WIRTSCHAFTSINFORMATIK, Springer;Gesellschaft für Informatik e.V. (GI), vol. 61(3), pages 299-309, June.
    16. Shen, Liji & Buscher, Udo, 2012. "Solving the serial batching problem in job shop manufacturing systems," European Journal of Operational Research, Elsevier, vol. 221(1), pages 14-26.
    17. Rossi, Andrea, 2014. "Flexible job shop scheduling with sequence-dependent setup and transportation times by ant colony with reinforced pheromone relationships," International Journal of Production Economics, Elsevier, vol. 153(C), pages 253-267.
    18. Agata Sudolska & Justyna Łapińska, 2020. "Exploring Determinants of Innovation Capability in Manufacturing Companies Operating in Poland," Sustainability, MDPI, vol. 12(17), pages 1-21, August.
    19. K. Z. Gao & P. N. Suganthan & Q. K. Pan & T. J. Chua & T. X. Cai & C. S. Chong, 2016. "Discrete harmony search algorithm for flexible job shop scheduling problem with multiple objectives," Journal of Intelligent Manufacturing, Springer, vol. 27(2), pages 363-374, April.
    20. Wang, Ling & Sun, Lin-Yan & Sun, Lin-Hui & Wang, Ji-Bo, 2010. "On three-machine flow shop scheduling with deteriorating jobs," International Journal of Production Economics, Elsevier, vol. 125(1), pages 185-189, May.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jsusta:v:14:y:2022:i:9:p:5177-:d:801702. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.