IDEAS home Printed from https://ideas.repec.org/a/spr/joinma/v36y2025i1d10.1007_s10845-023-02252-8.html
   My bibliography  Save this article

Research on flexible job shop scheduling problem with AGV using double DQN

Author

Listed:
  • Minghai Yuan

    (Hohai University
    Hohai University)

  • Liang Zheng

    (Hohai University
    Hohai University)

  • Hanyu Huang

    (Hohai University
    Hohai University)

  • Kaiwen Zhou

    (Hohai University
    Hohai University)

  • Fengque Pei

    (Hohai University
    Hohai University)

  • Wenbin Gu

    (Hohai University
    Hohai University)

Abstract

In the context of Industry 4.0 and intelligent manufacturing, AGVs are widely used in flexible job shop resource transportation, which sharply increases the uncertainty and complexity of the scheduling process. For this reason, an improved double Deep Q Network (DDQN) real-time scheduling method is proposed for the Flexible Job Shop Scheduling Problem with Automated Guided Vehicle (FJSP-AGV) to minimize the makespan. Firstly, the optimization model of the FJSP-AGV is established, and the corresponding constraints and the objective function are defined. Then, the FJSP-AGV is converted into a Markov Decision Process (MDP), in which the state space, action space, and reward function are defined in detail. Next, an improved DDQN is proposed to generate the optimal scheduling policy considering AGV. Finally, the computational experiments are conducted based on data from public benchmarks and the real-world flexible job shop, and the results demonstrate the accuracy and effectiveness of the proposed algorithm.

Suggested Citation

  • Minghai Yuan & Liang Zheng & Hanyu Huang & Kaiwen Zhou & Fengque Pei & Wenbin Gu, 2025. "Research on flexible job shop scheduling problem with AGV using double DQN," Journal of Intelligent Manufacturing, Springer, vol. 36(1), pages 509-535, January.
    • Handle: RePEc:spr:joinma:v:36:y:2025:i:1:d:10.1007_s10845-023-02252-8
    DOI: 10.1007/s10845-023-02252-8
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10845-023-02252-8
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s10845-023-02252-8?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Ümit Bilge & Gündüz Ulusoy, 1995. "A Time Window Approach to Simultaneous Scheduling of Machines and Material Handling System in an FMS," Operations Research, INFORMS, vol. 43(6), pages 1058-1070, December.
    2. Dalila B. M. M. Fontes & Seyed Mahdi Homayouni, 2019. "Joint production and transportation scheduling in flexible manufacturing systems," Journal of Global Optimization, Springer, vol. 74(4), pages 879-908, August.
    3. Weibo Ren & Yan Yan & Yaoguang Hu & Yu Guan, 2022. "Joint optimisation for dynamic flexible job-shop scheduling problem with transportation time and resource constraints," International Journal of Production Research, Taylor & Francis Journals, vol. 60(18), pages 5675-5696, September.
    4. Jianxun Li & Wenjie Cheng & Kin Keung Lai & Bhagwat Ram, 2022. "Multi-AGV Flexible Manufacturing Cell Scheduling Considering Charging," Mathematics, MDPI, vol. 10(19), pages 1-15, September.
    5. Renke Liu & Rajesh Piplani & Carlos Toro, 2022. "Deep reinforcement learning for dynamic scheduling of a flexible job shop," International Journal of Production Research, Taylor & Francis Journals, vol. 60(13), pages 4049-4069, July.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Hosseini, Amir & Otto, Alena & Pesch, Erwin, 2024. "Scheduling in manufacturing with transportation: Classification and solution techniques," European Journal of Operational Research, Elsevier, vol. 315(3), pages 821-843.
    2. Andy Ham, 2020. "Transfer-robot task scheduling in flexible job shop," Journal of Intelligent Manufacturing, Springer, vol. 31(7), pages 1783-1793, October.
    3. Fontes, Dalila B.M.M. & Homayouni, S. Mahdi & Gonçalves, José F., 2023. "A hybrid particle swarm optimization and simulated annealing algorithm for the job shop scheduling problem with transport resources," European Journal of Operational Research, Elsevier, vol. 306(3), pages 1140-1157.
    4. Zhangying Xu & Qi Jia & Kaizhou Gao & Yaping Fu & Li Yin & Qiangqiang Sun, 2024. "Integrated Scheduling of Multi-Objective Job Shops and Material Handling Robots with Reinforcement Learning Guided Meta-Heuristics," Mathematics, MDPI, vol. 13(1), pages 1-28, December.
    5. Jianxun Li & Wenjie Cheng & Kin Keung Lai & Bhagwat Ram, 2022. "Multi-AGV Flexible Manufacturing Cell Scheduling Considering Charging," Mathematics, MDPI, vol. 10(19), pages 1-15, September.
    6. Seyed Mahdi Homayouni & Dalila B. M. M. Fontes, 2021. "Production and transport scheduling in flexible job shop manufacturing systems," Journal of Global Optimization, Springer, vol. 79(2), pages 463-502, February.
    7. Dalila B. M. M. Fontes & S. Mahdi Homayouni & Mauricio G. C. Resende, 2022. "Job-shop scheduling-joint consideration of production, transport, and storage/retrieval systems," Journal of Combinatorial Optimization, Springer, vol. 44(2), pages 1284-1322, September.
    8. Dalila B. M. M. Fontes & Seyed Mahdi Homayouni, 2019. "Joint production and transportation scheduling in flexible manufacturing systems," Journal of Global Optimization, Springer, vol. 74(4), pages 879-908, August.
    9. Xiangtong Qi, 2005. "A logistics scheduling model: Inventory cost reduction by batching," Naval Research Logistics (NRL), John Wiley & Sons, vol. 52(4), pages 312-320, June.
    10. Vis, Iris F.A., 2006. "Survey of research in the design and control of automated guided vehicle systems," European Journal of Operational Research, Elsevier, vol. 170(3), pages 677-709, May.
    11. James T. Lin & Chun-Chih Chiu & Edward Huang & Hung-Ming Chen, 2018. "A Multi-Fidelity Model Approach for Simultaneous Scheduling of Machines and Vehicles in Flexible Manufacturing Systems," Asia-Pacific Journal of Operational Research (APJOR), World Scientific Publishing Co. Pte. Ltd., vol. 35(01), pages 1-20, February.
    12. Le-Anh, T. & de Koster, M.B.M., 2004. "A Review Of Design And Control Of Automated Guided Vehicle Systems," ERIM Report Series Research in Management ERS;2004-030-LIS, Erasmus Research Institute of Management (ERIM), ERIM is the joint research institute of the Rotterdam School of Management, Erasmus University and the Erasmus School of Economics (ESE) at Erasmus University Rotterdam.
    13. Arash Amirteimoori & Reza Kia, 2023. "Concurrent scheduling of jobs and AGVs in a flexible job shop system: a parallel hybrid PSO-GA meta-heuristic," Flexible Services and Manufacturing Journal, Springer, vol. 35(3), pages 727-753, September.
    14. Beixin Xia & Yuan Li & Jiayi Gu & Yunfang Peng, 2024. "Research on Sustainable Scheduling of Material-Handling Systems in Mixed-Model Assembly Workshops Based on Deep Reinforcement Learning," Sustainability, MDPI, vol. 16(22), pages 1-15, November.
    15. Yun, Lingxiang & Li, Lin & Ma, Shuaiyin, 2022. "Demand response for manufacturing systems considering the implications of fast-charging battery powered material handling equipment," Applied Energy, Elsevier, vol. 310(C).
    16. Du, Yu & Li, Jun-qing, 2024. "A deep reinforcement learning based algorithm for a distributed precast concrete production scheduling," International Journal of Production Economics, Elsevier, vol. 268(C).
    17. Chen, Lu & Langevin, André & Lu, Zhiqiang, 2013. "Integrated scheduling of crane handling and truck transportation in a maritime container terminal," European Journal of Operational Research, Elsevier, vol. 225(1), pages 142-152.
    18. Mohammad Reza Komari Alaei & Mehmet Soysal & Atabak Elmi & Audrius Banaitis & Nerija Banaitiene & Reza Rostamzadeh & Shima Javanmard, 2021. "A Bender’s Algorithm of Decomposition Used for the Parallel Machine Problem of Robotic Cell," Mathematics, MDPI, vol. 9(15), pages 1-15, July.
    19. Yuan Gao & Qian Zhang & Chun Kit Lau & Bhagwat Ram, 2022. "Robust Appointment Scheduling in Healthcare," Mathematics, MDPI, vol. 10(22), pages 1-15, November.
    20. James T. Lin & Chun-Chih Chiu & Yu-Hsiang Chang, 2019. "Simulation-based optimization approach for simultaneous scheduling of vehicles and machines with processing time uncertainty in FMS," Flexible Services and Manufacturing Journal, Springer, vol. 31(1), pages 104-141, March.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:joinma:v:36:y:2025:i:1:d:10.1007_s10845-023-02252-8. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.