IDEAS home Printed from https://ideas.repec.org/a/spr/joinma/v35y2024i8d10.1007_s10845-023-02285-z.html
   My bibliography  Save this article

Enterprise and service−level scheduling of robot production services in cloud manufacturing with deep reinforcement learning

Author

Listed:
  • Yaoyao Ping

    (Xidian University)

  • Yongkui Liu

    (Xidian University)

  • Lin Zhang

    (Beihang University)

  • Lihui Wang

    (KTH Royal Institute of Technology)

  • Xun Xu

    (The University of Auckland)

Abstract

Cloud manufacturing is a manufacturing paradigm that integrates wide-area distributed manufacturing resources for distributed services over the Internet. Scheduling is a critical technique that determines the overall performance of a cloud manufacturing system. Robots are an important type of manufacturing resource in cloud manufacturing. Scheduling of robot production services is therefore an important research issue in cloud manufacturing. In cloud manufacturing, services can be selected at an enterprise level or a service level, which represents two types of ways of scheduling. Which way is better and how to select the optimal robot production services are issues that have rarely been considered. Recently, deep reinforcement learning (DRL) has been successfully applied to solving various scheduling problems from different fields. Given this, this paper investigates enterprise and service-level scheduling of robot production services in cloud manufacturing and explores the optimal ways and methods of scheduling with DRL. Deep Q-Networks (DQN) and its three modified algorithms, including Double DQN, Dueling DQN, and Average-DQN based on scheduling approaches are proposed. Effects of enterprise- and service-level robot production services selection methods in cloud manufacturing are studied. Comparative results indicate that overall the service-level selection method outperforms the enterprise-level method. The performance of the above-mentioned scheduling algorithms is further studied with the service-level selection method. Results indicate that the Average-DQN-based approach is able to generate scheduling solutions more efficiently and performs the best with respect to each metric.

Suggested Citation

  • Yaoyao Ping & Yongkui Liu & Lin Zhang & Lihui Wang & Xun Xu, 2024. "Enterprise and service−level scheduling of robot production services in cloud manufacturing with deep reinforcement learning," Journal of Intelligent Manufacturing, Springer, vol. 35(8), pages 3889-3916, December.
    • Handle: RePEc:spr:joinma:v:35:y:2024:i:8:d:10.1007_s10845-023-02285-z
    DOI: 10.1007/s10845-023-02285-z
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10845-023-02285-z
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s10845-023-02285-z?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Andy Ham, 2021. "Transfer-robot task scheduling in job shop," International Journal of Production Research, Taylor & Francis Journals, vol. 59(3), pages 813-823, February.
    2. Fei Qiao & Juan Liu & Yumin Ma, 2021. "Industrial big-data-driven and CPS-based adaptive production scheduling for smart manufacturing," International Journal of Production Research, Taylor & Francis Journals, vol. 59(23), pages 7139-7159, December.
    3. Zhi Li & Ali Vatankhah Barenji & Jiazhi Jiang & Ray Y. Zhong & Gangyan Xu, 2020. "A mechanism for scheduling multi robot intelligent warehouse system face with dynamic demand," Journal of Intelligent Manufacturing, Springer, vol. 31(2), pages 469-480, February.
    4. Daniel Tonke & Martin Grunow, 2018. "Maintenance, shutdown and production scheduling in semiconductor robotic cells," International Journal of Production Research, Taylor & Francis Journals, vol. 56(9), pages 3306-3325, May.
    5. Alexandre Dolgui & Dmitry Ivanov & Suresh P. Sethi & Boris Sokolov, 2019. "Scheduling in production, supply chain and Industry 4.0 systems by optimal control: fundamentals, state-of-the-art and applications," International Journal of Production Research, Taylor & Francis Journals, vol. 57(2), pages 411-432, January.
    6. Zengqiang Jiang & Shuai Yuan & Jing Ma & Qiang Wang, 2022. "The evolution of production scheduling from Industry 3.0 through Industry 4.0," International Journal of Production Research, Taylor & Francis Journals, vol. 60(11), pages 3534-3554, June.
    7. Yongkui Liu & Lihui Wang & Xi Vincent Wang & Xun Xu & Lin Zhang, 2019. "Scheduling in cloud manufacturing: state-of-the-art and research challenges," International Journal of Production Research, Taylor & Francis Journals, vol. 57(15-16), pages 4854-4879, August.
    8. Miguel Vieira & Samuel Moniz & Bruno S. Gonçalves & Tânia Pinto-Varela & Ana Paula Barbosa-Póvoa & Pedro Neto, 2022. "A two-level optimisation-simulation method for production planning and scheduling: the industrial case of a human–robot collaborative assembly line," International Journal of Production Research, Taylor & Francis Journals, vol. 60(9), pages 2942-2962, May.
    9. Foivos Psarommatis & Gökan May & Paul-Arthur Dreyfus & Dimitris Kiritsis, 2020. "Zero defect manufacturing: state-of-the-art review, shortcomings and future directions in research," International Journal of Production Research, Taylor & Francis Journals, vol. 58(1), pages 1-17, January.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Dalila B. M. M. Fontes & S. Mahdi Homayouni & Mauricio G. C. Resende, 2022. "Job-shop scheduling-joint consideration of production, transport, and storage/retrieval systems," Journal of Combinatorial Optimization, Springer, vol. 44(2), pages 1284-1322, September.
    2. Jianxin Fang & Brenda Cheang & Andrew Lim, 2023. "Problems and Solution Methods of Machine Scheduling in Semiconductor Manufacturing Operations: A Survey," Sustainability, MDPI, vol. 15(17), pages 1-44, August.
    3. Vincenzo Varriale & Antonello Cammarano & Francesca Michelino & Mauro Caputo, 2025. "Critical analysis of the impact of artificial intelligence integration with cutting-edge technologies for production systems," Journal of Intelligent Manufacturing, Springer, vol. 36(1), pages 61-93, January.
    4. Goli, Alireza, 2024. "Efficient optimization of robust project scheduling for industry 4.0: A hybrid approach based on machine learning and meta-heuristic algorithms," International Journal of Production Economics, Elsevier, vol. 278(C).
    5. Metzker Soares, Paula & Thevenin, Simon & Adulyasak, Yossiri & Dolgui, Alexandre, 2024. "Adaptive robust optimization for lot-sizing under yield uncertainty," European Journal of Operational Research, Elsevier, vol. 313(2), pages 513-526.
    6. Dong Yang & Qidong Liu & Jia Li & Yongji Jia, 2020. "Multi-Objective Optimization of Service Selection and Scheduling in Cloud Manufacturing Considering Environmental Sustainability," Sustainability, MDPI, vol. 12(18), pages 1-19, September.
    7. Runze Liu & Qi Jia & Hui Yu & Kaizhou Gao & Yaping Fu & Li Yin, 2025. "Bi-Objective Integrated Scheduling of Job Shop Problems and Material Handling Robots with Setup Time," Mathematics, MDPI, vol. 13(3), pages 1-33, January.
    8. Zhitao Xu & Adel Elomri & Roberto Baldacci & Laoucine Kerbache & Zhenyong Wu, 2024. "Frontiers and trends of supply chain optimization in the age of industry 4.0: an operations research perspective," Annals of Operations Research, Springer, vol. 338(2), pages 1359-1401, July.
    9. Rosmaini Ahmad & Rabiatul Fakhira Mohd Amin & Shaliza Azreen Mustafa, 2022. "Value stream mapping with lean thinking model for effective non-value added identification, evaluation and solution processes," Operations Management Research, Springer, vol. 15(3), pages 1490-1509, December.
    10. Hosseini, Amir & Otto, Alena & Pesch, Erwin, 2024. "Scheduling in manufacturing with transportation: Classification and solution techniques," European Journal of Operational Research, Elsevier, vol. 315(3), pages 821-843.
    11. Gokan May & Foivos Psarommatis, 2023. "Maximizing Energy Efficiency in Additive Manufacturing: A Review and Framework for Future Research," Energies, MDPI, vol. 16(10), pages 1-28, May.
    12. Wang Shijie & Zhang Yingfeng, 2021. "A credit-based dynamical evaluation method for the smart configuration of manufacturing services under Industrial Internet of Things," Journal of Intelligent Manufacturing, Springer, vol. 32(4), pages 1091-1115, April.
    13. Foivos Psarommatis & Gokan May, 2025. "A Cost–Benefit Model for Sustainable Product Reuse and Repurposing in Circular Remanufacturing," Sustainability, MDPI, vol. 17(1), pages 1-16, January.
    14. Usama Awan & Robert Sroufe & Muhammad Shahbaz, 2021. "Industry 4.0 and the circular economy: A literature review and recommendations for future research," Business Strategy and the Environment, Wiley Blackwell, vol. 30(4), pages 2038-2060, May.
    15. Siregar Martha Leni & Muskananfola Max Rudolf, 2024. "Factors Associated with Medium Trucks Casualties in the Special Region of Yogyakarta, Indonesia," LOGI – Scientific Journal on Transport and Logistics, Sciendo, vol. 15(1), pages 1-11.
    16. Dubey, Rameshwar & Gunasekaran, Angappa & Childe, Stephen J. & Bryde, David J. & Giannakis, Mihalis & Foropon, Cyril & Roubaud, David & Hazen, Benjamin T., 2020. "Big data analytics and artificial intelligence pathway to operational performance under the effects of entrepreneurial orientation and environmental dynamism: A study of manufacturing organisations," International Journal of Production Economics, Elsevier, vol. 226(C).
    17. Ke-Liang Wang & Ting-Ting Sun & Ru-Yu Xu, 2023. "The impact of artificial intelligence on total factor productivity: empirical evidence from China’s manufacturing enterprises," Economic Change and Restructuring, Springer, vol. 56(2), pages 1113-1146, April.
    18. Seokgi Lee & Hyun Woo Jeon & Mona Issabakhsh & Ahmad Ebrahimi, 2022. "An electric forklift routing problem with battery charging and energy penalty constraints," Journal of Intelligent Manufacturing, Springer, vol. 33(6), pages 1761-1777, August.
    19. R. Micale & C. M. La Fata & M. Enea & G. La Scalia, 2021. "Regenerative scheduling problem in engineer to order manufacturing: an economic assessment," Journal of Intelligent Manufacturing, Springer, vol. 32(7), pages 1913-1925, October.
    20. Pournader, Mehrdokht & Ghaderi, Hadi & Hassanzadegan, Amir & Fahimnia, Behnam, 2021. "Artificial intelligence applications in supply chain management," International Journal of Production Economics, Elsevier, vol. 241(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:joinma:v:35:y:2024:i:8:d:10.1007_s10845-023-02285-z. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.