IDEAS home Printed from https://ideas.repec.org/a/spr/joinma/v35y2024i8d10.1007_s10845-023-02258-2.html
   My bibliography  Save this article

Reinforcement learning for sustainability enhancement of production lines

Author

Listed:
  • Alberto Loffredo

    (Politecnico di Milano)

  • Marvin Carl May

    (Karlsruhe Institute of Technology (KIT))

  • Andrea Matta

    (Politecnico di Milano)

  • Gisela Lanza

    (Karlsruhe Institute of Technology (KIT))

Abstract

The importance of sustainability in industry is dramatically rising in recent years. Controlling machine states to achieve the best trade-off between production rate and energy demand is an effective method for improving the energy efficiency of production systems. This technique is referred to as energy-efficient control (EEC) and it triggers machines in a standby state with low power requests. Reinforcement Learning (RL) algorithms can be used to successfully control production systems without the requirement of prior knowledge about system parameters. Due to the difficulty in acquiring comprehensive information about system dynamics in real-world scenarios, this is considered an important factor. The goal of this work is to create a novel RL-based model to apply EEC to multi-stage production lines with parallel machine workstations without relying on full knowledge of the system dynamics. Numerical results confirm model benefits when applied to a real line from the automotive sector. Further experiments confirm the effectiveness and generality of the approach.

Suggested Citation

  • Alberto Loffredo & Marvin Carl May & Andrea Matta & Gisela Lanza, 2024. "Reinforcement learning for sustainability enhancement of production lines," Journal of Intelligent Manufacturing, Springer, vol. 35(8), pages 3775-3791, December.
    • Handle: RePEc:spr:joinma:v:35:y:2024:i:8:d:10.1007_s10845-023-02258-2
    DOI: 10.1007/s10845-023-02258-2
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10845-023-02258-2
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s10845-023-02258-2?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Zhiyang Jia & Liang Zhang & Jorge Arinez & Guoxian Xiao, 2016. "Performance analysis for serial production lines with Bernoulli Machines and Real-time WIP-based Machine switch-on/off control," International Journal of Production Research, Taylor & Francis Journals, vol. 54(21), pages 6285-6301, November.
    2. Andreas Kuhnle & Jan-Philipp Kaiser & Felix Theiß & Nicole Stricker & Gisela Lanza, 2021. "Designing an adaptive production control system using reinforcement learning," Journal of Intelligent Manufacturing, Springer, vol. 32(3), pages 855-876, March.
    3. Hu, Luoke & Peng, Chen & Evans, Steve & Peng, Tao & Liu, Ying & Tang, Renzhong & Tiwari, Ashutosh, 2017. "Minimising the machining energy consumption of a machine tool by sequencing the features of a part," Energy, Elsevier, vol. 121(C), pages 292-305.
    4. Marcel Panzer & Benedict Bender, 2022. "Deep reinforcement learning in production systems: a systematic literature review," International Journal of Production Research, Taylor & Francis Journals, vol. 60(13), pages 4316-4341, July.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Sebastian Mayer & Tobias Classen & Christian Endisch, 2021. "Modular production control using deep reinforcement learning: proximal policy optimization," Journal of Intelligent Manufacturing, Springer, vol. 32(8), pages 2335-2351, December.
    2. Konstantinos S. Boulas & Georgios D. Dounias & Chrissoleon T. Papadopoulos, 2023. "A hybrid evolutionary algorithm approach for estimating the throughput of short reliable approximately balanced production lines," Journal of Intelligent Manufacturing, Springer, vol. 34(2), pages 823-852, February.
    3. Cai, Wei & Liu, Fei & Xie, Jun & Liu, Peiji & Tuo, Junbo, 2017. "A tool for assessing the energy demand and efficiency of machining systems: Energy benchmarking," Energy, Elsevier, vol. 138(C), pages 332-347.
    4. Shang, Zhendong & Gao, Dong & Jiang, Zhipeng & Lu, Yong, 2019. "Towards less energy intensive heavy-duty machine tools: Power consumption characteristics and energy-saving strategies," Energy, Elsevier, vol. 178(C), pages 263-276.
    5. Cai, Wei & Wang, Lianguo & Li, Li & Xie, Jun & Jia, Shun & Zhang, Xugang & Jiang, Zhigang & Lai, Kee-hung, 2022. "A review on methods of energy performance improvement towards sustainable manufacturing from perspectives of energy monitoring, evaluation, optimization and benchmarking," Renewable and Sustainable Energy Reviews, Elsevier, vol. 159(C).
    6. He, Zhiliang & Thürer, Matthias & Zhou, Wanling, 2024. "The use of reinforcement learning for material flow control: An assessment by simulation," International Journal of Production Economics, Elsevier, vol. 274(C).
    7. Seokgi Lee & Mona Issabakhsh & Hyun Woo Jeon & Seong Wook Hwang & Byung Chung, 2020. "Idle time and capacity control for a single machine scheduling problem with dynamic electricity pricing," Operations Management Research, Springer, vol. 13(3), pages 197-217, December.
    8. Beixin Xia & Yuan Li & Jiayi Gu & Yunfang Peng, 2024. "Research on Sustainable Scheduling of Material-Handling Systems in Mixed-Model Assembly Workshops Based on Deep Reinforcement Learning," Sustainability, MDPI, vol. 16(22), pages 1-15, November.
    9. Tangbin Xia & Xiangxin An & Huaqiang Yang & Yimin Jiang & Yuhui Xu & Meimei Zheng & Ershun Pan, 2023. "Efficient Energy Use in Manufacturing Systems—Modeling, Assessment, and Management Strategy," Energies, MDPI, vol. 16(3), pages 1-20, January.
    10. Jia, Shun & Yuan, Qinghe & Lv, Jingxiang & Liu, Ying & Ren, Dawei & Zhang, Zhongwei, 2017. "Therblig-embedded value stream mapping method for lean energy machining," Energy, Elsevier, vol. 138(C), pages 1081-1098.
    11. Cai, Wei & Liu, Fei & Zhang, Hua & Liu, Peiji & Tuo, Junbo, 2017. "Development of dynamic energy benchmark for mass production in machining systems for energy management and energy-efficiency improvement," Applied Energy, Elsevier, vol. 202(C), pages 715-725.
    12. Minhyeok Lee, 2023. "Game-Theoretical Analysis of Reviewer Rewards in Peer-Review Journal Systems: Analysis and Experimental Evaluation using Deep Reinforcement Learning," Papers 2305.12088, arXiv.org.
    13. Zhang, Jiaqi & Han, Xin & Li, Li & Jia, Shun & Jiang, Zhigang & Duan, Xiangmin & Lai, Kee-hung & Cai, Wei, 2023. "Multi-objective optimisation for energy saving and high efficiency production oriented multidirectional turning based on improved fireworks algorithm considering energy, efficiency and quality," Energy, Elsevier, vol. 284(C).
    14. Meihang Zhang & Hua Zhang & Wei Yan & Zhigang Jiang & Shuo Zhu, 2023. "An Integrated Deep-Learning-Based Approach for Energy Consumption Prediction of Machining Systems," Sustainability, MDPI, vol. 15(7), pages 1-17, March.
    15. Hu, Luoke & Liu, Ying & Lohse, Niels & Tang, Renzhong & Lv, Jingxiang & Peng, Chen & Evans, Steve, 2017. "Sequencing the features to minimise the non-cutting energy consumption in machining considering the change of spindle rotation speed," Energy, Elsevier, vol. 139(C), pages 935-946.
    16. Junfeng Wang & Zicheng Fei & Qing Chang & Shiqi Li, 2019. "Energy Saving Operation of Manufacturing System Based on Dynamic Adaptive Fuzzy Reasoning Petri Net," Energies, MDPI, vol. 12(11), pages 1-17, June.
    17. Hien Nguyen Ngoc & Ganix Lasa & Ion Iriarte, 2022. "Human-centred design in industry 4.0: case study review and opportunities for future research," Journal of Intelligent Manufacturing, Springer, vol. 33(1), pages 35-76, January.
    18. Liu, Conghu & Cai, Wei & Dinolov, Ognyan & Zhang, Cuixia & Rao, Weizhen & Jia, Shun & Li, Li & Chan, Felix T.S., 2018. "Emergy based sustainability evaluation of remanufacturing machining systems," Energy, Elsevier, vol. 150(C), pages 670-680.
    19. Chen, Xingzheng & Li, Congbo & Tang, Ying & Li, Li & Du, Yanbin & Li, Lingling, 2019. "Integrated optimization of cutting tool and cutting parameters in face milling for minimizing energy footprint and production time," Energy, Elsevier, vol. 175(C), pages 1021-1037.
    20. Viktor Ložar & Neven Hadžić & Tihomir Opetuk & Vedran Slapničar, 2021. "Accuracy of Semi-Analytical and Numerical Approaches in the Evaluation of Serial Bernoulli Production Lines," Mathematics, MDPI, vol. 9(13), pages 1-25, June.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:joinma:v:35:y:2024:i:8:d:10.1007_s10845-023-02258-2. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.