IDEAS home Printed from https://ideas.repec.org/a/gam/jsusta/v16y2024i8p3379-d1377692.html
   My bibliography  Save this article

Tugboat Scheduling Method Based on the NRPER-DDPG Algorithm: An Integrated DDPG Algorithm with Prioritized Experience Replay and Noise Reduction

Author

Listed:
  • Jiachen Li

    (College of Navigation, Jimei Universiy, Xiamen 361021, China)

  • Xingfeng Duan

    (College of Navigation, Jimei Universiy, Xiamen 361021, China
    Maritime and Maritime Law Institute, Jimei University, Xiamen 361021, China)

  • Zhennan Xiong

    (College of Navigation, Jimei Universiy, Xiamen 361021, China)

  • Peng Yao

    (College of Navigation, Jimei Universiy, Xiamen 361021, China)

Abstract

The scheduling of harbor tugboats is a crucial task in port operations, aiming to optimize resource allocation and reduce operational costs, including fuel consumption of tugboats and the time cost of vessels waiting for operations. Due to the complexity of the port environment, traditional scheduling methods, often based on experience and practice, lack scientific and systematic decision support, making it difficult to cope with real-time changes in vessel dynamics and environmental factors. This often leads to scheduling delays and resource waste. To address this issue, this study proposes a mathematical model based on fuzzy programming, accounting for the uncertainty of the arrival time of target vessels. Additionally, we introduce the NRPER-DDPG algorithm (DDPG Algorithm with Prioritized Experience Replay and Noise Reduction), which combines a prioritized replay mechanism with a decaying noise strategy based on the DDPG algorithm. This approach optimizes the time for tugboats to reach the task location as a continuous action space, aiming to minimize the total system cost and improve scheduling efficiency. To verify the effectiveness of the mathematical model and algorithm, this study conducted experimental validation. Firstly, the optimal algorithm hyperparameter combinations were adjusted through random examples to ensure the stability and reliability of the algorithm. Subsequently, large-scale examples and actual port cases were used to further verify the performance advantages of the algorithm in practical applications. Experimental results demonstrate that the proposed mathematical model and algorithm significantly reduce system costs and improve scheduling efficiency, providing new insights and methods for the sustainable development of port operations.

Suggested Citation

  • Jiachen Li & Xingfeng Duan & Zhennan Xiong & Peng Yao, 2024. "Tugboat Scheduling Method Based on the NRPER-DDPG Algorithm: An Integrated DDPG Algorithm with Prioritized Experience Replay and Noise Reduction," Sustainability, MDPI, vol. 16(8), pages 1-27, April.
  • Handle: RePEc:gam:jsusta:v:16:y:2024:i:8:p:3379-:d:1377692
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2071-1050/16/8/3379/pdf
    Download Restriction: no

    File URL: https://www.mdpi.com/2071-1050/16/8/3379/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Wei, Xiaoyang & Jia, Shuai & Meng, Qiang & Tan, Kok Choon, 2020. "Tugboat scheduling for container ports," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 142(C).
    2. Park, Hyungjun & Choi, Dong Gu & Min, Daiki, 2023. "Adaptive inventory replenishment using structured reinforcement learning by exploiting a policy structure," International Journal of Production Economics, Elsevier, vol. 266(C).
    3. Zhu, Jianyun & Chen, Li & Wang, Bin & Xia, Lijuan, 2018. "Optimal design of a hybrid electric propulsive system for an anchor handling tug supply vessel," Applied Energy, Elsevier, vol. 226(C), pages 423-436.
    4. Tofighi, S. & Torabi, S.A. & Mansouri, S.A., 2016. "Humanitarian logistics network design under mixed uncertainty," European Journal of Operational Research, Elsevier, vol. 250(1), pages 239-250.
    5. Khalil Bachiri & Ali Yahyaouy & Hamid Gualous & Maria Malek & Younes Bennani & Philippe Makany & Nicoleta Rogovschi, 2023. "Multi-Agent DDPG Based Electric Vehicles Charging Station Recommendation," Energies, MDPI, vol. 16(16), pages 1-17, August.
    6. Kang, Liujiang & Meng, Qiang & Tan, Kok Choon, 2020. "Tugboat scheduling under ship arrival and tugging process time uncertainty," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 144(C).
    7. Jin, Jiahuan & Cui, Tianxiang & Bai, Ruibin & Qu, Rong, 2024. "Container port truck dispatching optimization using Real2Sim based deep reinforcement learning," European Journal of Operational Research, Elsevier, vol. 315(1), pages 161-175.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Guo, Zijian & Cao, Zhen & Wang, Wenyuan & Jiang, Ying & Xu, Xinglu & Feng, Peng, 2021. "An integrated model for vessel traffic and deballasting scheduling in coal export terminals," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 152(C).
    2. Hao, Luyao & Jin, Jian Gang & Zhao, Ke, 2023. "Joint scheduling of barges and tugboats for river–sea intermodal transport," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 173(C).
    3. Abou Kasm, Omar & Diabat, Ali & Bierlaire, Michel, 2021. "Vessel scheduling with pilotage and tugging considerations," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 148(C).
    4. Zhao, Ke & Jin, Jian Gang & Zhang, Di & Ji, Sheng & Lee, Der-Horng, 2023. "A variable neighborhood search heuristic for real-time barge scheduling in a river-to-sea channel with tidal restrictions," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 179(C).
    5. Chen, Shukai & Wang, Hua & Meng, Qiang, 2021. "Autonomous truck scheduling for container transshipment between two seaport terminals considering platooning and speed optimization," Transportation Research Part B: Methodological, Elsevier, vol. 154(C), pages 289-315.
    6. Liu, Baoli & Li, Zhi-Chun & Wang, Yadong, 2022. "A two-stage stochastic programming model for seaport berth and channel planning with uncertainties in ship arrival and handling times," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 167(C).
    7. Sun, Xiaojun & Yao, Chong & Song, Enzhe & Yang, Qidong & Yang, Xuchang, 2022. "Optimal control of transient processes in marine hybrid propulsion systems: Modeling, optimization and performance enhancement," Applied Energy, Elsevier, vol. 321(C).
    8. Hu, Shaolong & Han, Chuanfeng & Dong, Zhijie Sasha & Meng, Lingpeng, 2019. "A multi-stage stochastic programming model for relief distribution considering the state of road network," Transportation Research Part B: Methodological, Elsevier, vol. 123(C), pages 64-87.
    9. Li Zhu & Yeming Gong & Yishui Xu & Jun Gu, 2019. "Emergency Relief Routing Models for Injured Victims Considering Equity and Priority," Post-Print hal-02879681, HAL.
    10. Aniruddh Nain & Deepika Jain & Shivam Gupta & Ashwani Kumar, 2023. "Improving First Responders' Effectiveness in Post-Disaster Scenarios Through a Hybrid Framework for Damage Assessment and Prioritization," Global Journal of Flexible Systems Management, Springer;Global Institute of Flexible Systems Management, vol. 24(3), pages 409-437, September.
    11. Hou, Jun & Song, Ziyou & Park, Hyeongjun & Hofmann, Heath & Sun, Jing, 2018. "Implementation and evaluation of real-time model predictive control for load fluctuations mitigation in all-electric ship propulsion systems," Applied Energy, Elsevier, vol. 230(C), pages 62-77.
    12. Chen, Yingzhen & Zhao, Qiuhong & Huang, Kai & Xi, Xunzhuo, 2022. "A Bi-objective optimization model for contract design of humanitarian relief goods procurement considering extreme disasters," Socio-Economic Planning Sciences, Elsevier, vol. 81(C).
    13. Hosseini-Motlagh, Seyyed-Mahdi & Samani, Mohammad Reza Ghatreh & Homaei, Shamim, 2020. "Toward a coordination of inventory and distribution schedules for blood in disasters," Socio-Economic Planning Sciences, Elsevier, vol. 72(C).
    14. Afshin Kamyabniya & M. M. Lotfi & Mohsen Naderpour & Yuehwern Yih, 2018. "Robust Platelet Logistics Planning in Disaster Relief Operations Under Uncertainty: a Coordinated Approach," Information Systems Frontiers, Springer, vol. 20(4), pages 759-782, August.
    15. Feiyue Wang & Ziling Xie & Hui Liu & Zhongwei Pei & Dingli Liu, 2022. "Multiobjective Emergency Resource Allocation under the Natural Disaster Chain with Path Planning," IJERPH, MDPI, vol. 19(13), pages 1-19, June.
    16. Zhalechian, M. & Torabi, S. Ali & Mohammadi, M., 2018. "Hub-and-spoke network design under operational and disruption risks," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 109(C), pages 20-43.
    17. Si, Yupeng & Wang, Rongjie & Zhang, Shiqi & Zhou, Wenting & Lin, Anhui & Zeng, Guangmiao, 2022. "Configuration optimization and energy management of hybrid energy system for marine using quantum computing," Energy, Elsevier, vol. 253(C).
    18. Milovan Kovač & Snežana Tadić & Mladen Krstić & Miloš Veljović, 2023. "A Methodology for Planning City Logistics Concepts Based on City-Dry Port Micro-Consolidation Centres," Mathematics, MDPI, vol. 11(15), pages 1-21, July.
    19. Madhukar Chhimwal & Saurabh Agrawal & Girish Kumar, 2021. "Measuring Circular Supply Chain Risk: A Bayesian Network Methodology," Sustainability, MDPI, vol. 13(15), pages 1-22, July.
    20. Prima Denny Sentia & Syaimak Abdul Shukor & Amelia Natasya Abdul Wahab & Muriati Mukhtar, 2023. "Logistic distribution in humanitarian supply chain management: a thematic literature review and future research," Annals of Operations Research, Springer, vol. 323(1), pages 175-201, April.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jsusta:v:16:y:2024:i:8:p:3379-:d:1377692. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.