IDEAS home Printed from https://ideas.repec.org/a/gam/jmathe/v12y2024i16p2557-d1459250.html
   My bibliography  Save this article

Dynamic Target Assignment by Unmanned Surface Vehicles Based on Reinforcement Learning

Author

Listed:
  • Tao Hu

    (National Key Laboratory of Information Systems Engineering, National University of Defense Technology, Changsha 410073, China
    These authors contributed equally to this work.)

  • Xiaoxue Zhang

    (National Key Laboratory of Information Systems Engineering, National University of Defense Technology, Changsha 410073, China
    These authors contributed equally to this work.)

  • Xueshan Luo

    (National Key Laboratory of Information Systems Engineering, National University of Defense Technology, Changsha 410073, China)

  • Tao Chen

    (National Key Laboratory of Information Systems Engineering, National University of Defense Technology, Changsha 410073, China)

Abstract

Due to the dynamic complexities of the multi-unmanned vessel target assignment problem at sea, especially when addressing moving targets, traditional optimization algorithms often fail to quickly find an adequate solution. To overcome this, we have developed a multi-agent reinforcement learning algorithm. This approach involves defining a state space, employing preferential experience replay, and integrating self-attention mechanisms, which are applied to a novel offshore unmanned vessel model designed for dynamic target allocation. We have conducted a thorough analysis of strike positions and times, establishing robust mathematical models. Additionally, we designed several experiments to test the effectiveness of the algorithm. The proposed algorithm improves the quality of the solution by at least 30% in larger scale scenarios compared to the genetic algorithm (GA), and the average solution speed is less than 10% of the GA, demonstrating the feasibility of the algorithm in solving the problem.

Suggested Citation

  • Tao Hu & Xiaoxue Zhang & Xueshan Luo & Tao Chen, 2024. "Dynamic Target Assignment by Unmanned Surface Vehicles Based on Reinforcement Learning," Mathematics, MDPI, vol. 12(16), pages 1-20, August.
    • Handle: RePEc:gam:jmathe:v:12:y:2024:i:16:p:2557-:d:1459250
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2227-7390/12/16/2557/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2227-7390/12/16/2557/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Grangier, Philippe & Gendreau, Michel & Lehuédé, Fabien & Rousseau, Louis-Martin, 2016. "An adaptive large neighborhood search for the two-echelon multiple-trip vehicle routing problem with satellite synchronization," European Journal of Operational Research, Elsevier, vol. 254(1), pages 80-91.
    2. Žulj, Ivan & Kramer, Sergej & Schneider, Michael, 2018. "A hybrid of adaptive large neighborhood search and tabu search for the order-batching problem," European Journal of Operational Research, Elsevier, vol. 264(2), pages 653-664.
    3. Hao Xu & Qinghua Xing & Zhenhao Tian, 2017. "MOQPSO-D/S for Air and Missile Defense WTA Problem under Uncertainty," Mathematical Problems in Engineering, Hindawi, vol. 2017, pages 1-13, December.
    4. Paraskevopoulos, Dimitris C. & Laporte, Gilbert & Repoussis, Panagiotis P. & Tarantilis, Christos D., 2017. "Resource constrained routing and scheduling: Review and research prospects," European Journal of Operational Research, Elsevier, vol. 263(3), pages 737-754.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Soares, Ricardo & Marques, Alexandra & Amorim, Pedro & Parragh, Sophie N., 2024. "Synchronisation in vehicle routing: Classification schema, modelling framework and literature review," European Journal of Operational Research, Elsevier, vol. 313(3), pages 817-840.
    2. Vidal, Thibaut & Laporte, Gilbert & Matl, Piotr, 2020. "A concise guide to existing and emerging vehicle routing problem variants," European Journal of Operational Research, Elsevier, vol. 286(2), pages 401-416.
    3. Singh, Nitish & Dang, Quang-Vinh & Akcay, Alp & Adan, Ivo & Martagan, Tugce, 2022. "A matheuristic for AGV scheduling with battery constraints," European Journal of Operational Research, Elsevier, vol. 298(3), pages 855-873.
    4. Turkeš, Renata & Sörensen, Kenneth & Hvattum, Lars Magnus, 2021. "Meta-analysis of metaheuristics: Quantifying the effect of adaptiveness in adaptive large neighborhood search," European Journal of Operational Research, Elsevier, vol. 292(2), pages 423-442.
    5. Dumez, Dorian & Tilk, Christian & Irnich, Stefan & Lehuédé, Fabien & Olkis, Katharina & Péton, Olivier, 2023. "A matheuristic for a 2-echelon vehicle routing problem with capacitated satellites and reverse flows," European Journal of Operational Research, Elsevier, vol. 305(1), pages 64-84.
    6. SteadieSeifi, M. & Dellaert, N.P. & Nuijten, W. & Van Woensel, T., 2017. "A metaheuristic for the multimodal network flow problem with product quality preservation and empty repositioning," Transportation Research Part B: Methodological, Elsevier, vol. 106(C), pages 321-344.
    7. He, Dongdong & Guan, Wei, 2023. "Promoting service quality with incentive contracts in rural bus integrated passenger-freight service," Transportation Research Part A: Policy and Practice, Elsevier, vol. 175(C).
    8. Yue Lu & Maoxiang Lang & Xueqiao Yu & Shiqi Li, 2019. "A Sustainable Multimodal Transport System: The Two-Echelon Location-Routing Problem with Consolidation in the Euro–China Expressway," Sustainability, MDPI, vol. 11(19), pages 1-25, October.
    9. Qin, Hu & Moriakin, Anton & Xu, Gangyan & Li, Jiliu, 2024. "The generator distribution problem for base stations during emergency power outage: A branch-and-price-and-cut approach," European Journal of Operational Research, Elsevier, vol. 318(3), pages 752-767.
    10. Chen, Enming & Zhou, Zhongbao & Li, Ruiyang & Chang, Zhongxiang & Shi, Jianmai, 2024. "The multi-fleet delivery problem combined with trucks, tricycles, and drones for last-mile logistics efficiency requirements under multiple budget constraints," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 187(C).
    11. Raeesi, Ramin & Zografos, Konstantinos G., 2020. "The electric vehicle routing problem with time windows and synchronised mobile battery swapping," Transportation Research Part B: Methodological, Elsevier, vol. 140(C), pages 101-129.
    12. Hendri Sutrisno & Chao-Lung Yang, 2023. "A two-echelon location routing problem with mobile satellites for last-mile delivery: mathematical formulation and clustering-based heuristic method," Annals of Operations Research, Springer, vol. 323(1), pages 203-228, April.
    13. Michiel A. J. uit het Broek & Albert H. Schrotenboer & Bolor Jargalsaikhan & Kees Jan Roodbergen & Leandro C. Coelho, 2021. "Asymmetric Multidepot Vehicle Routing Problems: Valid Inequalities and a Branch-and-Cut Algorithm," Operations Research, INFORMS, vol. 69(2), pages 380-409, March.
    14. Li, Hongqi & Wang, Haotian & Chen, Jun & Bai, Ming, 2020. "Two-echelon vehicle routing problem with time windows and mobile satellites," Transportation Research Part B: Methodological, Elsevier, vol. 138(C), pages 179-201.
    15. Nan Ding & Manman Li & Jianming Hao, 2023. "A Two-Phase Approach to Routing a Mixed Fleet with Intermediate Depots," Mathematics, MDPI, vol. 11(8), pages 1-21, April.
    16. Magnus Bolstad Holm & Carl Axel Benjamin Medbøen & Kjetil Fagerholt & Peter Schütz, 2019. "Shortsea liner network design with transhipments at sea: a case study from Western Norway," Flexible Services and Manufacturing Journal, Springer, vol. 31(3), pages 598-619, September.
    17. Michael Drexl, 2018. "On the One-to-One Pickup-and-Delivery Problem with Time Windows and Trailers," Working Papers 1816, Gutenberg School of Management and Economics, Johannes Gutenberg-Universität Mainz.
    18. Julian Hof & Michael Schneider, 2021. "Intraroute Resource Replenishment with Mobile Depots," Transportation Science, INFORMS, vol. 55(3), pages 660-686, May.
    19. Fabiola Regis-Hernández & Giuliana Carello & Ettore Lanzarone, 2020. "An optimization tool to dimension innovative home health care services with devices and disposable materials," Flexible Services and Manufacturing Journal, Springer, vol. 32(3), pages 561-598, September.
    20. Li, Hongqi & Liu, Yinying & Jian, Xiaorong & Lu, Yingrong, 2018. "The two-echelon distribution system considering the real-time transshipment capacity varying," Transportation Research Part B: Methodological, Elsevier, vol. 110(C), pages 239-260.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jmathe:v:12:y:2024:i:16:p:2557-:d:1459250. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.