IDEAS home Printed from https://ideas.repec.org/a/gam/jmathe/v12y2024i16p2557-d1459250.html
   My bibliography  Save this article

Dynamic Target Assignment by Unmanned Surface Vehicles Based on Reinforcement Learning

Author

Listed:
  • Tao Hu

    (National Key Laboratory of Information Systems Engineering, National University of Defense Technology, Changsha 410073, China
    These authors contributed equally to this work.)

  • Xiaoxue Zhang

    (National Key Laboratory of Information Systems Engineering, National University of Defense Technology, Changsha 410073, China
    These authors contributed equally to this work.)

  • Xueshan Luo

    (National Key Laboratory of Information Systems Engineering, National University of Defense Technology, Changsha 410073, China)

  • Tao Chen

    (National Key Laboratory of Information Systems Engineering, National University of Defense Technology, Changsha 410073, China)

Abstract

Due to the dynamic complexities of the multi-unmanned vessel target assignment problem at sea, especially when addressing moving targets, traditional optimization algorithms often fail to quickly find an adequate solution. To overcome this, we have developed a multi-agent reinforcement learning algorithm. This approach involves defining a state space, employing preferential experience replay, and integrating self-attention mechanisms, which are applied to a novel offshore unmanned vessel model designed for dynamic target allocation. We have conducted a thorough analysis of strike positions and times, establishing robust mathematical models. Additionally, we designed several experiments to test the effectiveness of the algorithm. The proposed algorithm improves the quality of the solution by at least 30% in larger scale scenarios compared to the genetic algorithm (GA), and the average solution speed is less than 10% of the GA, demonstrating the feasibility of the algorithm in solving the problem.

Suggested Citation

  • Tao Hu & Xiaoxue Zhang & Xueshan Luo & Tao Chen, 2024. "Dynamic Target Assignment by Unmanned Surface Vehicles Based on Reinforcement Learning," Mathematics, MDPI, vol. 12(16), pages 1-20, August.
    • Handle: RePEc:gam:jmathe:v:12:y:2024:i:16:p:2557-:d:1459250
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2227-7390/12/16/2557/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2227-7390/12/16/2557/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Grangier, Philippe & Gendreau, Michel & Lehuédé, Fabien & Rousseau, Louis-Martin, 2016. "An adaptive large neighborhood search for the two-echelon multiple-trip vehicle routing problem with satellite synchronization," European Journal of Operational Research, Elsevier, vol. 254(1), pages 80-91.
    2. Žulj, Ivan & Kramer, Sergej & Schneider, Michael, 2018. "A hybrid of adaptive large neighborhood search and tabu search for the order-batching problem," European Journal of Operational Research, Elsevier, vol. 264(2), pages 653-664.
    3. Hao Xu & Qinghua Xing & Zhenhao Tian, 2017. "MOQPSO-D/S for Air and Missile Defense WTA Problem under Uncertainty," Mathematical Problems in Engineering, Hindawi, vol. 2017, pages 1-13, December.
    4. Paraskevopoulos, Dimitris C. & Laporte, Gilbert & Repoussis, Panagiotis P. & Tarantilis, Christos D., 2017. "Resource constrained routing and scheduling: Review and research prospects," European Journal of Operational Research, Elsevier, vol. 263(3), pages 737-754.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Singh, Nitish & Dang, Quang-Vinh & Akcay, Alp & Adan, Ivo & Martagan, Tugce, 2022. "A matheuristic for AGV scheduling with battery constraints," European Journal of Operational Research, Elsevier, vol. 298(3), pages 855-873.
    2. Soares, Ricardo & Marques, Alexandra & Amorim, Pedro & Parragh, Sophie N., 2024. "Synchronisation in vehicle routing: Classification schema, modelling framework and literature review," European Journal of Operational Research, Elsevier, vol. 313(3), pages 817-840.
    3. Turkeš, Renata & Sörensen, Kenneth & Hvattum, Lars Magnus, 2021. "Meta-analysis of metaheuristics: Quantifying the effect of adaptiveness in adaptive large neighborhood search," European Journal of Operational Research, Elsevier, vol. 292(2), pages 423-442.
    4. Dumez, Dorian & Tilk, Christian & Irnich, Stefan & Lehuédé, Fabien & Olkis, Katharina & Péton, Olivier, 2023. "A matheuristic for a 2-echelon vehicle routing problem with capacitated satellites and reverse flows," European Journal of Operational Research, Elsevier, vol. 305(1), pages 64-84.
    5. Vidal, Thibaut & Laporte, Gilbert & Matl, Piotr, 2020. "A concise guide to existing and emerging vehicle routing problem variants," European Journal of Operational Research, Elsevier, vol. 286(2), pages 401-416.
    6. SteadieSeifi, M. & Dellaert, N.P. & Nuijten, W. & Van Woensel, T., 2017. "A metaheuristic for the multimodal network flow problem with product quality preservation and empty repositioning," Transportation Research Part B: Methodological, Elsevier, vol. 106(C), pages 321-344.
    7. He, Dongdong & Guan, Wei, 2023. "Promoting service quality with incentive contracts in rural bus integrated passenger-freight service," Transportation Research Part A: Policy and Practice, Elsevier, vol. 175(C).
    8. Zhu, Stuart X. & Ursavas, Evrim, 2018. "Design and analysis of a satellite network with direct delivery in the pharmaceutical industry," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 116(C), pages 190-207.
    9. Li, Hongqi & Zhang, Lu & Lv, Tan & Chang, Xinyu, 2016. "The two-echelon time-constrained vehicle routing problem in linehaul-delivery systems," Transportation Research Part B: Methodological, Elsevier, vol. 94(C), pages 169-188.
    10. Yue Lu & Maoxiang Lang & Xueqiao Yu & Shiqi Li, 2019. "A Sustainable Multimodal Transport System: The Two-Echelon Location-Routing Problem with Consolidation in the Euro–China Expressway," Sustainability, MDPI, vol. 11(19), pages 1-25, October.
    11. Qin, Hu & Moriakin, Anton & Xu, Gangyan & Li, Jiliu, 2024. "The generator distribution problem for base stations during emergency power outage: A branch-and-price-and-cut approach," European Journal of Operational Research, Elsevier, vol. 318(3), pages 752-767.
    12. Albert H. Schrotenboer & Evrim Ursavas & Iris F. A. Vis, 2019. "A Branch-and-Price-and-Cut Algorithm for Resource-Constrained Pickup and Delivery Problems," Transportation Science, INFORMS, vol. 53(4), pages 1001-1022, July.
    13. Chen, Enming & Zhou, Zhongbao & Li, Ruiyang & Chang, Zhongxiang & Shi, Jianmai, 2024. "The multi-fleet delivery problem combined with trucks, tricycles, and drones for last-mile logistics efficiency requirements under multiple budget constraints," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 187(C).
    14. Raeesi, Ramin & Zografos, Konstantinos G., 2020. "The electric vehicle routing problem with time windows and synchronised mobile battery swapping," Transportation Research Part B: Methodological, Elsevier, vol. 140(C), pages 101-129.
    15. Michael Drexl, 2018. "On Testing Capacity Constraints in Pickup-and-Delivery Problems with Trailers in Amortized Constant Time," Working Papers 1823, Gutenberg School of Management and Economics, Johannes Gutenberg-Universität Mainz.
    16. Kallestad, Jakob & Hasibi, Ramin & Hemmati, Ahmad & Sörensen, Kenneth, 2023. "A general deep reinforcement learning hyperheuristic framework for solving combinatorial optimization problems," European Journal of Operational Research, Elsevier, vol. 309(1), pages 446-468.
    17. Ruf, Moritz & Cordeau, Jean-François, 2021. "Adaptive large neighborhood search for integrated planning in railroad classification yards," Transportation Research Part B: Methodological, Elsevier, vol. 150(C), pages 26-51.
    18. Zhang, Lele & Ding, Pengyuan & Thompson, Russell G., 2023. "A stochastic formulation of the two-echelon vehicle routing and loading bay reservation problem," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 177(C).
    19. Zhang, Yimeng & Li, Xinlei & van Hassel, Edwin & Negenborn, Rudy R. & Atasoy, Bilge, 2022. "Synchromodal transport planning considering heterogeneous and vague preferences of shippers," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 164(C).
    20. Hendri Sutrisno & Chao-Lung Yang, 2023. "A two-echelon location routing problem with mobile satellites for last-mile delivery: mathematical formulation and clustering-based heuristic method," Annals of Operations Research, Springer, vol. 323(1), pages 203-228, April.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jmathe:v:12:y:2024:i:16:p:2557-:d:1459250. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.