IDEAS home Printed from https://ideas.repec.org/a/gam/jmathe/v12y2024i16p2557-d1459250.html
   My bibliography  Save this article

Dynamic Target Assignment by Unmanned Surface Vehicles Based on Reinforcement Learning

Author

Listed:
  • Tao Hu

    (National Key Laboratory of Information Systems Engineering, National University of Defense Technology, Changsha 410073, China
    These authors contributed equally to this work.)

  • Xiaoxue Zhang

    (National Key Laboratory of Information Systems Engineering, National University of Defense Technology, Changsha 410073, China
    These authors contributed equally to this work.)

  • Xueshan Luo

    (National Key Laboratory of Information Systems Engineering, National University of Defense Technology, Changsha 410073, China)

  • Tao Chen

    (National Key Laboratory of Information Systems Engineering, National University of Defense Technology, Changsha 410073, China)

Abstract

Due to the dynamic complexities of the multi-unmanned vessel target assignment problem at sea, especially when addressing moving targets, traditional optimization algorithms often fail to quickly find an adequate solution. To overcome this, we have developed a multi-agent reinforcement learning algorithm. This approach involves defining a state space, employing preferential experience replay, and integrating self-attention mechanisms, which are applied to a novel offshore unmanned vessel model designed for dynamic target allocation. We have conducted a thorough analysis of strike positions and times, establishing robust mathematical models. Additionally, we designed several experiments to test the effectiveness of the algorithm. The proposed algorithm improves the quality of the solution by at least 30% in larger scale scenarios compared to the genetic algorithm (GA), and the average solution speed is less than 10% of the GA, demonstrating the feasibility of the algorithm in solving the problem.

Suggested Citation

  • Tao Hu & Xiaoxue Zhang & Xueshan Luo & Tao Chen, 2024. "Dynamic Target Assignment by Unmanned Surface Vehicles Based on Reinforcement Learning," Mathematics, MDPI, vol. 12(16), pages 1-20, August.
    • Handle: RePEc:gam:jmathe:v:12:y:2024:i:16:p:2557-:d:1459250
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2227-7390/12/16/2557/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2227-7390/12/16/2557/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Žulj, Ivan & Kramer, Sergej & Schneider, Michael, 2018. "A hybrid of adaptive large neighborhood search and tabu search for the order-batching problem," European Journal of Operational Research, Elsevier, vol. 264(2), pages 653-664.
    2. Hao Xu & Qinghua Xing & Zhenhao Tian, 2017. "MOQPSO-D/S for Air and Missile Defense WTA Problem under Uncertainty," Mathematical Problems in Engineering, Hindawi, vol. 2017, pages 1-13, December.
    3. Paraskevopoulos, Dimitris C. & Laporte, Gilbert & Repoussis, Panagiotis P. & Tarantilis, Christos D., 2017. "Resource constrained routing and scheduling: Review and research prospects," European Journal of Operational Research, Elsevier, vol. 263(3), pages 737-754.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Albert H. Schrotenboer & Evrim Ursavas & Iris F. A. Vis, 2019. "A Branch-and-Price-and-Cut Algorithm for Resource-Constrained Pickup and Delivery Problems," Transportation Science, INFORMS, vol. 53(4), pages 1001-1022, July.
    2. Michiel A. J. uit het Broek & Albert H. Schrotenboer & Bolor Jargalsaikhan & Kees Jan Roodbergen & Leandro C. Coelho, 2021. "Asymmetric Multidepot Vehicle Routing Problems: Valid Inequalities and a Branch-and-Cut Algorithm," Operations Research, INFORMS, vol. 69(2), pages 380-409, March.
    3. Amine Masmoudi, M. & Mancini, Simona & Baldacci, Roberto & Kuo, Yong-Hong, 2022. "Vehicle routing problems with drones equipped with multi-package payload compartments," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 164(C).
    4. Singh, Nitish & Dang, Quang-Vinh & Akcay, Alp & Adan, Ivo & Martagan, Tugce, 2022. "A matheuristic for AGV scheduling with battery constraints," European Journal of Operational Research, Elsevier, vol. 298(3), pages 855-873.
    5. Soares, Ricardo & Marques, Alexandra & Amorim, Pedro & Parragh, Sophie N., 2024. "Synchronisation in vehicle routing: Classification schema, modelling framework and literature review," European Journal of Operational Research, Elsevier, vol. 313(3), pages 817-840.
    6. Lei He & Mathijs Weerdt & Neil Yorke-Smith, 2020. "Time/sequence-dependent scheduling: the design and evaluation of a general purpose tabu-based adaptive large neighbourhood search algorithm," Journal of Intelligent Manufacturing, Springer, vol. 31(4), pages 1051-1078, April.
    7. Ju Chen & Yuan Gao & Mohd Shareduwan Mohd Kasihmuddin & Chengfeng Zheng & Nurul Atiqah Romli & Mohd. Asyraf Mansor & Nur Ezlin Zamri & Chuanbiao When, 2024. "MTS-PRO2SAT: Hybrid Mutation Tabu Search Algorithm in Optimizing Probabilistic 2 Satisfiability in Discrete Hopfield Neural Network," Mathematics, MDPI, vol. 12(5), pages 1-40, February.
    8. Boysen, Nils & de Koster, René & Weidinger, Felix, 2019. "Warehousing in the e-commerce era: A survey," European Journal of Operational Research, Elsevier, vol. 277(2), pages 396-411.
    9. Fabiola Regis-Hernández & Giuliana Carello & Ettore Lanzarone, 2020. "An optimization tool to dimension innovative home health care services with devices and disposable materials," Flexible Services and Manufacturing Journal, Springer, vol. 32(3), pages 561-598, September.
    10. Jalel Euchi & Malek Masmoudi & Patrick Siarry, 2022. "Home health care routing and scheduling problems: a literature review," 4OR, Springer, vol. 20(3), pages 351-389, September.
    11. Ahmad Ebrahimi & Hyun-woo Jeon & Sang-yeop Jung, 2023. "Improving Energy Consumption and Order Tardiness in Picker-to-Part Warehouses with Electric Forklifts: A Comparison of Four Evolutionary Algorithms," Sustainability, MDPI, vol. 15(13), pages 1-28, July.
    12. Qiu, Huaxin & Wang, Sutong & Yin, Yunqiang & Wang, Dujuan & Wang, Yanzhang, 2022. "A deep reinforcement learning-based approach for the home delivery and installation routing problem," International Journal of Production Economics, Elsevier, vol. 244(C).
    13. Wang, Yuan & Lei, Linfei & Zhang, Dongxiang & Lee, Loo Hay, 2020. "Towards delivery-as-a-service: Effective neighborhood search strategies for integrated delivery optimization of E-commerce and static O2O parcels," Transportation Research Part B: Methodological, Elsevier, vol. 139(C), pages 38-63.
    14. Yang, Peng & Zhao, Zhijie & Guo, Huijie, 2020. "Order batch picking optimization under different storage scenarios for e-commerce warehouses," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 136(C).
    15. Fazi, Stefano & Choudhary, Sourabh Kumar & Dong, Jing-Xin, 2023. "The multi-trip container drayage problem with synchronization for efficient empty containers re-usage," European Journal of Operational Research, Elsevier, vol. 310(1), pages 343-359.
    16. Kaibo Liang & Li Zhou & Jianglong Yang & Huwei Liu & Yakun Li & Fengmei Jing & Man Shan & Jin Yang, 2023. "Research on a Dynamic Task Update Assignment Strategy Based on a “Parts to Picker” Picking System," Mathematics, MDPI, vol. 11(7), pages 1-29, March.
    17. Žulj, Ivan & Salewski, Hagen & Goeke, Dominik & Schneider, Michael, 2022. "Order batching and batch sequencing in an AMR-assisted picker-to-parts system," European Journal of Operational Research, Elsevier, vol. 298(1), pages 182-201.
    18. Ghazaleh Ahmadi & Reza Tavakkoli-Moghaddam & Armand Baboli & Mehdi Najafi, 2022. "A decision support model for robust allocation and routing of search and rescue resources after earthquake: a case study," Operational Research, Springer, vol. 22(2), pages 1039-1081, April.
    19. Ardjmand, Ehsan & Shakeri, Heman & Singh, Manjeet & Sanei Bajgiran, Omid, 2018. "Minimizing order picking makespan with multiple pickers in a wave picking warehouse," International Journal of Production Economics, Elsevier, vol. 206(C), pages 169-183.
    20. Çağla Cergibozan & A. Serdar Tasan, 2022. "Genetic algorithm based approaches to solve the order batching problem and a case study in a distribution center," Journal of Intelligent Manufacturing, Springer, vol. 33(1), pages 137-149, January.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jmathe:v:12:y:2024:i:16:p:2557-:d:1459250. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.