IDEAS home Printed from https://ideas.repec.org/a/gam/jmathe/v13y2025i5p754-d1599535.html
   My bibliography  Save this article

Deep Reinforcement Learning-Based Multi-Agent System with Advanced Actor–Critic Framework for Complex Environment

Author

Listed:
  • Zihao Cui

    (College of Information Science and Technology, Donghua University, Shanghai 201620, China
    Engineering Research Center of Digitized Textile and Apparel Technology, Ministry of Education, Shanghai 201620, China)

  • Kailian Deng

    (College of Information Science and Technology, Donghua University, Shanghai 201620, China
    Engineering Research Center of Digitized Textile and Apparel Technology, Ministry of Education, Shanghai 201620, China)

  • Hongtao Zhang

    (College of Information Science and Technology, Donghua University, Shanghai 201620, China
    Engineering Research Center of Digitized Textile and Apparel Technology, Ministry of Education, Shanghai 201620, China
    This author is second contribution to this work.)

  • Zhongyi Zha

    (College of Information Science and Technology, Donghua University, Shanghai 201620, China
    School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, Wuhan 430074, China)

  • Sayed Jobaer

    (College of Information Science and Technology, Donghua University, Shanghai 201620, China
    Engineering Research Center of Digitized Textile and Apparel Technology, Ministry of Education, Shanghai 201620, China)

Abstract

The development of artificial intelligence (AI) game agents that use deep reinforcement learning (DRL) algorithms to process visual information for decision-making has emerged as a key research focus in both academia and industry. However, previous game agents have struggled to execute multiple commands simultaneously in a single decision, failing to accurately replicate the complex control patterns that characterize human gameplay. In this paper, we utilize the ViZDoom environment as the DRL research platform and transform the agent–environment interactions into a Partially Observable Markov Decision Process (POMDP). We introduce an advanced multi-agent deep reinforcement learning (DRL) framework, specifically a Multi-Agent Proximal Policy Optimization (MA-PPO), designed to optimize target acquisition while operating within defined ammunition and time constraints. In MA-PPO, each agent handles distinct parallel tasks with custom reward functions for performance evaluation. The agents make independent decisions while simultaneously executing multiple commands to mimic human-like gameplay behavior. Our evaluation compares MA-PPO against other DRL algorithms, showing a 30.67% performance improvement over the baseline algorithm.

Suggested Citation

  • Zihao Cui & Kailian Deng & Hongtao Zhang & Zhongyi Zha & Sayed Jobaer, 2025. "Deep Reinforcement Learning-Based Multi-Agent System with Advanced Actor–Critic Framework for Complex Environment," Mathematics, MDPI, vol. 13(5), pages 1-22, February.
  • Handle: RePEc:gam:jmathe:v:13:y:2025:i:5:p:754-:d:1599535
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2227-7390/13/5/754/pdf
    Download Restriction: no

    File URL: https://www.mdpi.com/2227-7390/13/5/754/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Kathryn Tunyasuvunakool & Jonas Adler & Zachary Wu & Tim Green & Michal Zielinski & Augustin Žídek & Alex Bridgland & Andrew Cowie & Clemens Meyer & Agata Laydon & Sameer Velankar & Gerard J. Kleywegt, 2021. "Highly accurate protein structure prediction for the human proteome," Nature, Nature, vol. 596(7873), pages 590-596, August.
    2. John Jumper & Richard Evans & Alexander Pritzel & Tim Green & Michael Figurnov & Olaf Ronneberger & Kathryn Tunyasuvunakool & Russ Bates & Augustin Žídek & Anna Potapenko & Alex Bridgland & Clemens Me, 2021. "Highly accurate protein structure prediction with AlphaFold," Nature, Nature, vol. 596(7873), pages 583-589, August.
    3. David Silver & Julian Schrittwieser & Karen Simonyan & Ioannis Antonoglou & Aja Huang & Arthur Guez & Thomas Hubert & Lucas Baker & Matthew Lai & Adrian Bolton & Yutian Chen & Timothy Lillicrap & Fan , 2017. "Mastering the game of Go without human knowledge," Nature, Nature, vol. 550(7676), pages 354-359, October.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yang, Kaiyuan & Huang, Houjing & Vandans, Olafs & Murali, Adithya & Tian, Fujia & Yap, Roland H.C. & Dai, Liang, 2023. "Applying deep reinforcement learning to the HP model for protein structure prediction," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 609(C).
    2. Zhenchong Mo & Lin Gong & Mingren Zhu & Junde Lan, 2024. "The Generative Generic-Field Design Method Based on Design Cognition and Knowledge Reasoning," Sustainability, MDPI, vol. 16(22), pages 1-34, November.
    3. Xuan-Kun Li & Jian-Xu Ma & Xiang-Yu Li & Jun-Jie Hu & Chuan-Yang Ding & Feng-Kai Han & Xiao-Min Guo & Xi Tan & Xian-Min Jin, 2024. "High-efficiency reinforcement learning with hybrid architecture photonic integrated circuit," Nature Communications, Nature, vol. 15(1), pages 1-10, December.
    4. Wanyang Dai, 2024. "Stochastic Differential Games and a Unified Forward–Backward Coupled Stochastic Partial Differential Equation with Lévy Jumps," Mathematics, MDPI, vol. 12(18), pages 1-46, September.
    5. Minkyu Shin & Jin Kim & Bas van Opheusden & Thomas L. Griffiths, 2023. "Superhuman Artificial Intelligence Can Improve Human Decision Making by Increasing Novelty," Papers 2303.07462, arXiv.org, revised Apr 2023.
    6. Cui, Tianxiang & Du, Nanjiang & Yang, Xiaoying & Ding, Shusheng, 2024. "Multi-period portfolio optimization using a deep reinforcement learning hyper-heuristic approach," Technological Forecasting and Social Change, Elsevier, vol. 198(C).
    7. Xiaoxuan Pan & Zhide Lu & Weiting Wang & Ziyue Hua & Yifang Xu & Weikang Li & Weizhou Cai & Xuegang Li & Haiyan Wang & Yi-Pu Song & Chang-Ling Zou & Dong-Ling Deng & Luyan Sun, 2023. "Deep quantum neural networks on a superconducting processor," Nature Communications, Nature, vol. 14(1), pages 1-7, December.
    8. Ye Yuan & Lei Chen & Kexu Song & Miaomiao Cheng & Ling Fang & Lingfei Kong & Lanlan Yu & Ruonan Wang & Zhendong Fu & Minmin Sun & Qian Wang & Chengjun Cui & Haojue Wang & Jiuyang He & Xiaonan Wang & Y, 2024. "Stable peptide-assembled nanozyme mimicking dual antifungal actions," Nature Communications, Nature, vol. 15(1), pages 1-17, December.
    9. Ivica Odorčić & Mohamed Belal Hamed & Sam Lismont & Lucía Chávez-Gutiérrez & Rouslan G. Efremov, 2024. "Apo and Aβ46-bound γ-secretase structures provide insights into amyloid-β processing by the APH-1B isoform," Nature Communications, Nature, vol. 15(1), pages 1-14, December.
    10. Stella Vitt & Simone Prinz & Martin Eisinger & Ulrich Ermler & Wolfgang Buckel, 2022. "Purification and structural characterization of the Na+-translocating ferredoxin: NAD+ reductase (Rnf) complex of Clostridium tetanomorphum," Nature Communications, Nature, vol. 13(1), pages 1-11, December.
    11. Pierre Azoulay & Joshua Krieger & Abhishek Nagaraj, 2024. "Old Moats for New Models: Openness, Control, and Competition in Generative Artificial Intelligence," NBER Chapters, in: Entrepreneurship and Innovation Policy and the Economy, volume 4, pages 7-46, National Bureau of Economic Research, Inc.
    12. Riya Shah & Thomas C. Panagiotou & Gregory B. Cole & Trevor F. Moraes & Brigitte D. Lavoie & Christopher A. McCulloch & Andrew Wilde, 2024. "The DIAPH3 linker specifies a β-actin network that maintains RhoA and Myosin-II at the cytokinetic furrow," Nature Communications, Nature, vol. 15(1), pages 1-17, December.
    13. Yashan Yang & Qianqian Shao & Mingcheng Guo & Lin Han & Xinyue Zhao & Aohan Wang & Xiangyun Li & Bo Wang & Ji-An Pan & Zhenguo Chen & Andrei Fokine & Lei Sun & Qianglin Fang, 2024. "Capsid structure of bacteriophage ΦKZ provides insights into assembly and stabilization of jumbo phages," Nature Communications, Nature, vol. 15(1), pages 1-12, December.
    14. Bret M. Boyd & Ian James & Kevin P. Johnson & Robert B. Weiss & Sarah E. Bush & Dale H. Clayton & Colin Dale, 2024. "Stochasticity, determinism, and contingency shape genome evolution of endosymbiotic bacteria," Nature Communications, Nature, vol. 15(1), pages 1-14, December.
    15. Jun-Yu Si & Yuan-Mei Chen & Ye-Hui Sun & Meng-Xue Gu & Mei-Ling Huang & Lu-Lu Shi & Xiao Yu & Xiao Yang & Qing Xiong & Cheng-Bao Ma & Peng Liu & Zheng-Li Shi & Huan Yan, 2024. "Sarbecovirus RBD indels and specific residues dictating multi-species ACE2 adaptiveness," Nature Communications, Nature, vol. 15(1), pages 1-15, December.
    16. Deyun Qiu & Jinxin V. Pei & James E. O. Rosling & Vandana Thathy & Dongdi Li & Yi Xue & John D. Tanner & Jocelyn Sietsma Penington & Yi Tong Vincent Aw & Jessica Yi Han Aw & Guoyue Xu & Abhai K. Tripa, 2022. "A G358S mutation in the Plasmodium falciparum Na+ pump PfATP4 confers clinically-relevant resistance to cipargamin," Nature Communications, Nature, vol. 13(1), pages 1-18, December.
    17. Shuo-Shuo Liu & Tian-Xia Jiang & Fan Bu & Ji-Lan Zhao & Guang-Fei Wang & Guo-Heng Yang & Jie-Yan Kong & Yun-Fan Qie & Pei Wen & Li-Bin Fan & Ning-Ning Li & Ning Gao & Xiao-Bo Qiu, 2024. "Molecular mechanisms underlying the BIRC6-mediated regulation of apoptosis and autophagy," Nature Communications, Nature, vol. 15(1), pages 1-16, December.
    18. Ahrum Son & Hyunsoo Kim & Jolene K. Diedrich & Casimir Bamberger & Daniel B. McClatchy & Stuart A. Lipton & John R. Yates, 2024. "Using in vivo intact structure for system-wide quantitative analysis of changes in proteins," Nature Communications, Nature, vol. 15(1), pages 1-17, December.
    19. Weizhu Huang & Nan Jin & Jia Guo & Cangsong Shen & Chanjuan Xu & Kun Xi & Léo Bonhomme & Robert B. Quast & Dan-Dan Shen & Jiao Qin & Yi-Ru Liu & Yuxuan Song & Yang Gao & Emmanuel Margeat & Philippe Ro, 2024. "Structural basis of orientated asymmetry in a mGlu heterodimer," Nature Communications, Nature, vol. 15(1), pages 1-15, December.
    20. Justin N. Vaughn & Sandra E. Branham & Brian Abernathy & Amanda M. Hulse-Kemp & Adam R. Rivers & Amnon Levi & William P. Wechter, 2022. "Graph-based pangenomics maximizes genotyping density and reveals structural impacts on fungal resistance in melon," Nature Communications, Nature, vol. 13(1), pages 1-14, December.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jmathe:v:13:y:2025:i:5:p:754-:d:1599535. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.