IDEAS home Printed from https://ideas.repec.org/a/eee/apmaco/v369y2020ics0096300319308136.html
   My bibliography  Save this article

A novel optimal bipartite consensus control scheme for unknown multi-agent systems via model-free reinforcement learning

Author

Listed:
  • Peng, Zhinan
  • Hu, Jiangping
  • Shi, Kaibo
  • Luo, Rui
  • Huang, Rui
  • Ghosh, Bijoy Kumar
  • Huang, Jiuke

Abstract

In this paper, the optimal bipartite consensus control (OBCC) problem is investigated for unknown multi-agent systems (MASs) with coopetition networks. A novel distributed OBCC scheme is proposed based on model-free reinforcement learning method to achieve OBCC, where the agent’s dynamics are no longer required. First, The coopetition networks are applied to establish the cooperative and competitive interactions among agents, and then the OBCC problem is formulated by introducing local neighbor bipartite consensus errors and performance index functions (PIFs) for each agent. Second, in order to obtain the OBCC laws, a policy iteration algorithm (PIA) is employed to learn the solutions to discrete-time (DT) Hamilton-Jacobi-Bellman (HJB) equations. Third, to implement the proposed methods, we adopt a data-driven actor-critic-based neural networks (NNs) framework to approximate the control laws and the PIFs, respectively, in an online learning manner. Finally, some simulation results are given to demonstrate the effectiveness of the developed approaches.

Suggested Citation

  • Peng, Zhinan & Hu, Jiangping & Shi, Kaibo & Luo, Rui & Huang, Rui & Ghosh, Bijoy Kumar & Huang, Jiuke, 2020. "A novel optimal bipartite consensus control scheme for unknown multi-agent systems via model-free reinforcement learning," Applied Mathematics and Computation, Elsevier, vol. 369(C).
    • Handle: RePEc:eee:apmaco:v:369:y:2020:i:c:s0096300319308136
    DOI: 10.1016/j.amc.2019.124821
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0096300319308136
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.amc.2019.124821?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Yu, Zhiyong & Jiang, Haijun & Mei, Xuehui & Hu, Cheng, 2018. "Guaranteed cost consensus for second-order multi-agent systems with heterogeneous inertias," Applied Mathematics and Computation, Elsevier, vol. 338(C), pages 739-757.
    2. Hu, Jiangping & Hong, Yiguang, 2007. "Leader-following coordination of multi-agent systems with coupling time delays," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 374(2), pages 853-863.
    3. Ye, Dan & Yang, Xiang & Su, Lei, 2017. "Fault-tolerant synchronization control for complex dynamical networks with semi-Markov jump topology," Applied Mathematics and Computation, Elsevier, vol. 312(C), pages 36-48.
    4. Shi, Kaibo & Wang, Jun & Zhong, Shouming & Zhang, Xiaojun & Liu, Yajuan & Cheng, Jun, 2019. "New reliable nonuniform sampling control for uncertain chaotic neural networks under Markov switching topologies," Applied Mathematics and Computation, Elsevier, vol. 347(C), pages 169-193.
    5. Hongwen Ma & Derong Liu & Ding Wang & Biao Luo, 2016. "Bipartite output consensus in networked multi-agent systems of high-order power integrators with signed digraph and input noises," International Journal of Systems Science, Taylor & Francis Journals, vol. 47(13), pages 3116-3131, October.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Zhao, Huarong & Peng, Li & Yu, Hongnian, 2022. "Quantized model-free adaptive iterative learning bipartite consensus tracking for unknown nonlinear multi-agent systems," Applied Mathematics and Computation, Elsevier, vol. 412(C).
    2. Wang, Xiaoling & Su, Housheng, 2020. "Completely model-free RL-based consensus of continuous-time multi-agent systems," Applied Mathematics and Computation, Elsevier, vol. 382(C).
    3. Wang, Yun & Fang, Tian & Kong, Qingkai & Li, Feng, 2024. "Zero-sum game-based optimal control for discrete-time Markov jump systems: A parallel off-policy Q-learning method," Applied Mathematics and Computation, Elsevier, vol. 467(C).
    4. Li, Xiaoqing & Nguang, Sing Kiong & She, Kun & Cheng, Jun & Zhong, Shouming, 2021. "Resilient controller synthesis for Markovian jump systems with probabilistic faults and gain fluctuations under stochastic sampling operational mechanism," Applied Mathematics and Computation, Elsevier, vol. 392(C).
    5. Wang, Changlin, 2024. "Social media platform-oriented topic mining and information security analysis by big data and deep convolutional neural network," Technological Forecasting and Social Change, Elsevier, vol. 199(C).
    6. Li, Baoxing & Han, Tao & Xiao, Bo & Zhan, Xi-Sheng & Yan, Huaicheng, 2022. "Leader-following bipartite consensus of multiple uncertain Euler-Lagrange systems under deception attacks," Applied Mathematics and Computation, Elsevier, vol. 428(C).
    7. Meng, Hao & Pang, Denghao & Cao, Jinde & Guo, Yechen & Niazi, Azmat Ullah Khan, 2024. "Optimal bipartite consensus control for heterogeneous unknown multi-agent systems via reinforcement learning," Applied Mathematics and Computation, Elsevier, vol. 476(C).
    8. Jinfeng Wang & Hui Dong & Fenghua Chen & Mai The Vu & Ali Dokht Shakibjoo & Ardashir Mohammadzadeh, 2023. "Formation Control of Non-Holonomic Mobile Robots: Predictive Data-Driven Fuzzy Compensator," Mathematics, MDPI, vol. 11(8), pages 1-21, April.
    9. Hou, Rui & Cui, Lizhi & Bu, Xuhui & Yang, Junqi, 2021. "Distributed formation control for multiple non-holonomic wheeled mobile robots with velocity constraint by using improved data-driven iterative learning," Applied Mathematics and Computation, Elsevier, vol. 395(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jian, Long & Hu, Jiangping & Wang, Jun & Shi, Kaibo, 2019. "Distributed event-triggered protocols with Kx-functional observer for leader-following multi-agent systems," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 535(C).
    2. Jian, Long & Hu, Jiangping & Wang, Jun & Shi, Kaibo, 2019. "Observer-based output feedback distributed event-triggered control for linear multi-agent systems under general directed graphs," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 534(C).
    3. Jing Bai & Guoguang Wen & Ahmed Rahmani & Xing Chu & Yongguang Yu, 2016. "Consensus with a reference state for fractional-order multi-agent systems," International Journal of Systems Science, Taylor & Francis Journals, vol. 47(1), pages 222-234, January.
    4. Ruan, Xiaoli & Xu, Chen & Feng, Jianwen & Wang, Jingyi & Zhao, Yi, 2022. "Adaptive dynamic event-triggered control for multi-agent systems with matched uncertainties under directed topologies," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 586(C).
    5. Chen, Feng & Chen, Yuming & Zhu, Quanxin & Zhang, Qimin, 2023. "Stability of stochastic systems with semi-Markovian switching and impulses," Chaos, Solitons & Fractals, Elsevier, vol. 177(C).
    6. Zhang, Zhiming & Zheng, Wei & Lam, H.K. & Wen, Shuhuan & Sun, Fuchun & Xie, Ping, 2020. "Stability analysis and output feedback control for stochastic networked systems with multiple communication delays and nonlinearities using fuzzy control technique," Applied Mathematics and Computation, Elsevier, vol. 386(C).
    7. Guo, Beibei & Xiao, Yu, 2023. "Intermittent synchronization for multi-link and multi-delayed large-scale systems with semi-Markov jump and its application of Chua’s circuits," Chaos, Solitons & Fractals, Elsevier, vol. 174(C).
    8. Xia, ZeLiang & He, Shuping, 2022. "Finite-time asynchronous H∞ fault-tolerant control for nonlinear hidden markov jump systems with actuator and sensor faults," Applied Mathematics and Computation, Elsevier, vol. 428(C).
    9. Long, Mingkang & Su, Housheng & Liu, Bo, 2019. "Second-order controllability of two-time-scale multi-agent systems," Applied Mathematics and Computation, Elsevier, vol. 343(C), pages 299-313.
    10. Xi, Jianxiang & Shi, Zongying & Zhong, Yisheng, 2012. "Admissible consensus and consensualization of high-order linear time-invariant singular swarm systems," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 391(23), pages 5839-5849.
    11. Chen, Yonghui & Zhang, Xian & Xue, Yu, 2022. "Global exponential synchronization of high-order quaternion Hopfield neural networks with unbounded distributed delays and time-varying discrete delays," Mathematics and Computers in Simulation (MATCOM), Elsevier, vol. 193(C), pages 173-189.
    12. Zhang, Chunmei & Han, Bang-Sheng, 2020. "Stability analysis of stochastic delayed complex networks with multi-weights based on Razumikhin technique and graph theory," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 538(C).
    13. Zhao, Can & Liu, Xinzhi & Zhong, Shouming & Shi, Kaibo & Liao, Daixi & Zhong, Qishui, 2021. "Leader-following consensus of multi-agent systems via novel sampled-data event-triggered control," Applied Mathematics and Computation, Elsevier, vol. 395(C).
    14. Mo, Lipo & Yuan, Xiaolin & Yu, Yongguang, 2018. "Target-encirclement control of fractional-order multi-agent systems with a leader," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 509(C), pages 479-491.
    15. Xianyong Zhang & Yaohong Huang & Li Li & Wei-Chang Yeh, 2018. "Power and Capacity Consensus Tracking of Distributed Battery Storage Systems in Modular Microgrids," Energies, MDPI, vol. 11(6), pages 1-25, June.
    16. Wu, Zhihai & Peng, Li & Xie, Linbo & Wen, Jiwei, 2013. "Stochastic bounded consensus tracking of leader–follower multi-agent systems with measurement noises based on sampled-data with small sampling delay," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 392(4), pages 918-928.
    17. Wu, Tao & Cao, Jinde & Xiong, Lianglin & Zhang, Haiyang & Shu, Jinlong, 2022. "Sampled-data synchronization criteria for Markovian jumping neural networks with additive time-varying delays using new techniques," Applied Mathematics and Computation, Elsevier, vol. 413(C).
    18. He, Xiaoyan & Wang, Qingyun & Yu, Wenwu, 2015. "Finite-time distributed cooperative attitude tracking control for multiple rigid spacecraft," Applied Mathematics and Computation, Elsevier, vol. 256(C), pages 724-734.
    19. Zhang, Jiangbo & Hong, Yiguang, 2013. "Opinion evolution analysis for short-range and long-range Deffuant–Weisbuch models," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 392(21), pages 5289-5297.
    20. Wu, Zhenyu & Chen, Jiawei & Zhang, Xuexi & Xiao, Zehui & Tao, Jie & Wang, Xiaofeng, 2022. "Dynamic event-triggered synchronization of complex networks with switching topologies: Asynchronous observer-based case," Applied Mathematics and Computation, Elsevier, vol. 435(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:apmaco:v:369:y:2020:i:c:s0096300319308136. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: https://www.journals.elsevier.com/applied-mathematics-and-computation .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.