IDEAS home Printed from https://ideas.repec.org/a/eee/chsofr/v167y2023ics0960077922012139.html
   My bibliography  Save this article

Reinforcement learning-based decentralized fault tolerant control for constrained interconnected nonlinear systems

Author

Listed:
  • Zhao, Yanwei
  • Wang, Huanqing
  • Xu, Ning
  • Zong, Guangdeng
  • Zhao, Xudong

Abstract

This paper addresses the decentralized fault tolerant control problem for interconnected nonlinear systems under a reinforcement learning strategy. The system under consideration includes unknown actuator faults and asymmetric input constraints. By constructing an improved cost function related to the estimation of the actuator faults for each auxiliary subsystem, the original control issue is converted into finding an array of decentralized optimal control policies. Then, we prove that these optimal control policies can ensure the entire system to be stable in the sense of uniform ultimate boundedness. Moreover, a single critic network architecture is developed to acquire the solutions of Hamilton–Jacobi–Bellman equations, which simplifies the architecture of the reinforcement learning algorithm. All signals in the closed-loop auxiliary subsystems are uniformly ultimately bounded based on the Lyapunov theory, and numerical and practical simulation examples are conducted to validate the effectiveness of the designed method.

Suggested Citation

  • Zhao, Yanwei & Wang, Huanqing & Xu, Ning & Zong, Guangdeng & Zhao, Xudong, 2023. "Reinforcement learning-based decentralized fault tolerant control for constrained interconnected nonlinear systems," Chaos, Solitons & Fractals, Elsevier, vol. 167(C).
  • Handle: RePEc:eee:chsofr:v:167:y:2023:i:c:s0960077922012139
    DOI: 10.1016/j.chaos.2022.113034
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0960077922012139
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.chaos.2022.113034?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Tian, Meng & Dong, Zhengcheng & Wang, Xianpei, 2021. "Reinforcement learning approach for robustness analysis of complex networks with incomplete information," Chaos, Solitons & Fractals, Elsevier, vol. 144(C).
    2. Hadipour Lakmesari, S. & Mahmoodabadi, M.J. & Yousef Ibrahim, M., 2021. "Fuzzy logic and gradient descent-based optimal adaptive robust controller with inverted pendulum verification," Chaos, Solitons & Fractals, Elsevier, vol. 151(C).
    3. Hung, Meei-Ling & Yan, Jun-Juh, 2007. "Decentralized model-reference adaptive control for a class of uncertain large-scale time-varying delayed systems with series nonlinearities," Chaos, Solitons & Fractals, Elsevier, vol. 33(5), pages 1558-1568.
    4. Yulin Li & Ben Niu & Guangdeng Zong & Jinfeng Zhao & Xudong Zhao, 2022. "Command filter-based adaptive neural finite-time control for stochastic nonlinear systems with time-varying full-state constraints and asymmetric input saturation," International Journal of Systems Science, Taylor & Francis Journals, vol. 53(1), pages 199-221, January.
    5. Xi, Changjiang & Zhai, Ding & Li, Xiaojian & Zhang, Qingling, 2017. "Decentralized adaptive delay-dependent neural network control for a class of large-scale interconnected nonlinear systems," Applied Mathematics and Computation, Elsevier, vol. 311(C), pages 148-163.
    6. Yanwei Zhao & Haoyan Zhang & Zhongyu Chen & Huanqing Wang & Xudong Zhao, 2022. "Adaptive neural decentralised control for switched interconnected nonlinear systems with backlash-like hysteresis and output constraints," International Journal of Systems Science, Taylor & Francis Journals, vol. 53(7), pages 1545-1561, May.
    7. Cui, Lili & Zhang, Yong & Wang, Xiaowei & Xie, Xiangpeng, 2021. "Event-triggered distributed self-learning robust tracking control for uncertain nonlinear interconnected systems," Applied Mathematics and Computation, Elsevier, vol. 395(C).
    8. Pishro, Aboozar & Shahrokhi, Mohammad & Sadeghi, Hamed, 2022. "Fault-tolerant adaptive fractional controller design for incommensurate fractional-order nonlinear dynamic systems subject to input and output restrictions," Chaos, Solitons & Fractals, Elsevier, vol. 157(C).
    9. Cui, Lili & Xie, Xiangpeng & Wang, Xiaowei & Luo, Yanhong & Liu, Jingbo, 2019. "Event-triggered single-network ADP method for constrained optimal tracking control of continuous-time non-linear systems," Applied Mathematics and Computation, Elsevier, vol. 352(C), pages 220-234.
    10. Heydari, M.H. & Razzaghi, M., 2021. "A numerical approach for a class of nonlinear optimal control problems with piecewise fractional derivative," Chaos, Solitons & Fractals, Elsevier, vol. 152(C).
    11. Qiuye Wu & Bo Zhao & Derong Liu, 2020. "Adaptive dynamic programming-based decentralised control for large-scale nonlinear systems subject to mismatched interconnections with unknown time-delay," International Journal of Systems Science, Taylor & Francis Journals, vol. 51(15), pages 2883-2898, November.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Pessim, Paulo S.P. & Coutinho, Pedro Henrique Silva & Lacerda, Márcio J. & Palhares, Reinaldo Martínez, 2023. "Distributed event-triggered fuzzy control for nonlinear interconnected systems," Chaos, Solitons & Fractals, Elsevier, vol. 177(C).
    2. Rahimi, Farshad, 2024. "Adaptive dynamic programming-based fault tolerant control for nonlinear time-delay systems," Chaos, Solitons & Fractals, Elsevier, vol. 188(C).
    3. Liang Cao & Shuangyin Liu & Longqin Xu, 2024. "An Intelligent Fault-Tolerant Control Method for a Flexible-Link Manipulator with an Uncertain Dead-Zone and Intermittent Actuator Faults," Mathematics, MDPI, vol. 12(14), pages 1-18, July.
    4. Yin, Rongrong & Wang, Yumeng & Li, Linhui & Zhang, Le & Hao, Zhenyang & Lang, Chun, 2024. "A mobile node path optimization approach based on Q-learning to defend against cascading failures on static-mobile networks," Chaos, Solitons & Fractals, Elsevier, vol. 182(C).
    5. Dong, Qian & Yuan, Wenying & Tong, Tianchi & Sun, Jinsheng, 2024. "The distributed fault diagnosis for nonlinear complex dynamical networks based on σ-detection criterion," Chaos, Solitons & Fractals, Elsevier, vol. 186(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Liu, Shanlin & Niu, Ben & Zong, Guangdeng & Zhao, Xudong & Xu, Ning, 2022. "Adaptive fixed-time hierarchical sliding mode control for switched under-actuated systems with dead-zone constraints via event-triggered strategy," Applied Mathematics and Computation, Elsevier, vol. 435(C).
    2. Mazare, Mahmood, 2024. "Adaptive optimal secure wind power generation control for variable speed wind turbine systems via reinforcement learning," Applied Energy, Elsevier, vol. 353(PA).
    3. Guo, Shiyu & Zhao, Xudong & Wang, Huanqing & Xu, Ning, 2023. "Distributed consensus of heterogeneous switched nonlinear multiagent systems with input quantization and DoS attacks," Applied Mathematics and Computation, Elsevier, vol. 456(C).
    4. Wojciech Giernacki, 2022. "Minimum Energy Control of Quadrotor UAV: Synthesis and Performance Analysis of Control System with Neurobiologically Inspired Intelligent Controller (BELBIC)," Energies, MDPI, vol. 15(20), pages 1-23, October.
    5. Zeghlache, Samir & Ghellab, Mohammed Zinelaabidine & Djerioui, Ali & Bouderah, Brahim & Benkhoris, Mohamed Fouad, 2023. "Adaptive fuzzy fast terminal sliding mode control for inverted pendulum-cart system with actuator faults," Mathematics and Computers in Simulation (MATCOM), Elsevier, vol. 210(C), pages 207-234.
    6. Pishro, Aboozar & Shahrokhi, Mohammad & Mohit, Mohammaderfan, 2023. "Adaptive neural quantized control for fractional-order full-state constrained non-strict feedback systems subject to input fault and nonlinearity," Chaos, Solitons & Fractals, Elsevier, vol. 166(C).
    7. Zhai, Ganghui & Tian, Engang & Luo, Yuqiang & Liang, Dong, 2024. "Data-driven optimal output regulation for unknown linear discrete-time systems based on parameterization approach," Applied Mathematics and Computation, Elsevier, vol. 461(C).
    8. Marzban, Hamid Reza, 2022. "A generalization of Müntz-Legendre polynomials and its implementation in optimal control of nonlinear fractional delay systems," Chaos, Solitons & Fractals, Elsevier, vol. 158(C).
    9. Ahmad Taher Azar & Farah Ayad Abdul-Majeed & Hasan Sh. Majdi & Ibrahim A. Hameed & Nashwa Ahmad Kamal & Anwar Jaafar Mohamad Jawad & Ali Hashim Abbas & Wameedh Riyadh Abdul-Adheem & Ibraheem Kasim Ibr, 2022. "Parameterization of a Novel Nonlinear Estimator for Uncertain SISO Systems with Noise Scenario," Mathematics, MDPI, vol. 10(13), pages 1-17, June.
    10. Jinxia Wu & Pengfei Cui, 2024. "Cooperative Adaptive Fuzzy Control for the Synchronization of Nonlinear Multi-Agent Systems under Input Saturation," Mathematics, MDPI, vol. 12(10), pages 1-14, May.
    11. Yan, Xuehua & Song, Xinmin & Wang, Zhonghua & Zhang, Yun, 2017. "Global Output-Feedback Adaptive Stabilization for Planar Nonlinear Systems with Unknown Growth Rate and Output Function," Applied Mathematics and Computation, Elsevier, vol. 314(C), pages 299-309.
    12. Sakthivel, R. & Sweetha, S. & Tatar, N.E. & Panneerselvam, V., 2023. "Delayed reset control design for uncertain fractional-order systems with actuator faults via dynamic output feedback scheme," Chaos, Solitons & Fractals, Elsevier, vol. 169(C).
    13. Xu, Bo & Liang, Yanjun & Li, Yuan-Xin & Hou, Zhongsheng, 2022. "Adaptive command filtered fixed-time control of nonlinear systems with input quantization," Applied Mathematics and Computation, Elsevier, vol. 427(C).
    14. Yin, Rongrong & Wang, Yumeng & Li, Linhui & Zhang, Le & Hao, Zhenyang & Lang, Chun, 2024. "A mobile node path optimization approach based on Q-learning to defend against cascading failures on static-mobile networks," Chaos, Solitons & Fractals, Elsevier, vol. 182(C).
    15. Yoo, Sung Jin, 2021. "Decentralized event-triggered adaptive control of a class of uncertain interconnected nonlinear systems using local state feedback corrupted by unknown injection data," Applied Mathematics and Computation, Elsevier, vol. 399(C).
    16. Tu, Haicheng & Gu, Fengqiang & Zhang, Xi & Xia, Yongxiang, 2023. "Robustness analysis of power system under sequential attacks with incomplete information," Reliability Engineering and System Safety, Elsevier, vol. 232(C).
    17. Mahmoodabadi, M.J., 2023. "An optimal robust fuzzy adaptive integral sliding mode controller based upon a multi-objective grey wolf optimization algorithm for a nonlinear uncertain chaotic system," Chaos, Solitons & Fractals, Elsevier, vol. 167(C).
    18. Weijun Hu & Jiale Quan & Xianlong Ma & Mostafa M. Salah & Ahmed Shaker, 2023. "Analytical Design of Optimal Model Predictive Control and Its Application in Small-Scale Helicopters," Mathematics, MDPI, vol. 11(8), pages 1-15, April.
    19. Gu, Yang & Shao, Yiyu & Li, Liwei & Shen, Mouquan, 2024. "Event-triggered fault tolerant control for Markov jump systems via a proportional–integral intermediate estimator," Chaos, Solitons & Fractals, Elsevier, vol. 180(C).
    20. Li, Yuanen & Zhang, Huasheng & Xie, Xiangpeng & Xia, Jianwei, 2023. "Stability analysis of a cart-pendulum model with variable convergence rate: A sliding mode control approach for impulsive stochastic systems," Chaos, Solitons & Fractals, Elsevier, vol. 175(P2).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:chsofr:v:167:y:2023:i:c:s0960077922012139. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Thayer, Thomas R. (email available below). General contact details of provider: https://www.journals.elsevier.com/chaos-solitons-and-fractals .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.