IDEAS home Printed from https://ideas.repec.org/a/eee/renene/v181y2022icp445-456.html
   My bibliography  Save this article

FALCON- FArm Level CONtrol for wind turbines using multi-agent deep reinforcement learning

Author

Listed:
  • Padullaparthi, Venkata Ramakrishna
  • Nagarathinam, Srinarayana
  • Vasan, Arunchandar
  • Menon, Vishnu
  • Sudarsanam, Depak

Abstract

Turbines in a wind farm dynamically influence each other through wakes. Therefore trade-offs exist between energy output of upstream turbines and the health of downstream turbines. Using both model-based predictive control (MPC) and machine learning techniques, existing works have explored the energy-fatigue trade-off either in a single turbine or only with few turbines due to issues of scalability and complexity. To address this gap, this paper proposes a multi-agent deep reinforcement learning-based coordinated control for wind farms, called FALCON. FALCON addresses the multi-objective optimization problem of maximizing energy while minimizing fatigue damage by jointly controlling pitch and yaw of all turbines. FALCON achieves scale by using multiple reinforcement learning agents; capturing the global state-space efficiently using an auto-encoder; and pruning the action-space using domain knowledge. FALCON is evaluated through a real-world wind-farm case study with 21 turbines; and performs better than the default baseline PID controller and a learning-based distributed control.

Suggested Citation

  • Padullaparthi, Venkata Ramakrishna & Nagarathinam, Srinarayana & Vasan, Arunchandar & Menon, Vishnu & Sudarsanam, Depak, 2022. "FALCON- FArm Level CONtrol for wind turbines using multi-agent deep reinforcement learning," Renewable Energy, Elsevier, vol. 181(C), pages 445-456.
    • Handle: RePEc:eee:renene:v:181:y:2022:i:c:p:445-456
    DOI: 10.1016/j.renene.2021.09.023
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0960148121013227
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.renene.2021.09.023?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Jinghan Cui & Su Liu & Jinfeng Liu & Xiangjie Liu, 2018. "A Comparative Study of MPC and Economic MPC of Wind Energy Conversion Systems," Energies, MDPI, vol. 11(11), pages 1-23, November.
    2. van Dijk, Mike T. & van Wingerden, Jan-Willem & Ashuri, Turaj & Li, Yaoyu, 2017. "Wind farm multi-objective wake redirection for optimizing power production and loads," Energy, Elsevier, vol. 121(C), pages 561-569.
    3. Hosseini, Ehsan & Aghadavoodi, Ehsan & Fernández Ramírez, Luis M., 2020. "Improving response of wind turbines by pitch angle controller based on gain-scheduled recurrent ANFIS type 2 with passive reinforcement learning," Renewable Energy, Elsevier, vol. 157(C), pages 897-910.
    4. Jain, Achin & Schildbach, Georg & Fagiano, Lorenzo & Morari, Manfred, 2015. "On the design and tuning of linear model predictive control for wind turbines," Renewable Energy, Elsevier, vol. 80(C), pages 664-673.
    5. Aitor Saenz-Aguirre & Ekaitz Zulueta & Unai Fernandez-Gamiz & Javier Lozano & Jose Manuel Lopez-Guede, 2019. "Artificial Neural Network Based Reinforcement Learning for Wind Turbine Yaw Control," Energies, MDPI, vol. 12(3), pages 1-17, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Kim, Taewan & Kim, Changwook & Song, Jeonghwan & You, Donghyun, 2024. "Optimal control of a wind farm in time-varying wind using deep reinforcement learning," Energy, Elsevier, vol. 303(C).
    2. Zhang, Yubao & Chen, Xin & Gong, Sumei & Chen, Jiehao, 2023. "Collective large-scale wind farm multivariate power output control based on hierarchical communication multi-agent proximal policy optimization," Renewable Energy, Elsevier, vol. 219(P2).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Xiangjie Liu & Le Feng & Xiaobing Kong, 2022. "A Comparative Study of Robust MPC and Stochastic MPC of Wind Power Generation System," Energies, MDPI, vol. 15(13), pages 1-22, June.
    2. Azizi, Askar & Nourisola, Hamid & Shoja-Majidabad, Sajjad, 2019. "Fault tolerant control of wind turbines with an adaptive output feedback sliding mode controller," Renewable Energy, Elsevier, vol. 135(C), pages 55-65.
    3. He, Ruiyang & Yang, Hongxing & Lu, Lin, 2023. "Optimal yaw strategy and fatigue analysis of wind turbines under the combined effects of wake and yaw control," Applied Energy, Elsevier, vol. 337(C).
    4. Cai, Wei & Hu, Yang & Fang, Fang & Yao, Lujin & Liu, Jizhen, 2023. "Wind farm power production and fatigue load optimization based on dynamic partitioning and wake redirection of wind turbines," Applied Energy, Elsevier, vol. 339(C).
    5. Dou, Bingzheng & Qu, Timing & Lei, Liping & Zeng, Pan, 2020. "Optimization of wind turbine yaw angles in a wind farm using a three-dimensional yawed wake model," Energy, Elsevier, vol. 209(C).
    6. Daeil Lee & Seoryong Koo & Inseok Jang & Jonghyun Kim, 2022. "Comparison of Deep Reinforcement Learning and PID Controllers for Automatic Cold Shutdown Operation," Energies, MDPI, vol. 15(8), pages 1-25, April.
    7. Wang, Tengyuan & Cai, Chang & Wang, Xinbao & Wang, Zekun & Chen, Yewen & Song, Juanjuan & Xu, Jianzhong & Zhang, Yuning & Li, Qingan, 2023. "A new Gaussian analytical wake model validated by wind tunnel experiment and LiDAR field measurements under different turbulent flow," Energy, Elsevier, vol. 271(C).
    8. Jesús Enrique Sierra-García & Matilde Santos, 2021. "Lookup Table and Neural Network Hybrid Strategy for Wind Turbine Pitch Control," Sustainability, MDPI, vol. 13(6), pages 1-17, March.
    9. Amira Elkodama & Amr Ismaiel & A. Abdellatif & S. Shaaban & Shigeo Yoshida & Mostafa A. Rushdi, 2023. "Control Methods for Horizontal Axis Wind Turbines (HAWT): State-of-the-Art Review," Energies, MDPI, vol. 16(17), pages 1-32, September.
    10. Hongmin Meng & Tingting Yang & Ji-zhen Liu & Zhongwei Lin, 2017. "A Flexible Maximum Power Point Tracking Control Strategy Considering Both Conversion Efficiency and Power Fluctuation for Large-inertia Wind Turbines," Energies, MDPI, vol. 10(7), pages 1-19, July.
    11. Lasheen, Ahmed & Saad, Mohamed S. & Emara, Hassan M. & Elshafei, Abdel Latif, 2019. "Tube-based explicit model predictive output-feedback controller for collective pitching of wind turbines," Renewable Energy, Elsevier, vol. 131(C), pages 549-562.
    12. Mou Lin & Fernando Porté-Agel, 2023. "Power Production and Blade Fatigue of a Wind Turbine Array Subjected to Active Yaw Control," Energies, MDPI, vol. 16(6), pages 1-17, March.
    13. Francesco Castellani & Marco Buzzoni & Davide Astolfi & Gianluca D’Elia & Giorgio Dalpiaz & Ludovico Terzi, 2017. "Wind Turbine Loads Induced by Terrain and Wakes: An Experimental Study through Vibration Analysis and Computational Fluid Dynamics," Energies, MDPI, vol. 10(11), pages 1-19, November.
    14. van der Hoek, Daan & Kanev, Stoyan & Allin, Julian & Bieniek, David & Mittelmeier, Niko, 2019. "Effects of axial induction control on wind farm energy production - A field test," Renewable Energy, Elsevier, vol. 140(C), pages 994-1003.
    15. Davide Astolfi & Francesco Castellani & Matteo Becchetti & Andrea Lombardi & Ludovico Terzi, 2020. "Wind Turbine Systematic Yaw Error: Operation Data Analysis Techniques for Detecting It and Assessing Its Performance Impact," Energies, MDPI, vol. 13(9), pages 1-17, May.
    16. Mazare, Mahmood, 2024. "Adaptive optimal secure wind power generation control for variable speed wind turbine systems via reinforcement learning," Applied Energy, Elsevier, vol. 353(PA).
    17. Shibuya, Koichiro & Uchida, Takanori, 2023. "Wake asymmetry of yaw state wind turbines induced by interference with wind towers," Energy, Elsevier, vol. 280(C).
    18. Wenting Chen & Hang Liu & Yonggang Lin & Wei Li & Yong Sun & Di Zhang, 2020. "LSTM-NN Yaw Control of Wind Turbines Based on Upstream Wind Information," Energies, MDPI, vol. 13(6), pages 1-23, March.
    19. Perera, A.T.D. & Kamalaruban, Parameswaran, 2021. "Applications of reinforcement learning in energy systems," Renewable and Sustainable Energy Reviews, Elsevier, vol. 137(C).
    20. Yingming Liu & Yingwei Wang & Xiaodong Wang & Jiangsheng Zhu & Wai Hou Lio, 2019. "Active Power Dispatch for Supporting Grid Frequency Regulation in Wind Farms Considering Fatigue Load," Energies, MDPI, vol. 12(8), pages 1-23, April.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:renene:v:181:y:2022:i:c:p:445-456. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.journals.elsevier.com/renewable-energy .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.