IDEAS home Printed from https://ideas.repec.org/a/eee/renene/v181y2022icp445-456.html
   My bibliography  Save this article

FALCON- FArm Level CONtrol for wind turbines using multi-agent deep reinforcement learning

Author

Listed:
  • Padullaparthi, Venkata Ramakrishna
  • Nagarathinam, Srinarayana
  • Vasan, Arunchandar
  • Menon, Vishnu
  • Sudarsanam, Depak

Abstract

Turbines in a wind farm dynamically influence each other through wakes. Therefore trade-offs exist between energy output of upstream turbines and the health of downstream turbines. Using both model-based predictive control (MPC) and machine learning techniques, existing works have explored the energy-fatigue trade-off either in a single turbine or only with few turbines due to issues of scalability and complexity. To address this gap, this paper proposes a multi-agent deep reinforcement learning-based coordinated control for wind farms, called FALCON. FALCON addresses the multi-objective optimization problem of maximizing energy while minimizing fatigue damage by jointly controlling pitch and yaw of all turbines. FALCON achieves scale by using multiple reinforcement learning agents; capturing the global state-space efficiently using an auto-encoder; and pruning the action-space using domain knowledge. FALCON is evaluated through a real-world wind-farm case study with 21 turbines; and performs better than the default baseline PID controller and a learning-based distributed control.

Suggested Citation

  • Padullaparthi, Venkata Ramakrishna & Nagarathinam, Srinarayana & Vasan, Arunchandar & Menon, Vishnu & Sudarsanam, Depak, 2022. "FALCON- FArm Level CONtrol for wind turbines using multi-agent deep reinforcement learning," Renewable Energy, Elsevier, vol. 181(C), pages 445-456.
    • Handle: RePEc:eee:renene:v:181:y:2022:i:c:p:445-456
    DOI: 10.1016/j.renene.2021.09.023
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0960148121013227
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.renene.2021.09.023?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Jinghan Cui & Su Liu & Jinfeng Liu & Xiangjie Liu, 2018. "A Comparative Study of MPC and Economic MPC of Wind Energy Conversion Systems," Energies, MDPI, vol. 11(11), pages 1-23, November.
    2. van Dijk, Mike T. & van Wingerden, Jan-Willem & Ashuri, Turaj & Li, Yaoyu, 2017. "Wind farm multi-objective wake redirection for optimizing power production and loads," Energy, Elsevier, vol. 121(C), pages 561-569.
    3. Aitor Saenz-Aguirre & Ekaitz Zulueta & Unai Fernandez-Gamiz & Javier Lozano & Jose Manuel Lopez-Guede, 2019. "Artificial Neural Network Based Reinforcement Learning for Wind Turbine Yaw Control," Energies, MDPI, vol. 12(3), pages 1-17, January.
    4. Jain, Achin & Schildbach, Georg & Fagiano, Lorenzo & Morari, Manfred, 2015. "On the design and tuning of linear model predictive control for wind turbines," Renewable Energy, Elsevier, vol. 80(C), pages 664-673.
    5. Hosseini, Ehsan & Aghadavoodi, Ehsan & Fernández Ramírez, Luis M., 2020. "Improving response of wind turbines by pitch angle controller based on gain-scheduled recurrent ANFIS type 2 with passive reinforcement learning," Renewable Energy, Elsevier, vol. 157(C), pages 897-910.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Zhang, Yubao & Chen, Xin & Gong, Sumei & Chen, Jiehao, 2023. "Collective large-scale wind farm multivariate power output control based on hierarchical communication multi-agent proximal policy optimization," Renewable Energy, Elsevier, vol. 219(P2).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Xiangjie Liu & Le Feng & Xiaobing Kong, 2022. "A Comparative Study of Robust MPC and Stochastic MPC of Wind Power Generation System," Energies, MDPI, vol. 15(13), pages 1-22, June.
    2. He, Ruiyang & Yang, Hongxing & Lu, Lin & Gao, Xiaoxia, 2024. "Site-specific wake steering strategy for combined power enhancement and fatigue mitigation within wind farms," Renewable Energy, Elsevier, vol. 225(C).
    3. Xu, Zongyuan & Gao, Xiaoxia & Zhang, Huanqiang & Lv, Tao & Han, Zhonghe & Zhu, Xiaoxun & Wang, Yu, 2023. "Analysis of the anisotropy aerodynamic characteristics of downstream wind turbine considering the 3D wake expansion based on coupling method," Energy, Elsevier, vol. 263(PD).
    4. Azizi, Askar & Nourisola, Hamid & Shoja-Majidabad, Sajjad, 2019. "Fault tolerant control of wind turbines with an adaptive output feedback sliding mode controller," Renewable Energy, Elsevier, vol. 135(C), pages 55-65.
    5. Kanev, Stoyan, 2020. "Dynamic wake steering and its impact on wind farm power production and yaw actuator duty," Renewable Energy, Elsevier, vol. 146(C), pages 9-15.
    6. He, Ruiyang & Yang, Hongxing & Lu, Lin, 2023. "Optimal yaw strategy and fatigue analysis of wind turbines under the combined effects of wake and yaw control," Applied Energy, Elsevier, vol. 337(C).
    7. Cai, Wei & Hu, Yang & Fang, Fang & Yao, Lujin & Liu, Jizhen, 2023. "Wind farm power production and fatigue load optimization based on dynamic partitioning and wake redirection of wind turbines," Applied Energy, Elsevier, vol. 339(C).
    8. Dou, Bingzheng & Qu, Timing & Lei, Liping & Zeng, Pan, 2020. "Optimization of wind turbine yaw angles in a wind farm using a three-dimensional yawed wake model," Energy, Elsevier, vol. 209(C).
    9. Daeil Lee & Seoryong Koo & Inseok Jang & Jonghyun Kim, 2022. "Comparison of Deep Reinforcement Learning and PID Controllers for Automatic Cold Shutdown Operation," Energies, MDPI, vol. 15(8), pages 1-25, April.
    10. Wang, Tengyuan & Cai, Chang & Wang, Xinbao & Wang, Zekun & Chen, Yewen & Song, Juanjuan & Xu, Jianzhong & Zhang, Yuning & Li, Qingan, 2023. "A new Gaussian analytical wake model validated by wind tunnel experiment and LiDAR field measurements under different turbulent flow," Energy, Elsevier, vol. 271(C).
    11. Jesús Enrique Sierra-García & Matilde Santos, 2021. "Lookup Table and Neural Network Hybrid Strategy for Wind Turbine Pitch Control," Sustainability, MDPI, vol. 13(6), pages 1-17, March.
    12. Amira Elkodama & Amr Ismaiel & A. Abdellatif & S. Shaaban & Shigeo Yoshida & Mostafa A. Rushdi, 2023. "Control Methods for Horizontal Axis Wind Turbines (HAWT): State-of-the-Art Review," Energies, MDPI, vol. 16(17), pages 1-32, September.
    13. Hongmin Meng & Tingting Yang & Ji-zhen Liu & Zhongwei Lin, 2017. "A Flexible Maximum Power Point Tracking Control Strategy Considering Both Conversion Efficiency and Power Fluctuation for Large-inertia Wind Turbines," Energies, MDPI, vol. 10(7), pages 1-19, July.
    14. Lasheen, Ahmed & Saad, Mohamed S. & Emara, Hassan M. & Elshafei, Abdel Latif, 2019. "Tube-based explicit model predictive output-feedback controller for collective pitching of wind turbines," Renewable Energy, Elsevier, vol. 131(C), pages 549-562.
    15. Sohaib Abdeslam Boulanouar & Ameur Miloud Kaddouri & Abdellah Kouzou & Amar Benaissa & Ali Teta & Ahmed Hafaifa & Ralph Kennel & Mohamed Abdelrahem, 2023. "Multifunctional Control Technique for Grid-Tied Hybrid Distributed Generation System Taking into Account Power Quality Issues," Energies, MDPI, vol. 16(18), pages 1-22, September.
    16. Mou Lin & Fernando Porté-Agel, 2023. "Power Production and Blade Fatigue of a Wind Turbine Array Subjected to Active Yaw Control," Energies, MDPI, vol. 16(6), pages 1-17, March.
    17. Ander Sánchez-Chica & Ekaitz Zulueta & Daniel Teso-Fz-Betoño & Pablo Martínez-Filgueira & Unai Fernandez-Gamiz, 2019. "ANN-Based Stop Criteria for a Genetic Algorithm Applied to Air Impingement Design," Energies, MDPI, vol. 13(1), pages 1-17, December.
    18. Xiaodong Wang & Zhaoliang Ye & Shun Kang & Hui Hu, 2019. "Investigations on the Unsteady Aerodynamic Characteristics of a Horizontal-Axis Wind Turbine during Dynamic Yaw Processes," Energies, MDPI, vol. 12(16), pages 1-23, August.
    19. Francesco Castellani & Marco Buzzoni & Davide Astolfi & Gianluca D’Elia & Giorgio Dalpiaz & Ludovico Terzi, 2017. "Wind Turbine Loads Induced by Terrain and Wakes: An Experimental Study through Vibration Analysis and Computational Fluid Dynamics," Energies, MDPI, vol. 10(11), pages 1-19, November.
    20. Dimitrios Vamvakas & Panagiotis Michailidis & Christos Korkas & Elias Kosmatopoulos, 2023. "Review and Evaluation of Reinforcement Learning Frameworks on Smart Grid Applications," Energies, MDPI, vol. 16(14), pages 1-38, July.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:renene:v:181:y:2022:i:c:p:445-456. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.journals.elsevier.com/renewable-energy .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.