IDEAS home Printed from https://ideas.repec.org/a/gam/jeners/v12y2019i3p436-d201953.html
   My bibliography  Save this article

Artificial Neural Network Based Reinforcement Learning for Wind Turbine Yaw Control

Author

Listed:
  • Aitor Saenz-Aguirre

    (Automatic Control and System Engineering Department, University of the Basque Country (UPV/EHU), Nieves Cano, 12, 01006 Vitoria-Gasteiz, Spain)

  • Ekaitz Zulueta

    (Automatic Control and System Engineering Department, University of the Basque Country (UPV/EHU), Nieves Cano, 12, 01006 Vitoria-Gasteiz, Spain)

  • Unai Fernandez-Gamiz

    (Nuclear Engineering and Fluid Mechanics Department, University of the Basque Country (UPV/EHU), Nieves Cano, 12, 01006 Vitoria-Gasteiz, Spain)

  • Javier Lozano

    (Faculty of Engineering, Mondragon University, Goiru, 2, 20500 Arrasate-Mondragon, Spain)

  • Jose Manuel Lopez-Guede

    (Automatic Control and System Engineering Department, University of the Basque Country (UPV/EHU), Nieves Cano, 12, 01006 Vitoria-Gasteiz, Spain)

Abstract

This paper introduces a novel data driven yaw control algorithm synthesis method based on Reinforcement Learning (RL) for a variable pitch variable speed wind turbine. Yaw control has not been extendedly studied in the literature; in fact, most of the currently considered developments in the scope of the wind energy are oriented to the pitch and speed control. The most important drawbacks of the yaw control are the very large time constants and the strict yaw angle change rate constraints due to the high mechanical loads when the wind turbine angle is changed in order to adequate it to the wind speed orientation. An optimal yaw control algorithm needs to be designed in order to adapt the rotor orientation depending on the wind turbine dynamics and the local wind speed regime. Consequently, the biggest challenge of the yaw control algorithm is to decide the moment and the quantity of the wind turbine orientation variation to achieve the highest quantity of power at each instant, taking into account the constraints derived from the mechanical limitations of the yawing system and the mechanical loads. In this paper, a novel based algorithm based on the RL Q-Learning algorithm is introduced. The first step is to obtain a model of the power generated by the wind turbine (a real onshore wind turbine in this paper) through a power curve, that in conjunction with a conventional proportional regulator will be used to obtain a dataset that explains the actual behaviour of the real wind turbine when a variety of different yaw control commands are imposed. That knowledge is then used to learn the best control action for each different state of the wind turbine with respect to the wind direction represented by the yaw angle, storing that knowledge in a matrix Q ( s , a ). The last step is to model that matrix through a MultiLayer Perceptron with BackPropagation (MLP-BP) Artificial Neural Network (ANN) to avoid large matrix management and quantification problems. Once that the optimal yaw controller has been synthetized, its performance has been assessed using a number of wind speed realizations obtained using the software application TurbSim, in order to analyze how the introduced novel algorithm deals with different wind speed scenarios.

Suggested Citation

  • Aitor Saenz-Aguirre & Ekaitz Zulueta & Unai Fernandez-Gamiz & Javier Lozano & Jose Manuel Lopez-Guede, 2019. "Artificial Neural Network Based Reinforcement Learning for Wind Turbine Yaw Control," Energies, MDPI, vol. 12(3), pages 1-17, January.
  • Handle: RePEc:gam:jeners:v:12:y:2019:i:3:p:436-:d:201953
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/1996-1073/12/3/436/pdf
    Download Restriction: no

    File URL: https://www.mdpi.com/1996-1073/12/3/436/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Song, Dongran & Fan, Xinyu & Yang, Jian & Liu, Anfeng & Chen, Sifan & Joo, Young Hoon, 2018. "Power extraction efficiency optimization of horizontal-axis wind turbines through optimizing control parameters of yaw control systems using an intelligent method," Applied Energy, Elsevier, vol. 224(C), pages 267-279.
    2. Asier González-González & Ismael Etxeberria-Agiriano & Ekaitz Zulueta & Fernando Oterino-Echavarri & Jose Manuel Lopez-Guede, 2014. "Pitch Based Wind Turbine Intelligent Speed Setpoint Adjustment Algorithms," Energies, MDPI, vol. 7(6), pages 1-17, June.
    3. Davide Astolfi & Francesco Castellani & Ludovico Terzi, 2018. "Wind Turbine Power Curve Upgrades," Energies, MDPI, vol. 11(5), pages 1-17, May.
    4. Unai Fernandez-Gamiz & Ekaitz Zulueta & Ana Boyano & Igor Ansoategui & Irantzu Uriarte, 2017. "Five Megawatt Wind Turbine Power Output Improvements by Passive Flow Control Devices," Energies, MDPI, vol. 10(6), pages 1-15, May.
    5. Wim Munters & Johan Meyers, 2018. "Dynamic Strategies for Yaw and Induction Control of Wind Farms Based on Large-Eddy Simulation and Optimization," Energies, MDPI, vol. 11(1), pages 1-32, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Davide Astolfi & Francesco Castellani & Matteo Becchetti & Andrea Lombardi & Ludovico Terzi, 2020. "Wind Turbine Systematic Yaw Error: Operation Data Analysis Techniques for Detecting It and Assessing Its Performance Impact," Energies, MDPI, vol. 13(9), pages 1-17, May.
    2. Daeil Lee & Seoryong Koo & Inseok Jang & Jonghyun Kim, 2022. "Comparison of Deep Reinforcement Learning and PID Controllers for Automatic Cold Shutdown Operation," Energies, MDPI, vol. 15(8), pages 1-25, April.
    3. Ander Sánchez-Chica & Ekaitz Zulueta & Daniel Teso-Fz-Betoño & Pablo Martínez-Filgueira & Unai Fernandez-Gamiz, 2019. "ANN-Based Stop Criteria for a Genetic Algorithm Applied to Air Impingement Design," Energies, MDPI, vol. 13(1), pages 1-17, December.
    4. Xiangjie Liu & Le Feng & Xiaobing Kong, 2022. "A Comparative Study of Robust MPC and Stochastic MPC of Wind Power Generation System," Energies, MDPI, vol. 15(13), pages 1-22, June.
    5. Arne Gloe & Clemens Jauch & Bogdan Craciun & Arvid Zanter & Jörg Winkelmann, 2021. "Influence of Continuous Provision of Synthetic Inertia on the Mechanical Loads of a Wind Turbine," Energies, MDPI, vol. 14(16), pages 1-23, August.
    6. Han Peng & Songyin Li & Linjian Shangguan & Yisa Fan & Hai Zhang, 2023. "Analysis of Wind Turbine Equipment Failure and Intelligent Operation and Maintenance Research," Sustainability, MDPI, vol. 15(10), pages 1-35, May.
    7. Moreira, Túlio Marcondes & de Faria, Jackson Geraldo & Vaz-de-Melo, Pedro O.S. & Medeiros-Ribeiro, Gilberto, 2023. "Development and validation of an AI-Driven model for the La Rance tidal barrage: A generalisable case study," Applied Energy, Elsevier, vol. 332(C).
    8. Padullaparthi, Venkata Ramakrishna & Nagarathinam, Srinarayana & Vasan, Arunchandar & Menon, Vishnu & Sudarsanam, Depak, 2022. "FALCON- FArm Level CONtrol for wind turbines using multi-agent deep reinforcement learning," Renewable Energy, Elsevier, vol. 181(C), pages 445-456.
    9. Wenting Chen & Hang Liu & Yonggang Lin & Wei Li & Yong Sun & Di Zhang, 2020. "LSTM-NN Yaw Control of Wind Turbines Based on Upstream Wind Information," Energies, MDPI, vol. 13(6), pages 1-23, March.
    10. Amira Elkodama & Amr Ismaiel & A. Abdellatif & S. Shaaban & Shigeo Yoshida & Mostafa A. Rushdi, 2023. "Control Methods for Horizontal Axis Wind Turbines (HAWT): State-of-the-Art Review," Energies, MDPI, vol. 16(17), pages 1-32, September.
    11. Dai, Juchuan & He, Tao & Li, Mimi & Long, Xin, 2021. "Performance study of multi-source driving yaw system for aiding yaw control of wind turbines," Renewable Energy, Elsevier, vol. 163(C), pages 154-171.
    12. Xiaodong Wang & Zhaoliang Ye & Shun Kang & Hui Hu, 2019. "Investigations on the Unsteady Aerodynamic Characteristics of a Horizontal-Axis Wind Turbine during Dynamic Yaw Processes," Energies, MDPI, vol. 12(16), pages 1-23, August.
    13. Davide Astolfi & Francesco Castellani, 2019. "Wind Turbine Power Curve Upgrades: Part II," Energies, MDPI, vol. 12(8), pages 1-20, April.
    14. Manu Centeno-Telleria & Ekaitz Zulueta & Unai Fernandez-Gamiz & Daniel Teso-Fz-Betoño & Adrián Teso-Fz-Betoño, 2021. "Differential Evolution Optimal Parameters Tuning with Artificial Neural Network," Mathematics, MDPI, vol. 9(4), pages 1-20, February.
    15. Perera, A.T.D. & Kamalaruban, Parameswaran, 2021. "Applications of reinforcement learning in energy systems," Renewable and Sustainable Energy Reviews, Elsevier, vol. 137(C).
    16. Subbulakshmi, A. & Verma, Mohit & Keerthana, M. & Sasmal, Saptarshi & Harikrishna, P. & Kapuria, Santosh, 2022. "Recent advances in experimental and numerical methods for dynamic analysis of floating offshore wind turbines — An integrated review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 164(C).
    17. Dimitrios Vamvakas & Panagiotis Michailidis & Christos Korkas & Elias Kosmatopoulos, 2023. "Review and Evaluation of Reinforcement Learning Frameworks on Smart Grid Applications," Energies, MDPI, vol. 16(14), pages 1-38, July.
    18. Jesús Enrique Sierra-García & Matilde Santos, 2021. "Lookup Table and Neural Network Hybrid Strategy for Wind Turbine Pitch Control," Sustainability, MDPI, vol. 13(6), pages 1-17, March.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Han Peng & Songyin Li & Linjian Shangguan & Yisa Fan & Hai Zhang, 2023. "Analysis of Wind Turbine Equipment Failure and Intelligent Operation and Maintenance Research," Sustainability, MDPI, vol. 15(10), pages 1-35, May.
    2. Unai Elosegui & Igor Egana & Alain Ulazia & Gabriel Ibarra-Berastegi, 2018. "Pitch Angle Misalignment Correction Based on Benchmarking and Laser Scanner Measurement in Wind Farms," Energies, MDPI, vol. 11(12), pages 1-20, December.
    3. Amira Elkodama & Amr Ismaiel & A. Abdellatif & S. Shaaban & Shigeo Yoshida & Mostafa A. Rushdi, 2023. "Control Methods for Horizontal Axis Wind Turbines (HAWT): State-of-the-Art Review," Energies, MDPI, vol. 16(17), pages 1-32, September.
    4. Xiaodong Wang & Zhaoliang Ye & Shun Kang & Hui Hu, 2019. "Investigations on the Unsteady Aerodynamic Characteristics of a Horizontal-Axis Wind Turbine during Dynamic Yaw Processes," Energies, MDPI, vol. 12(16), pages 1-23, August.
    5. Unai Fernandez-Gamiz & Macarena Gomez-Mármol & Tomas Chacón-Rebollo, 2018. "Computational Modeling of Gurney Flaps and Microtabs by POD Method," Energies, MDPI, vol. 11(8), pages 1-19, August.
    6. Davide Astolfi & Francesco Castellani, 2019. "Wind Turbine Power Curve Upgrades: Part II," Energies, MDPI, vol. 12(8), pages 1-20, April.
    7. Iñigo Aramendia & Unai Fernandez-Gamiz & Ekaitz Zulueta & Aitor Saenz-Aguirre & Daniel Teso-Fz-Betoño, 2019. "Parametric Study of a Gurney Flap Implementation in a DU91W(2)250 Airfoil," Energies, MDPI, vol. 12(2), pages 1-14, January.
    8. Davide Astolfi & Francesco Castellani & Matteo Becchetti & Andrea Lombardi & Ludovico Terzi, 2020. "Wind Turbine Systematic Yaw Error: Operation Data Analysis Techniques for Detecting It and Assessing Its Performance Impact," Energies, MDPI, vol. 13(9), pages 1-17, May.
    9. Aitor Saenz-Aguirre & Unai Fernandez-Gamiz & Ekaitz Zulueta & Alain Ulazia & Jon Martinez-Rico, 2019. "Optimal Wind Turbine Operation by Artificial Neural Network-Based Active Gurney Flap Flow Control," Sustainability, MDPI, vol. 11(10), pages 1-17, May.
    10. Tanvir Ahmad & Abdul Basit & Muneeb Ahsan & Olivier Coupiac & Nicolas Girard & Behzad Kazemtabrizi & Peter C. Matthews, 2019. "Implementation and Analyses of Yaw Based Coordinated Control of Wind Farms," Energies, MDPI, vol. 12(7), pages 1-15, April.
    11. Kumarasamy Palanimuthu & Ganesh Mayilsamy & Ameerkhan Abdul Basheer & Seong-Ryong Lee & Dongran Song & Young Hoon Joo, 2022. "A Review of Recent Aerodynamic Power Extraction Challenges in Coordinated Pitch, Yaw, and Torque Control of Large-Scale Wind Turbine Systems," Energies, MDPI, vol. 15(21), pages 1-27, November.
    12. Haolan Liang & Zhangjie Liu & Hua Liu, 2019. "Stabilization Method Considering Disturbance Mitigation for DC Microgrids with Constant Power Loads," Energies, MDPI, vol. 12(5), pages 1-19, March.
    13. Cheng, Yi & Azizipanah-Abarghooee, Rasoul & Azizi, Sadegh & Ding, Lei & Terzija, Vladimir, 2020. "Smart frequency control in low inertia energy systems based on frequency response techniques: A review," Applied Energy, Elsevier, vol. 279(C).
    14. Zhang, Chaoyu & Zhang, Chengming & Li, Liyi & Guo, Qingbo, 2021. "Parameter analysis of power system for solar-powered unmanned aerial vehicle," Applied Energy, Elsevier, vol. 295(C).
    15. Cao, Yankai & Zavala, Victor M. & D’Amato, Fernando, 2018. "Using stochastic programming and statistical extrapolation to mitigate long-term extreme loads in wind turbines," Applied Energy, Elsevier, vol. 230(C), pages 1230-1241.
    16. Okulov, V.L. & Naumov, I.V. & Kabardin, I.K. & Litvinov, I.V. & Markovich, D.M. & Mikkelsen, R.F. & Sørensen, J.N. & Alekseenko, S.V. & Wood, D.H., 2021. "Experiments on line arrays of horizontal-axis hydroturbines," Renewable Energy, Elsevier, vol. 163(C), pages 15-21.
    17. Yao Liu & Lin Guan & Fang Guo & Jianping Zheng & Jianfu Chen & Chao Liu & Josep M. Guerrero, 2019. "A Reactive Power-Voltage Control Strategy of an AC Microgrid Based on Adaptive Virtual Impedance," Energies, MDPI, vol. 12(16), pages 1-15, August.
    18. Frederik, Joeri A. & van Wingerden, Jan-Willem, 2022. "On the load impact of dynamic wind farm wake mixing strategies," Renewable Energy, Elsevier, vol. 194(C), pages 582-595.
    19. Igor Ansoategui & Ekaitz Zulueta & Unai Fernandez-Gamiz & Jose Manuel Lopez-Guede, 2019. "Mechatronic Modeling and Frequency Analysis of the Drive Train of a Horizontal Wind Turbine," Energies, MDPI, vol. 12(4), pages 1-14, February.
    20. Chen, Kaixuan & Lin, Jin & Qiu, Yiwei & Liu, Feng & Song, Yonghua, 2022. "Joint optimization of wind farm layout considering optimal control," Renewable Energy, Elsevier, vol. 182(C), pages 787-796.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jeners:v:12:y:2019:i:3:p:436-:d:201953. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.