IDEAS home Printed from https://ideas.repec.org/a/gam/jeners/v17y2024i14p3578-d1439586.html
   My bibliography  Save this article

A Deep Reinforcement Learning Approach to DC-DC Power Electronic Converter Control with Practical Considerations

Author

Listed:
  • Nafiseh Mazaheri

    (Department of Electronics, Alcalá University (UAH), Plaza San Diego S/N, 28801 Madrid, Spain)

  • Daniel Santamargarita

    (Department of Electronics, Alcalá University (UAH), Plaza San Diego S/N, 28801 Madrid, Spain)

  • Emilio Bueno

    (Department of Electronics, Alcalá University (UAH), Plaza San Diego S/N, 28801 Madrid, Spain)

  • Daniel Pizarro

    (Department of Electronics, Alcalá University (UAH), Plaza San Diego S/N, 28801 Madrid, Spain)

  • Santiago Cobreces

    (Department of Electronics, Alcalá University (UAH), Plaza San Diego S/N, 28801 Madrid, Spain)

Abstract

In recent years, there has been a growing interest in using model-free deep reinforcement learning (DRL)-based controllers as an alternative approach to improve the dynamic behavior, efficiency, and other aspects of DC–DC power electronic converters, which are traditionally controlled based on small signal models. These conventional controllers often fail to self-adapt to various uncertainties and disturbances. This paper presents a design methodology using proximal policy optimization (PPO), a widely recognized and efficient DRL algorithm, to make near-optimal decisions for real buck converters operating in both continuous conduction mode (CCM) and discontinuous conduction mode (DCM) while handling resistive and inductive loads. Challenges associated with delays in real-time systems are identified. Key innovations include a chattering-reduction reward function, engineering of input features, and optimization of neural network architecture, which improve voltage regulation, ensure smoother operation, and optimize the computational cost of the neural network. The experimental and simulation results demonstrate the robustness and efficiency of the controller in real scenarios. The findings are believed to make significant contributions to the application of DRL controllers in real-time scenarios, providing guidelines and a starting point for designing controllers using the same method in this or other power electronic converter topologies.

Suggested Citation

  • Nafiseh Mazaheri & Daniel Santamargarita & Emilio Bueno & Daniel Pizarro & Santiago Cobreces, 2024. "A Deep Reinforcement Learning Approach to DC-DC Power Electronic Converter Control with Practical Considerations," Energies, MDPI, vol. 17(14), pages 1-22, July.
  • Handle: RePEc:gam:jeners:v:17:y:2024:i:14:p:3578-:d:1439586
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/1996-1073/17/14/3578/pdf
    Download Restriction: no

    File URL: https://www.mdpi.com/1996-1073/17/14/3578/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Zhang, Mengfan & Gómez, Pere Izquierdo & Xu, Qianwen & Dragicevic, Tomislav, 2023. "Review of online learning for control and diagnostics of power converters and drives: Algorithms, implementations and applications," Renewable and Sustainable Energy Reviews, Elsevier, vol. 186(C).
    2. Wang, Hanchen & Ye, Yiming & Zhang, Jiangfeng & Xu, Bin, 2023. "A comparative study of 13 deep reinforcement learning based energy management methods for a hybrid electric vehicle," Energy, Elsevier, vol. 266(C).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Angel Recalde & Ricardo Cajo & Washington Velasquez & Manuel S. Alvarez-Alvarado, 2024. "Machine Learning and Optimization in Energy Management Systems for Plug-In Hybrid Electric Vehicles: A Comprehensive Review," Energies, MDPI, vol. 17(13), pages 1-39, June.
    2. Jia, Bin & Li, Fan & Sun, Bo, 2024. "Knowledge-network-embedded deep reinforcement learning: An innovative way to high-efficiently develop an energy management strategy for the integrated energy system with renewable energy sources and m," Energy, Elsevier, vol. 301(C).
    3. Iqbal, Najam & Wang, Hu & Zheng, Zunqing & Yao, Mingfa, 2024. "Reinforcement learning-based heuristic planning for optimized energy management in power-split hybrid electric heavy duty vehicles," Energy, Elsevier, vol. 302(C).
    4. Alessia Musa & Pier Giuseppe Anselma & Giovanni Belingardi & Daniela Anna Misul, 2023. "Energy Management in Hybrid Electric Vehicles: A Q-Learning Solution for Enhanced Drivability and Energy Efficiency," Energies, MDPI, vol. 17(1), pages 1-20, December.
    5. He, Hongwen & Su, Qicong & Huang, Ruchen & Niu, Zegong, 2024. "Enabling intelligent transferable energy management of series hybrid electric tracked vehicle across motion dimensions via soft actor-critic algorithm," Energy, Elsevier, vol. 294(C).
    6. Zhou, Yujie & Huang, Yin & Mao, Xuping & Kang, Zehao & Huang, Xuejin & Xuan, Dongji, 2024. "Research on energy management strategy of fuel cell hybrid power via an improved TD3 deep reinforcement learning," Energy, Elsevier, vol. 293(C).
    7. Wilberforce, Tabbi & Anser, Afaaq & Swamy, Jangam Aishwarya & Opoku, Richard, 2023. "An investigation into hybrid energy storage system control and power distribution for hybrid electric vehicles," Energy, Elsevier, vol. 279(C).
    8. Peng, Jiankun & Shen, Yang & Wu, ChangCheng & Wang, Chunhai & Yi, Fengyan & Ma, Chunye, 2023. "Research on energy-saving driving control of hydrogen fuel bus based on deep reinforcement learning in freeway ramp weaving area," Energy, Elsevier, vol. 285(C).
    9. Liang, Zhaowen & Ruan, Jiageng & Wang, Zhenpo & Liu, Kai & Li, Bin, 2024. "Soft actor-critic-based EMS design for dual motor battery electric bus," Energy, Elsevier, vol. 288(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jeners:v:17:y:2024:i:14:p:3578-:d:1439586. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.