IDEAS home Printed from https://ideas.repec.org/a/gam/jeners/v17y2023i1p79-d1305634.html
   My bibliography  Save this article

Deep Reinforcement Learning-Based Battery Management Algorithm for Retired Electric Vehicle Batteries with a Heterogeneous State of Health in BESSs

Author

Listed:
  • Nhat Quang Doan

    (Department of Electrical, Electronic and Computer Engineering, University of Ulsan, Ulsan 44610, Republic of Korea)

  • Syed Maaz Shahid

    (Department of Electrical, Electronic and Computer Engineering, University of Ulsan, Ulsan 44610, Republic of Korea)

  • Sung-Jin Choi

    (Department of Electrical, Electronic and Computer Engineering, University of Ulsan, Ulsan 44610, Republic of Korea)

  • Sungoh Kwon

    (Department of Electrical, Electronic and Computer Engineering, University of Ulsan, Ulsan 44610, Republic of Korea)

Abstract

In this paper, we propose a battery management algorithm to optimize the lifetimes of retired lithium batteries with heterogeneous states of health in a battery energy storage system under dynamic power demand. A battery energy storage system allows for the use of retired lithium batteries for applications such as backup power in homes, data centers, etc. In a battery energy storage system, a battery pack consists of several retired batteries connected in parallel or in series to fulfill the required power demand. Owing to the retired batteries’ different capacity levels, i.e., states of health, a scheduling strategy is required to turn battery cells inside the battery pack on and off such that the secondary lifetimes of the retired batteries are extended. To establish the optimal scheduling policy, it is necessary to determine the correct states of each battery cell, including the state of charge and the state of health. To that end, the proposed algorithm first estimates the state of charge and state of health for all cells based on data measured using an extended Kalman filter. Then, a deep reinforcement learning scheduling algorithm is implemented to connect/disconnect the battery cells to/from the battery pack based on their states. Via simulation, we show that the proposed algorithm estimates the state of charge and state of health of each battery cell with low error and extends the lifetime of battery packs by 20.6%, compared to methods proposed in previous works.

Suggested Citation

  • Nhat Quang Doan & Syed Maaz Shahid & Sung-Jin Choi & Sungoh Kwon, 2023. "Deep Reinforcement Learning-Based Battery Management Algorithm for Retired Electric Vehicle Batteries with a Heterogeneous State of Health in BESSs," Energies, MDPI, vol. 17(1), pages 1-19, December.
    • Handle: RePEc:gam:jeners:v:17:y:2023:i:1:p:79-:d:1305634
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/1996-1073/17/1/79/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/1996-1073/17/1/79/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Song, Yuchen & Liu, Datong & Liao, Haitao & Peng, Yu, 2020. "A hybrid statistical data-driven method for on-line joint state estimation of lithium-ion batteries," Applied Energy, Elsevier, vol. 261(C).
    2. Ng, Kong Soon & Moo, Chin-Sien & Chen, Yi-Ping & Hsieh, Yao-Ching, 2009. "Enhanced coulomb counting method for estimating state-of-charge and state-of-health of lithium-ion batteries," Applied Energy, Elsevier, vol. 86(9), pages 1506-1511, September.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Panagiotis Eleftheriadis & Spyridon Giazitzis & Sonia Leva & Emanuele Ogliari, 2023. "Data-Driven Methods for the State of Charge Estimation of Lithium-Ion Batteries: An Overview," Forecasting, MDPI, vol. 5(3), pages 1-24, September.
    2. Sui, Xin & He, Shan & Vilsen, Søren B. & Meng, Jinhao & Teodorescu, Remus & Stroe, Daniel-Ioan, 2021. "A review of non-probabilistic machine learning-based state of health estimation techniques for Lithium-ion battery," Applied Energy, Elsevier, vol. 300(C).
    3. Chen, Junxiong & Zhang, Yu & Wu, Ji & Cheng, Weisong & Zhu, Qiao, 2023. "SOC estimation for lithium-ion battery using the LSTM-RNN with extended input and constrained output," Energy, Elsevier, vol. 262(PA).
    4. Semeraro, Concetta & Caggiano, Mariateresa & Olabi, Abdul-Ghani & Dassisti, Michele, 2022. "Battery monitoring and prognostics optimization techniques: Challenges and opportunities," Energy, Elsevier, vol. 255(C).
    5. Wang, Yujie & Tian, Jiaqiang & Sun, Zhendong & Wang, Li & Xu, Ruilong & Li, Mince & Chen, Zonghai, 2020. "A comprehensive review of battery modeling and state estimation approaches for advanced battery management systems," Renewable and Sustainable Energy Reviews, Elsevier, vol. 131(C).
    6. Wei, Jingwen & Chen, Chunlin, 2021. "A multi-timescale framework for state monitoring and lifetime prognosis of lithium-ion batteries," Energy, Elsevier, vol. 229(C).
    7. Yang, Duo & Wang, Yujie & Pan, Rui & Chen, Ruiyang & Chen, Zonghai, 2018. "State-of-health estimation for the lithium-ion battery based on support vector regression," Applied Energy, Elsevier, vol. 227(C), pages 273-283.
    8. Wenxian Duan & Chuanxue Song & Silun Peng & Feng Xiao & Yulong Shao & Shixin Song, 2020. "An Improved Gated Recurrent Unit Network Model for State-of-Charge Estimation of Lithium-Ion Battery," Energies, MDPI, vol. 13(23), pages 1-19, December.
    9. Guoqing Jin & Lan Li & Yidan Xu & Minghui Hu & Chunyun Fu & Datong Qin, 2020. "Comparison of SOC Estimation between the Integer-Order Model and Fractional-Order Model Under Different Operating Conditions," Energies, MDPI, vol. 13(7), pages 1-17, April.
    10. Hu, Lin & Hu, Xiaosong & Che, Yunhong & Feng, Fei & Lin, Xianke & Zhang, Zhiyong, 2020. "Reliable state of charge estimation of battery packs using fuzzy adaptive federated filtering," Applied Energy, Elsevier, vol. 262(C).
    11. Takyi-Aninakwa, Paul & Wang, Shunli & Zhang, Hongying & Yang, Xiao & Fernandez, Carlos, 2023. "A hybrid probabilistic correction model for the state of charge estimation of lithium-ion batteries considering dynamic currents and temperatures," Energy, Elsevier, vol. 273(C).
    12. Ozkurt, Celil & Camci, Fatih & Atamuradov, Vepa & Odorry, Christopher, 2016. "Integration of sampling based battery state of health estimation method in electric vehicles," Applied Energy, Elsevier, vol. 175(C), pages 356-367.
    13. Shahjalal, Mohammad & Roy, Probir Kumar & Shams, Tamanna & Fly, Ashley & Chowdhury, Jahedul Islam & Ahmed, Md. Rishad & Liu, Kailong, 2022. "A review on second-life of Li-ion batteries: prospects, challenges, and issues," Energy, Elsevier, vol. 241(C).
    14. S, Vignesh & Che, Hang Seng & Selvaraj, Jeyraj & Tey, Kok Soon & Lee, Jia Woon & Shareef, Hussain & Errouissi, Rachid, 2024. "State of Health (SoH) estimation methods for second life lithium-ion battery—Review and challenges," Applied Energy, Elsevier, vol. 369(C).
    15. Jae-Won Chang & Gyu-Sub Lee & Hyeon-Jin Moon & Mark B. Glick & Seung-Il Moon, 2019. "Coordinated Frequency and State-of-Charge Control with Multi-Battery Energy Storage Systems and Diesel Generators in an Isolated Microgrid," Energies, MDPI, vol. 12(9), pages 1-16, April.
    16. Guo, Feng & Hu, Guangdi & Xiang, Shun & Zhou, Pengkai & Hong, Ru & Xiong, Neng, 2019. "A multi-scale parameter adaptive method for state of charge and parameter estimation of lithium-ion batteries using dual Kalman filters," Energy, Elsevier, vol. 178(C), pages 79-88.
    17. Yong Tian & Bizhong Xia & Mingwang Wang & Wei Sun & Zhihui Xu, 2014. "Comparison Study on Two Model-Based Adaptive Algorithms for SOC Estimation of Lithium-Ion Batteries in Electric Vehicles," Energies, MDPI, vol. 7(12), pages 1-19, December.
    18. Tang, Xiaopeng & Liu, Boyang & Lv, Zhou & Gao, Furong, 2017. "Observer based battery SOC estimation: Using multi-gain-switching approach," Applied Energy, Elsevier, vol. 204(C), pages 1275-1283.
    19. Shyh-Chin Huang & Kuo-Hsin Tseng & Jin-Wei Liang & Chung-Liang Chang & Michael G. Pecht, 2017. "An Online SOC and SOH Estimation Model for Lithium-Ion Batteries," Energies, MDPI, vol. 10(4), pages 1-18, April.
    20. Bizhong Xia & Zizhou Lao & Ruifeng Zhang & Yong Tian & Guanghao Chen & Zhen Sun & Wei Wang & Wei Sun & Yongzhi Lai & Mingwang Wang & Huawen Wang, 2017. "Online Parameter Identification and State of Charge Estimation of Lithium-Ion Batteries Based on Forgetting Factor Recursive Least Squares and Nonlinear Kalman Filter," Energies, MDPI, vol. 11(1), pages 1-23, December.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jeners:v:17:y:2023:i:1:p:79-:d:1305634. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.