IDEAS home Printed from https://ideas.repec.org/a/eee/appene/v241y2019icp152-163.html
   My bibliography  Save this article

Power allocation smoothing strategy for hybrid energy storage system based on Markov decision process

Author

Listed:
  • Li, Guidan
  • Yang, Zhe
  • Li, Bin
  • Bi, Huakun

Abstract

The hybrid energy storage system (HESS) in electric vehicle (EV) requires power allocation for optimal performance. Recent researches show that the Markov decision process (MDP) provides promising characteristics for the energy management. However, the power fluctuation, which is rarely considered, can significantly affect the performance of HESS. The paper adopts the bilinear interpolation to smooth the MDP-based strategy. In this way, the power fluctuation can be mitigated, meanwhile the computation cost is not greatly increased. Given the battery and the ultracapacitor, two types of DC/DC converters are employed and the model of the HESS including the battery capacity degradation is introduced. Considering the energy loss and the energy reserve in the HESS, the reward function is built. Utilizing the cumulative reward function, the effect of ultracapacitor pack size on the performance of power allocation is analyzed in detail, the appropriate ultracapacitor size can be obtained further. Simulation results show that MDP strategy with the bilinear interpolation can not only reduce the energy loss by 5–10%, but also prolong the battery cycle life. Finally, the experimental workbench is built. The master-slave strategy is utilized to achieve the power allocation and DC/DC converter control. The proposed strategy is verified by the experimental results.

Suggested Citation

  • Li, Guidan & Yang, Zhe & Li, Bin & Bi, Huakun, 2019. "Power allocation smoothing strategy for hybrid energy storage system based on Markov decision process," Applied Energy, Elsevier, vol. 241(C), pages 152-163.
    • Handle: RePEc:eee:appene:v:241:y:2019:i:c:p:152-163
    DOI: 10.1016/j.apenergy.2019.03.001
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0306261919303939
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.apenergy.2019.03.001?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Song, Ziyou & Hofmann, Heath & Li, Jianqiu & Han, Xuebing & Ouyang, Minggao, 2015. "Optimization for a hybrid energy storage system in electric vehicles using dynamic programing approach," Applied Energy, Elsevier, vol. 139(C), pages 151-162.
    2. Song, Ziyou & Li, Jianqiu & Hou, Jun & Hofmann, Heath & Ouyang, Minggao & Du, Jiuyu, 2018. "The battery-supercapacitor hybrid energy storage system in electric vehicle applications: A case study," Energy, Elsevier, vol. 154(C), pages 433-441.
    3. Wang, Chun & Xiong, Rui & He, Hongwen & Ding, Xiaofeng & Shen, Weixiang, 2016. "Efficiency analysis of a bidirectional DC/DC converter in a hybrid energy storage system for plug-in hybrid electric vehicles," Applied Energy, Elsevier, vol. 183(C), pages 612-622.
    4. Li, Junqiu & Wang, Yihe & Chen, Jianwen & Zhang, Xiaopeng, 2017. "Study on energy management strategy and dynamic modeling for auxiliary power units in range-extended electric vehicles," Applied Energy, Elsevier, vol. 194(C), pages 363-375.
    5. Raslavičius, Laurencas & Azzopardi, Brian & Keršys, Artūras & Starevičius, Martynas & Bazaras, Žilvinas & Makaras, Rolandas, 2015. "Electric vehicles challenges and opportunities: Lithuanian review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 42(C), pages 786-800.
    6. Zou, Yuan & Liu, Teng & Liu, Dexing & Sun, Fengchun, 2016. "Reinforcement learning-based real-time energy management for a hybrid tracked vehicle," Applied Energy, Elsevier, vol. 171(C), pages 372-382.
    7. Suri, Girish & Onori, Simona, 2016. "A control-oriented cycle-life model for hybrid electric vehicle lithium-ion batteries," Energy, Elsevier, vol. 96(C), pages 644-653.
    8. Feroldi, Diego & Carignano, Mauro, 2016. "Sizing for fuel cell/supercapacitor hybrid vehicles based on stochastic driving cycles," Applied Energy, Elsevier, vol. 183(C), pages 645-658.
    9. Xiong, Rui & Cao, Jiayi & Yu, Quanqing, 2018. "Reinforcement learning-based real-time power management for hybrid energy storage system in the plug-in hybrid electric vehicle," Applied Energy, Elsevier, vol. 211(C), pages 538-548.
    10. Xu, Liangfei & Ouyang, Minggao & Li, Jianqiu & Yang, Fuyuan & Lu, Languang & Hua, Jianfeng, 2013. "Optimal sizing of plug-in fuel cell electric vehicles using models of vehicle performance and system cost," Applied Energy, Elsevier, vol. 103(C), pages 477-487.
    11. Vora, Ashish P. & Jin, Xing & Hoshing, Vaidehi & Saha, Tridib & Shaver, Gregory & Varigonda, Subbarao & Wasynczuk, Oleg & Tyner, Wallace E., 2017. "Design-space exploration of series plug-in hybrid electric vehicles for medium-duty truck applications in a total cost-of-ownership framework," Applied Energy, Elsevier, vol. 202(C), pages 662-672.
    12. Rahmani-Andebili, Mehdi, 2017. "Stochastic, adaptive, and dynamic control of energy storage systems integrated with renewable energy sources for power loss minimization," Renewable Energy, Elsevier, vol. 113(C), pages 1462-1471.
    13. Huakun Bi & Ping Wang & Zhishuang Wang, 2018. "Common Grounded H-Type Bidirectional DC-DC Converter with a Wide Voltage Conversion Ratio for a Hybrid Energy Storage System," Energies, MDPI, vol. 11(2), pages 1-22, February.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Wei Guo & Wenyi Fan & Yang Zhao & Jiakun An & Chunguang He & Xiaomei Guo & Yanan Qian & Libo Ma & Hongshan Zhao, 2023. "Grouping Control Strategy for Battery Energy Storage Power Stations Considering the Wind and Solar Power Generation Trend," Energies, MDPI, vol. 16(4), pages 1-15, February.
    2. Liu, Yue & Tian, Lixin & Sun, Huaping & Zhang, Xiling & Kong, Chuimin, 2022. "Option pricing of carbon asset and its application in digital decision-making of carbon asset," Applied Energy, Elsevier, vol. 310(C).
    3. Yue Liu & Lixin Tian & Zhuyun Xie & Zaili Zhen & Huaping Sun, 2021. "Option to survive or surrender: carbon asset management and optimization in thermal power enterprises from China," Papers 2104.04729, arXiv.org.
    4. Yan, Zhe & Zhang, Yongming & Liang, Runqi & Jin, Wenrui, 2020. "An allocative method of hybrid electrical and thermal energy storage capacity for load shifting based on seasonal difference in district energy planning," Energy, Elsevier, vol. 207(C).
    5. Sekhar Raghu Raman & Ka-Wai (Eric) Cheng & Xiang-Dang Xue & Yat-Chi Fong & Simon Cheung, 2021. "Hybrid Energy Storage System with Vehicle Body Integrated Super-Capacitor and Li-Ion Battery: Model, Design and Implementation, for Distributed Energy Storage," Energies, MDPI, vol. 14(20), pages 1-22, October.
    6. Zhang, Wei & Wang, Jixin & Xu, Zhenyu & Shen, Yuying & Gao, Guangzong, 2022. "A generalized energy management framework for hybrid construction vehicles via model-based reinforcement learning," Energy, Elsevier, vol. 260(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Wang, Chun & Yang, Ruixin & Yu, Quanqing, 2019. "Wavelet transform based energy management strategies for plug-in hybrid electric vehicles considering temperature uncertainty," Applied Energy, Elsevier, vol. 256(C).
    2. Xiao, B. & Ruan, J. & Yang, W. & Walker, P.D. & Zhang, N., 2021. "A review of pivotal energy management strategies for extended range electric vehicles," Renewable and Sustainable Energy Reviews, Elsevier, vol. 149(C).
    3. Wu, Yuankai & Tan, Huachun & Peng, Jiankun & Zhang, Hailong & He, Hongwen, 2019. "Deep reinforcement learning of energy management with continuous control strategy and traffic information for a series-parallel plug-in hybrid electric bus," Applied Energy, Elsevier, vol. 247(C), pages 454-466.
    4. Chi T. P. Nguyen & Bảo-Huy Nguyễn & Minh C. Ta & João Pedro F. Trovão, 2023. "Dual-Motor Dual-Source High Performance EV: A Comprehensive Review," Energies, MDPI, vol. 16(20), pages 1-28, October.
    5. Md. Sazal Miah & Molla Shahadat Hossain Lipu & Sheikh Tanzim Meraj & Kamrul Hasan & Shaheer Ansari & Taskin Jamal & Hasan Masrur & Rajvikram Madurai Elavarasan & Aini Hussain, 2021. "Optimized Energy Management Schemes for Electric Vehicle Applications: A Bibliometric Analysis towards Future Trends," Sustainability, MDPI, vol. 13(22), pages 1-38, November.
    6. Ye, Yiming & Wang, Hanchen & Xu, Bin & Zhang, Jiangfeng, 2023. "An imitation learning-based energy management strategy for electric vehicles considering battery aging," Energy, Elsevier, vol. 283(C).
    7. Yang, Weiwei & Ruan, Jiageng & Yang, Jue & Zhang, Nong, 2020. "Investigation of integrated uninterrupted dual input transmission and hybrid energy storage system for electric vehicles," Applied Energy, Elsevier, vol. 262(C).
    8. Zhu, Tao & Wills, Richard G.A. & Lot, Roberto & Ruan, Haijun & Jiang, Zhihao, 2021. "Adaptive energy management of a battery-supercapacitor energy storage system for electric vehicles based on flexible perception and neural network fitting," Applied Energy, Elsevier, vol. 292(C).
    9. Lian, Renzong & Peng, Jiankun & Wu, Yuankai & Tan, Huachun & Zhang, Hailong, 2020. "Rule-interposing deep reinforcement learning based energy management strategy for power-split hybrid electric vehicle," Energy, Elsevier, vol. 197(C).
    10. Daniel Egan & Qilun Zhu & Robert Prucka, 2023. "A Review of Reinforcement Learning-Based Powertrain Controllers: Effects of Agent Selection for Mixed-Continuity Control and Reward Formulation," Energies, MDPI, vol. 16(8), pages 1-31, April.
    11. Jiang, Hongliang & Xu, Liangfei & Li, Jianqiu & Hu, Zunyan & Ouyang, Minggao, 2019. "Energy management and component sizing for a fuel cell/battery/supercapacitor hybrid powertrain based on two-dimensional optimization algorithms," Energy, Elsevier, vol. 177(C), pages 386-396.
    12. Liu, Xinzhi & Qi, Nanjian & Dai, Keren & Yin, Yajiang & Zhao, Jiahao & Wang, Xiaofeng & You, Zheng, 2022. "Sponge Supercapacitor rule-based energy management strategy for wireless sensor nodes optimized by using dynamic programing algorithm," Energy, Elsevier, vol. 239(PE).
    13. da Silva, Samuel Filgueira & Eckert, Jony Javorski & Corrêa, Fernanda Cristina & Silva, Fabrício Leonardo & Silva, Ludmila C.A. & Dedini, Franco Giuseppe, 2022. "Dual HESS electric vehicle powertrain design and fuzzy control based on multi-objective optimization to increase driving range and battery life cycle," Applied Energy, Elsevier, vol. 324(C).
    14. Shi, Wenzhuo & Huangfu, Yigeng & Xu, Liangcai & Pang, Shengzhao, 2022. "Online energy management strategy considering fuel cell fault for multi-stack fuel cell hybrid vehicle based on multi-agent reinforcement learning," Applied Energy, Elsevier, vol. 328(C).
    15. Zhu, Tao & Wills, Richard G.A. & Lot, Roberto & Kong, Xiaodan & Yan, Xingda, 2021. "Optimal sizing and sensitivity analysis of a battery-supercapacitor energy storage system for electric vehicles," Energy, Elsevier, vol. 221(C).
    16. Dong, Peng & Zhao, Junwei & Liu, Xuewu & Wu, Jian & Xu, Xiangyang & Liu, Yanfang & Wang, Shuhan & Guo, Wei, 2022. "Practical application of energy management strategy for hybrid electric vehicles based on intelligent and connected technologies: Development stages, challenges, and future trends," Renewable and Sustainable Energy Reviews, Elsevier, vol. 170(C).
    17. Ruan, Jiageng & Song, Qiang & Yang, Weiwei, 2019. "The application of hybrid energy storage system with electrified continuously variable transmission in battery electric vehicle," Energy, Elsevier, vol. 183(C), pages 315-330.
    18. Zhang, Shuo & Hu, Xiaosong & Xie, Shaobo & Song, Ziyou & Hu, Lin & Hou, Cong, 2019. "Adaptively coordinated optimization of battery aging and energy management in plug-in hybrid electric buses," Applied Energy, Elsevier, vol. 256(C).
    19. Lei, Fei & Du, Bin & Liu, Xin & Xie, Xiaoping & Chai, Tian, 2016. "Optimization of an implicit constrained multi-physics system for motor wheels of electric vehicle," Energy, Elsevier, vol. 113(C), pages 980-990.
    20. Perera, A.T.D. & Kamalaruban, Parameswaran, 2021. "Applications of reinforcement learning in energy systems," Renewable and Sustainable Energy Reviews, Elsevier, vol. 137(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:appene:v:241:y:2019:i:c:p:152-163. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.