IDEAS home Printed from https://ideas.repec.org/a/spr/waterr/v34y2020i11d10.1007_s11269-020-02566-9.html
   My bibliography  Save this article

Multidimensional Parallel Dynamic Programming Algorithm Based on Spark for Large-Scale Hydropower Systems

Author

Listed:
  • Yufei Ma

    (Hohai University)

  • Ping-an Zhong

    (Hohai University)

  • Bin Xu

    (Hohai University
    Nanjing Hydraulic Research Institute)

  • Feilin Zhu

    (Hohai University)

  • Yao Xiao

    (Hohai University)

  • Qingwen Lu

    (Hohai University)

Abstract

The “curse of dimensionality” is a major problem in dynamic programming (DP) algorithms for large-scale hydropower systems. This study proposes a parallel DP algorithm based on Spark (PDPoS) to alleviate the “curse of dimensionality”. Parallel computing experiments are formulated by varying the number of reservoirs, the number of discrete water levels and the number of CPU cores to analyze the quality and efficiency of PDPoS. The methodologies were applied to a cascade reservoir system made up of eight reservoirs in the Yuanshui River Basin in China. The results are as follows. (1) The number of discrete water levels is the dominant factor in the solution quality, while the number of reservoirs is the dominant factor in the solving efficiency. (2) The runtime of PDPoS is markedly affected by the calculational scale (determined by the number of reservoirs and discrete water levels), and the relationship between the number of CPU cores and the runtime is triphasic with increasing calculational scale. (3) The larger the calculational scale is, the better the parallel performance (i.e., the parallel speedup and parallel efficiency). The proposed PDPoS method has strong generality, high parallel performance, and high practical value.

Suggested Citation

  • Yufei Ma & Ping-an Zhong & Bin Xu & Feilin Zhu & Yao Xiao & Qingwen Lu, 2020. "Multidimensional Parallel Dynamic Programming Algorithm Based on Spark for Large-Scale Hydropower Systems," Water Resources Management: An International Journal, Published for the European Water Resources Association (EWRA), Springer;European Water Resources Association (EWRA), vol. 34(11), pages 3427-3444, September.
    • Handle: RePEc:spr:waterr:v:34:y:2020:i:11:d:10.1007_s11269-020-02566-9
    DOI: 10.1007/s11269-020-02566-9
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11269-020-02566-9
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11269-020-02566-9?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Mohammad Ehteram & Hojat Karami & Saeed Farzin, 2018. "Reservoir Optimization for Energy Production Using a New Evolutionary Algorithm Based on Multi-Criteria Decision-Making Models," Water Resources Management: An International Journal, Published for the European Water Resources Association (EWRA), Springer;European Water Resources Association (EWRA), vol. 32(7), pages 2539-2560, May.
    2. Feng, Zhong-kai & Niu, Wen-jing & Cheng, Chun-tian & Liao, Sheng-li, 2017. "Hydropower system operation optimization by discrete differential dynamic programming based on orthogonal experiment design," Energy, Elsevier, vol. 126(C), pages 720-732.
    3. Sheng-li Liao & Ben-xi Liu & Chun-tian Cheng & Zhi-fu Li & Xin-yu Wu, 2017. "Long-Term Generation Scheduling of Hydropower System Using Multi-Core Parallelization of Particle Swarm Optimization," Water Resources Management: An International Journal, Published for the European Water Resources Association (EWRA), Springer;European Water Resources Association (EWRA), vol. 31(9), pages 2791-2807, July.
    4. Xu, Bin & Zhu, Feilin & Zhong, Ping-an & Chen, Juan & Liu, Weifeng & Ma, Yufei & Guo, Le & Deng, Xiaoliang, 2019. "Identifying long-term effects of using hydropower to complement wind power uncertainty through stochastic programming," Applied Energy, Elsevier, vol. 253(C), pages 1-1.
    5. Ping Sun & Zhi-qiang Jiang & Ting-ting Wang & Yan-ke Zhang, 2016. "Research and Application of Parallel Normal Cloud Mutation Shuffled Frog Leaping Algorithm in Cascade Reservoirs Optimal Operation," Water Resources Management: An International Journal, Published for the European Water Resources Association (EWRA), Springer;European Water Resources Association (EWRA), vol. 30(3), pages 1019-1035, February.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Shengli Liao & Yan Zhang & Jie Liu & Benxi Liu & Zhanwei Liu, 2021. "Short-Term Peak-Shaving Operation of Single-Reservoir and Multicascade Hydropower Plants Serving Multiple Power Grids," Water Resources Management: An International Journal, Published for the European Water Resources Association (EWRA), Springer;European Water Resources Association (EWRA), vol. 35(2), pages 689-705, January.
    2. Yufei Ma & Ping-an Zhong & Bin Xu & Feilin Zhu & Jieyu Li & Han Wang & Qingwen Lu, 2021. "Cloud-Based Multidimensional Parallel Dynamic Programming Algorithm for a Cascade Hydropower System," Water Resources Management: An International Journal, Published for the European Water Resources Association (EWRA), Springer;European Water Resources Association (EWRA), vol. 35(9), pages 2705-2721, July.
    3. Wen-jing Niu & Zhong-kai Feng & Yu-rong Li & Shuai Liu, 2021. "Cooperation Search Algorithm for Power Generation Production Operation Optimization of Cascade Hydropower Reservoirs," Water Resources Management: An International Journal, Published for the European Water Resources Association (EWRA), Springer;European Water Resources Association (EWRA), vol. 35(8), pages 2465-2485, June.
    4. Schäffer, Linn Emelie & Helseth, Arild & Korpås, Magnus, 2022. "A stochastic dynamic programming model for hydropower scheduling with state-dependent maximum discharge constraints," Renewable Energy, Elsevier, vol. 194(C), pages 571-581.
    5. Fang, Zhou & Liao, Shengli & Cheng, Chuntian & Zhao, Hongye & Liu, Benxi & Su, Huaying, 2023. "Parallel improved DPSA algorithm for medium-term optimal scheduling of large-scale cascade hydropower plants," Renewable Energy, Elsevier, vol. 210(C), pages 134-147.
    6. Zhao, Hongye & Liao, Shengli & Fang, Zhou & Liu, Benxi & Ma, Xiangyu & Lu, Jia, 2024. "Short-term peak-shaving operation of “N-reservoirs and multicascade” large-scale hydropower systems based on a decomposition-iteration strategy," Energy, Elsevier, vol. 288(C).
    7. Arya Yaghoubzadeh-Bavandpour & Omid Bozorg-Haddad & Mohammadreza Rajabi & Babak Zolghadr-Asli & Xuefeng Chu, 2022. "Application of Swarm Intelligence and Evolutionary Computation Algorithms for Optimal Reservoir Operation," Water Resources Management: An International Journal, Published for the European Water Resources Association (EWRA), Springer;European Water Resources Association (EWRA), vol. 36(7), pages 2275-2292, May.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Feng, Zhong-kai & Niu, Wen-jing & Cheng, Chun-tian, 2018. "Optimization of hydropower reservoirs operation balancing generation benefit and ecological requirement with parallel multi-objective genetic algorithm," Energy, Elsevier, vol. 153(C), pages 706-718.
    2. Jin, Xiaoyu & Liu, Benxi & Liao, Shengli & Cheng, Chuntian & Yan, Zhiyu, 2022. "A Wasserstein metric-based distributionally robust optimization approach for reliable-economic equilibrium operation of hydro-wind-solar energy systems," Renewable Energy, Elsevier, vol. 196(C), pages 204-219.
    3. Jin, Xiaoyu & Liu, Benxi & Liao, Shengli & Cheng, Chuntian & Li, Gang & Liu, Lingjun, 2022. "Impacts of different wind and solar power penetrations on cascade hydroplants operation," Renewable Energy, Elsevier, vol. 182(C), pages 227-244.
    4. Yang, Zhe & Wang, Yufeng & Yang, Kan, 2022. "The stochastic short-term hydropower generation scheduling considering uncertainty in load output forecasts," Energy, Elsevier, vol. 241(C).
    5. Yuhong Wang & Lei Chen & Hong Zhou & Xu Zhou & Zongsheng Zheng & Qi Zeng & Li Jiang & Liang Lu, 2021. "Flexible Transmission Network Expansion Planning Based on DQN Algorithm," Energies, MDPI, vol. 14(7), pages 1-21, April.
    6. Zhiqiang Jiang & Zhengyang Tang & Yi Liu & Yuyun Chen & Zhongkai Feng & Yang Xu & Hairong Zhang, 2019. "Area Moment and Error Based Forecasting Difficulty and its Application in Inflow Forecasting Level Evaluation," Water Resources Management: An International Journal, Published for the European Water Resources Association (EWRA), Springer;European Water Resources Association (EWRA), vol. 33(13), pages 4553-4568, October.
    7. Hu Hu & Kan Yang & Lyuwen Su & Zhe Yang, 2019. "A Novel Adaptive Multi-Objective Particle Swarm Optimization Based on Decomposition and Dominance for Long-term Generation Scheduling of Cascade Hydropower System," Water Resources Management: An International Journal, Published for the European Water Resources Association (EWRA), Springer;European Water Resources Association (EWRA), vol. 33(11), pages 4007-4026, September.
    8. Jiang, Zhiqiang & Li, Anqiang & Ji, Changming & Qin, Hui & Yu, Shan & Li, Yuanzheng, 2016. "Research and application of key technologies in drawing energy storage operation chart by discriminant coefficient method," Energy, Elsevier, vol. 114(C), pages 774-786.
    9. Qiao-feng Tan & Guo-hua Fang & Xin Wen & Xiao-hui Lei & Xu Wang & Chao Wang & Yi Ji, 2020. "Bayesian Stochastic Dynamic Programming for Hydropower Generation Operation Based on Copula Functions," Water Resources Management: An International Journal, Published for the European Water Resources Association (EWRA), Springer;European Water Resources Association (EWRA), vol. 34(5), pages 1589-1607, March.
    10. Thakur, Jagruti & Hesamzadeh, Mohammad Reza & Date, Paresh & Bunn, Derek, 2023. "Pricing and hedging wind power prediction risk with binary option contracts," Energy Economics, Elsevier, vol. 126(C).
    11. Zhang, Yi & Cheng, Chuntian & Yang, Tiantian & Jin, Xiaoyu & Jia, Zebin & Shen, Jianjian & Wu, Xinyu, 2022. "Assessment of climate change impacts on the hydro-wind-solar energy supply system," Renewable and Sustainable Energy Reviews, Elsevier, vol. 162(C).
    12. Ming, Bo & Liu, Pan & Guo, Shenglian & Zhang, Xiaoqi & Feng, Maoyuan & Wang, Xianxun, 2017. "Optimizing utility-scale photovoltaic power generation for integration into a hydropower reservoir by incorporating long- and short-term operational decisions," Applied Energy, Elsevier, vol. 204(C), pages 432-445.
    13. Huang, Kangdi & Liu, Pan & Kim, Jong-Suk & Xu, Weifeng & Gong, Yu & Cheng, Qian & Zhou, Yong, 2023. "A model coupling current non-adjustable, coming adjustable and remaining stages for daily generation scheduling of a wind-solar-hydro complementary system," Energy, Elsevier, vol. 263(PB).
    14. Wang, Xuekai & Tang, Tao & Su, Shuai & Yin, Jiateng & Gao, Ziyou & Lv, Nan, 2021. "An integrated energy-efficient train operation approach based on the space-time-speed network methodology," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 150(C).
    15. Gong, Yu & Liu, Pan & Ming, Bo & Li, Dingfang, 2021. "Identifying the effect of forecast uncertainties on hybrid power system operation: A case study of Longyangxia hydro–photovoltaic plant in China," Renewable Energy, Elsevier, vol. 178(C), pages 1303-1321.
    16. Shen, Jianjian & Cheng, Chuntian & Zhang, Xiufei & Zhou, Binbin, 2018. "Coordinated operations of multiple-reservoir cascaded hydropower plants with cooperation benefit allocation," Energy, Elsevier, vol. 153(C), pages 509-518.
    17. Ma, Chao & Xu, Ximeng & Pang, Xiulan & Li, Xiaofeng & Zhang, Pengfei & Liu, Lu, 2024. "Scenario-based ultra-short-term rolling optimal operation of a photovoltaic-energy storage system under forecast uncertainty," Applied Energy, Elsevier, vol. 356(C).
    18. Zhongzheng He & Chao Wang & Yongqiang Wang & Hairong Zhang & Heng Yin, 2022. "An Efficient Optimization Method for Long-term Power Generation Scheduling of Hydropower Station: Improved Dynamic Programming with a Relaxation Strategy," Water Resources Management: An International Journal, Published for the European Water Resources Association (EWRA), Springer;European Water Resources Association (EWRA), vol. 36(4), pages 1481-1497, March.
    19. Shaokun He & Shenglian Guo & Guang Yang & Kebing Chen & Dedi Liu & Yanlai Zhou, 2020. "Optimizing Operation Rules of Cascade Reservoirs for Adapting Climate Change," Water Resources Management: An International Journal, Published for the European Water Resources Association (EWRA), Springer;European Water Resources Association (EWRA), vol. 34(1), pages 101-120, January.
    20. Dong, Wenhui & Cao, Zezhou & Zhao, Pengchong & Yang, Zhenbiao & Yuan, Yichen & Zhao, Ziwen & Chen, Diyi & Wu, Yajun & Xu, Beibei & Venkateshkumar, M., 2023. "A segmented optimal PID method to consider both regulation performance and damping characteristic of hydroelectric power system," Renewable Energy, Elsevier, vol. 207(C), pages 1-12.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:waterr:v:34:y:2020:i:11:d:10.1007_s11269-020-02566-9. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.