IDEAS home Printed from https://ideas.repec.org/a/gam/jeners/v15y2022i5p1626-d755702.html
   My bibliography  Save this article

Automatic Verification Flow Shop Scheduling of Electric Energy Meters Based on an Improved Q-Learning Algorithm

Author

Listed:
  • Long Peng

    (Meteorology Center of Guangdong Power Grid Co., Ltd., Guangzhou 510600, China)

  • Jiajie Li

    (Meteorology Center of Guangdong Power Grid Co., Ltd., Guangzhou 510600, China)

  • Jingming Zhao

    (Meteorology Center of Guangdong Power Grid Co., Ltd., Guangzhou 510600, China)

  • Sanlei Dang

    (Meteorology Center of Guangdong Power Grid Co., Ltd., Guangzhou 510600, China)

  • Zhengmin Kong

    (School of Electrical Engineering and Automation, Wuhan University, Wuhan 430072, China)

  • Li Ding

    (School of Electrical Engineering and Automation, Wuhan University, Wuhan 430072, China)

Abstract

Considering the engineering problem of electric energy meter automatic verification and scheduling, this paper proposes a novel scheduling scheme based on an improved Q-learning algorithm. First, by introducing the state variables and behavior variables, the ranking problem of combinatorial optimization is transformed into a sequential decision problem. Then, a novel reward function is proposed to evaluate the pros and cons of the different strategies. In particular, this paper considers adopting the reinforcement learning algorithm to efficiently solve the problem. In addition, this paper also considers the ratio of exploration and utilization in the reinforcement learning process, and then provides reasonable exploration and utilization through an iterative updating scheme. Meanwhile, a decoupling strategy is introduced to address the restriction of over estimation. Finally, real time data from a provincial electric energy meter automatic verification center are used to verify the effectiveness of the proposed algorithm.

Suggested Citation

  • Long Peng & Jiajie Li & Jingming Zhao & Sanlei Dang & Zhengmin Kong & Li Ding, 2022. "Automatic Verification Flow Shop Scheduling of Electric Energy Meters Based on an Improved Q-Learning Algorithm," Energies, MDPI, vol. 15(5), pages 1-11, February.
    • Handle: RePEc:gam:jeners:v:15:y:2022:i:5:p:1626-:d:755702
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/1996-1073/15/5/1626/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/1996-1073/15/5/1626/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Ruiz, Ruben & Maroto, Concepcion, 2005. "A comprehensive review and evaluation of permutation flowshop heuristics," European Journal of Operational Research, Elsevier, vol. 165(2), pages 479-494, September.
    2. Rossit, Daniel Alejandro & Tohmé, Fernando & Frutos, Mariano, 2018. "The Non-Permutation Flow-Shop scheduling problem: A literature review," Omega, Elsevier, vol. 77(C), pages 143-153.
    3. Lu, Renzhi & Hong, Seung Ho & Zhang, Xiongfeng, 2018. "A Dynamic pricing demand response algorithm for smart grid: Reinforcement learning approach," Applied Energy, Elsevier, vol. 220(C), pages 220-230.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Mehravaran, Yasaman & Logendran, Rasaratnam, 2012. "Non-permutation flowshop scheduling in a supply chain with sequence-dependent setup times," International Journal of Production Economics, Elsevier, vol. 135(2), pages 953-963.
    2. Shafqat Jawad & Junyong Liu, 2020. "Electrical Vehicle Charging Services Planning and Operation with Interdependent Power Networks and Transportation Networks: A Review of the Current Scenario and Future Trends," Energies, MDPI, vol. 13(13), pages 1-24, July.
    3. Sun, Alexander Y., 2020. "Optimal carbon storage reservoir management through deep reinforcement learning," Applied Energy, Elsevier, vol. 278(C).
    4. Jin Qian & Haiyan Han, 2022. "Improved algorithms for proportionate flow shop scheduling with due-window assignment," Annals of Operations Research, Springer, vol. 309(1), pages 249-258, February.
    5. Angel Recalde & Ricardo Cajo & Washington Velasquez & Manuel S. Alvarez-Alvarado, 2024. "Machine Learning and Optimization in Energy Management Systems for Plug-In Hybrid Electric Vehicles: A Comprehensive Review," Energies, MDPI, vol. 17(13), pages 1-39, June.
    6. Brammer, Janis & Lutz, Bernhard & Neumann, Dirk, 2022. "Permutation flow shop scheduling with multiple lines and demand plans using reinforcement learning," European Journal of Operational Research, Elsevier, vol. 299(1), pages 75-86.
    7. Gerardo Minella & Rubén Ruiz & Michele Ciavotta, 2008. "A Review and Evaluation of Multiobjective Algorithms for the Flowshop Scheduling Problem," INFORMS Journal on Computing, INFORMS, vol. 20(3), pages 451-471, August.
    8. Omar Al-Ani & Sanjoy Das, 2022. "Reinforcement Learning: Theory and Applications in HEMS," Energies, MDPI, vol. 15(17), pages 1-37, September.
    9. Correa-Jullian, Camila & López Droguett, Enrique & Cardemil, José Miguel, 2020. "Operation scheduling in a solar thermal system: A reinforcement learning-based framework," Applied Energy, Elsevier, vol. 268(C).
    10. Ibrahim, Muhammad Sohail & Dong, Wei & Yang, Qiang, 2020. "Machine learning driven smart electric power systems: Current trends and new perspectives," Applied Energy, Elsevier, vol. 272(C).
    11. Hao, Zhifeng & Yeh, Wei-Chang & Tan, Shi-Yi, 2021. "One-batch preempt deterioration-effect multi-state multi-rework network reliability problem and algorithms," Reliability Engineering and System Safety, Elsevier, vol. 215(C).
    12. Zhang, Li & Gao, Yan & Zhu, Hongbo & Tao, Li, 2022. "Bi-level stochastic real-time pricing model in multi-energy generation system: A reinforcement learning approach," Energy, Elsevier, vol. 239(PA).
    13. Yuchun Li & Yinghua Han & Jinkuan Wang & Qiang Zhao, 2018. "A MBCRF Algorithm Based on Ensemble Learning for Building Demand Response Considering the Thermal Comfort," Energies, MDPI, vol. 11(12), pages 1-20, December.
    14. Wen, Lulu & Zhou, Kaile & Li, Jun & Wang, Shanyong, 2020. "Modified deep learning and reinforcement learning for an incentive-based demand response model," Energy, Elsevier, vol. 205(C).
    15. Perez-Gonzalez, Paz & Framinan, Jose M., 2024. "A review and classification on distributed permutation flowshop scheduling problems," European Journal of Operational Research, Elsevier, vol. 312(1), pages 1-21.
    16. Sabarathinam Srinivasan & Suresh Kumarasamy & Zacharias E. Andreadakis & Pedro G. Lind, 2023. "Artificial Intelligence and Mathematical Models of Power Grids Driven by Renewable Energy Sources: A Survey," Energies, MDPI, vol. 16(14), pages 1-56, July.
    17. Xu, Fangyuan & Zhu, Weidong & Wang, Yi Fei & Lai, Chun Sing & Yuan, Haoliang & Zhao, Yujia & Guo, Siming & Fu, Zhengxin, 2022. "A new deregulated demand response scheme for load over-shifting city in regulated power market," Applied Energy, Elsevier, vol. 311(C).
    18. Meng, Fanlin & Ma, Qian & Liu, Zixu & Zeng, Xiao-Jun, 2023. "Multiple dynamic pricing for demand response with adaptive clustering-based customer segmentation in smart grids," Applied Energy, Elsevier, vol. 333(C).
    19. Zhu, Ziqing & Hu, Ze & Chan, Ka Wing & Bu, Siqi & Zhou, Bin & Xia, Shiwei, 2023. "Reinforcement learning in deregulated energy market: A comprehensive review," Applied Energy, Elsevier, vol. 329(C).
    20. Ray, Manojit & Chakraborty, Basab, 2019. "Impact of evolving technology on collaborative energy access scaling," Renewable and Sustainable Energy Reviews, Elsevier, vol. 110(C), pages 13-27.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jeners:v:15:y:2022:i:5:p:1626-:d:755702. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.