IDEAS home Printed from https://ideas.repec.org/a/gam/jsusta/v12y2020i12p5161-d375816.html
   My bibliography  Save this article

Reinforcement Learning in Blockchain-Enabled IIoT Networks: A Survey of Recent Advances and Open Challenges

Author

Listed:
  • Furqan Jameel

    (Department of Communications and Networking, Aalto University, 02150 Espoo, Finland
    These authors contributed equally to this work.)

  • Uzair Javaid

    (Department of Electrical and Computer Engineering, National University of Singapore, 4 Engineering Drive 3, Singapore 117583, Singapore
    These authors contributed equally to this work.)

  • Wali Ullah Khan

    (School of Information Science and Engineering, Shandong University, Qingdao 266237, China
    These authors contributed equally to this work.)

  • Muhammad Naveed Aman

    (School of Computing, National University of Singapore, 13 Computing Drive, Singapore 117417, Singapore
    These authors contributed equally to this work.)

  • Haris Pervaiz

    (School of Computing and Communications, Lancaster University, Lancaster LA1 4WA, UK
    These authors contributed equally to this work.)

  • Riku Jäntti

    (Department of Communications and Networking, Aalto University, 02150 Espoo, Finland
    These authors contributed equally to this work.)

Abstract

Blockchain is emerging as a promising candidate for the uberization of Internet services. It is a decentralized, secure, and auditable solution for exchanging, and authenticating information via transactions, without the need of a trusted third party. Therefore, blockchain technology has recently been integrated with industrial Internet-of-things (IIoT) networks to help realize the fourth industrial revolution, Industry 4.0. Though blockchain-enabled IIoT networks may have the potential to support the services and demands of next-generation networks, the gap analysis presented in this work highlights some of the areas that need improvement. Based on these observations, the article then promotes the utility of reinforcement learning (RL) techniques to address some of the major issues of blockchain-enabled IIoT networks such as block time minimization and transaction throughput enhancement. This is followed by a comprehensive case study where a Q-learning technique is used for minimizing the occurrence of forking events by reducing the transmission delays for a miner. Extensive simulations have been performed and the results have been obtained for the average transmission delay which relates to the forking events. The obtained results demonstrate that the Q-learning approach outperforms the greedy policy while having a reasonable level of complexity. To further develop the blockchain-enabled IIoT networks, some future research directions are also documented. While this article highlights the applications of RL techniques in blockchain-enabled IIoT networks, the provided insights and results could pave the way for rapid adoption of blockchain technology.

Suggested Citation

  • Furqan Jameel & Uzair Javaid & Wali Ullah Khan & Muhammad Naveed Aman & Haris Pervaiz & Riku Jäntti, 2020. "Reinforcement Learning in Blockchain-Enabled IIoT Networks: A Survey of Recent Advances and Open Challenges," Sustainability, MDPI, vol. 12(12), pages 1-23, June.
    • Handle: RePEc:gam:jsusta:v:12:y:2020:i:12:p:5161-:d:375816
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2071-1050/12/12/5161/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2071-1050/12/12/5161/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Lee Won Park & Sanghoon Lee & Hangbae Chang, 2018. "A Sustainable Home Energy Prosumer-Chain Methodology with Energy Tags over the Blockchain," Sustainability, MDPI, vol. 10(3), pages 1-18, March.
    2. Aiya Li & Xianhua Wei & Zhou He, 2020. "Robust Proof of Stake: A New Consensus Protocol for Sustainable Blockchain Systems," Sustainability, MDPI, vol. 12(7), pages 1-15, April.
    3. Jesse Yli-Huumo & Deokyoon Ko & Sujin Choi & Sooyong Park & Kari Smolander, 2016. "Where Is Current Research on Blockchain Technology?—A Systematic Review," PLOS ONE, Public Library of Science, vol. 11(10), pages 1-27, October.
    4. Jiani Wu & Nguyen Khoi Tran, 2018. "Application of Blockchain Technology in Sustainable Energy Systems: An Overview," Sustainability, MDPI, vol. 10(9), pages 1-22, August.
    5. Dianhui Mao & Zhihao Hao & Fan Wang & Haisheng Li, 2018. "Innovative Blockchain-Based Approach for Sustainable and Credible Environment in Food Trade: A Case Study in Shandong Province, China," Sustainability, MDPI, vol. 10(9), pages 1-17, September.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Kun Jin & Wei Wang & Xuedong Hua & Wei Zhou, 2020. "Reinforcement Learning for Optimizing Driving Policies on Cruising Taxis Services," Sustainability, MDPI, vol. 12(21), pages 1-19, October.
    2. Kiana Asgari & Aida Afshar Mohammadian & Mojtaba Tefagh, 2022. "DyFEn: Agent-Based Fee Setting in Payment Channel Networks," Papers 2210.08197, arXiv.org.
    3. Andrzej Magruk, 2021. "Analysis of Uncertainties and Levels of Foreknowledge in Relation to Major Features of Emerging Technologies—The Context of Foresight Research for the Fourth Industrial Revolution," Sustainability, MDPI, vol. 13(17), pages 1-16, September.
    4. Yao Du & Zehua Wang & Victor C. M. Leung, 2021. "Blockchain-Enabled Edge Intelligence for IoT: Background, Emerging Trends and Open Issues," Future Internet, MDPI, vol. 13(2), pages 1-21, February.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ernest Barceló & Katarina Dimić-Mišić & Monir Imani & Vesna Spasojević Brkić & Michael Hummel & Patrick Gane, 2023. "Regulatory Paradigm and Challenge for Blockchain Integration of Decentralized Systems: Example—Renewable Energy Grids," Sustainability, MDPI, vol. 15(3), pages 1-27, January.
    2. Yahia Baashar & Gamal Alkawsi & Ammar Ahmed Alkahtani & Wahidah Hashim & Rina Azlin Razali & Sieh Kiong Tiong, 2021. "Toward Blockchain Technology in the Energy Environment," Sustainability, MDPI, vol. 13(16), pages 1-20, August.
    3. Thomas Puschmann & Christian Hugo Hoffmann & Valentyn Khmarskyi, 2020. "How Green FinTech Can Alleviate the Impact of Climate Change—The Case of Switzerland," Sustainability, MDPI, vol. 12(24), pages 1-30, December.
    4. Md Al Amin & Dewan Hafiz Nabil & Roberto Baldacci & Md. Habibur Rahman, 2023. "Exploring Blockchain Implementation Challenges for Sustainable Supply Chains: An Integrated Fuzzy TOPSIS–ISM Approach," Sustainability, MDPI, vol. 15(18), pages 1-25, September.
    5. Esther Salmerón-Manzano & Francisco Manzano-Agugliaro, 2019. "The Role of Smart Contracts in Sustainability: Worldwide Research Trends," Sustainability, MDPI, vol. 11(11), pages 1-16, May.
    6. Kouhizadeh, Mahtab & Saberi, Sara & Sarkis, Joseph, 2021. "Blockchain technology and the sustainable supply chain: Theoretically exploring adoption barriers," International Journal of Production Economics, Elsevier, vol. 231(C).
    7. Thomas Puschmann & Valentyn Khmarskyi, 2024. "Green fintech: Developing a research agenda," Corporate Social Responsibility and Environmental Management, John Wiley & Sons, vol. 31(4), pages 2823-2837, July.
    8. Adele Parmentola & Antonella Petrillo & Ilaria Tutore & Fabio De Felice, 2022. "Is blockchain able to enhance environmental sustainability? A systematic review and research agenda from the perspective of Sustainable Development Goals (SDGs)," Business Strategy and the Environment, Wiley Blackwell, vol. 31(1), pages 194-217, January.
    9. Marco Schletz & Ana Cardoso & Gabriela Prata Dias & Søren Salomo, 2020. "How Can Blockchain Technology Accelerate Energy Efficiency Interventions? A Use Case Comparison," Energies, MDPI, vol. 13(22), pages 1-23, November.
    10. Di Silvestre, Maria Luisa & Gallo, Pierluigi & Guerrero, Josep M. & Musca, Rossano & Riva Sanseverino, Eleonora & Sciumè, Giuseppe & Vásquez, Juan C. & Zizzo, Gaetano, 2020. "Blockchain for power systems: Current trends and future applications," Renewable and Sustainable Energy Reviews, Elsevier, vol. 119(C).
    11. Ante, L. & Steinmetz, F. & Fiedler, I., 2021. "Blockchain and energy: A bibliometric analysis and review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 137(C).
    12. Zhu, Shuai & Song, Malin & Lim, Ming Kim & Wang, Jianlin & Zhao, Jiajia, 2020. "The development of energy blockchain and its implications for China's energy sector," Resources Policy, Elsevier, vol. 66(C).
    13. Moritz Böhmecke‐Schwafert & Marie Wehinger & Robin Teigland, 2022. "Blockchain for the circular economy: Theorizing blockchain's role in the transition to a circular economy through an empirical investigation," Business Strategy and the Environment, Wiley Blackwell, vol. 31(8), pages 3786-3801, December.
    14. Kawther Saeedi, 2024. "Promoting Sustainable Household Engagement in Recycling via Blockchain-Based Loyalty Program," Sustainability, MDPI, vol. 16(21), pages 1-14, October.
    15. Yi Wang & Yafei Yang & Zhaoxiang Qin & Yefei Yang & Jun Li, 2023. "A Literature Review on the Application of Digital Technology in Achieving Green Supply Chain Management," Sustainability, MDPI, vol. 15(11), pages 1-18, May.
    16. Feng Xue & Kang Chang & Wei Li & Qin Wang & Haitao Zhao & Hui Zhang & Yiyang Ni & Wenchao Xia, 2022. "Blockchain Smart Contract-Enabled Secure Energy Trading for Electric Vehicles," Energies, MDPI, vol. 15(18), pages 1-15, September.
    17. Liu, Jicheng & Sun, Jiakang & Yuan, Hanying & Su, Yihan & Feng, Shuxian & Lu, Chaoran, 2022. "Behavior analysis of photovoltaic-storage-use value chain game evolution in blockchain environment," Energy, Elsevier, vol. 260(C).
    18. A. J. Jin & C. Li & J. Su & J. Tan, 2022. "Fundamental Studies of Smart Distributed Energy Resources along with Energy Blockchain," Energies, MDPI, vol. 15(21), pages 1-12, October.
    19. Wang Guizhou & Zhang Si & Yu Tao & Ning Yu, 2021. "A Systematic Overview of Blockchain Research," Journal of Systems Science and Information, De Gruyter, vol. 9(3), pages 205-238, June.
    20. Büttgen, Marion & al.,, 2021. "Blockchain in Service Management and Service Research - Developing a Research Agenda and Managerial Implications," SMR - Journal of Service Management Research, Nomos Verlagsgesellschaft mbH & Co. KG, vol. 5(2), pages 71-102.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jsusta:v:12:y:2020:i:12:p:5161-:d:375816. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.