IDEAS home Printed from https://ideas.repec.org/a/gam/jftint/v15y2022i1p14-d1016903.html
   My bibliography  Save this article

Human–Machine Interaction through Advanced Haptic Sensors: A Piezoelectric Sensory Glove with Edge Machine Learning for Gesture and Object Recognition

Author

Listed:
  • Roberto De Fazio

    (Department of Innovation Engineering, University of Salento, 73100 Lecce, Italy
    Facultad de Ingeniería, Universidad Panamericana, Aguascalientes 20290, Mexico)

  • Vincenzo Mariano Mastronardi

    (Department of Innovation Engineering, University of Salento, 73100 Lecce, Italy
    Center for Biomolecular Nanotechnologies, Italian Technology Institute IIT, 73010 Arnesano, Italy)

  • Matteo Petruzzi

    (Department of Innovation Engineering, University of Salento, 73100 Lecce, Italy)

  • Massimo De Vittorio

    (Department of Innovation Engineering, University of Salento, 73100 Lecce, Italy
    Center for Biomolecular Nanotechnologies, Italian Technology Institute IIT, 73010 Arnesano, Italy)

  • Paolo Visconti

    (Department of Innovation Engineering, University of Salento, 73100 Lecce, Italy
    Center for Biomolecular Nanotechnologies, Italian Technology Institute IIT, 73010 Arnesano, Italy)

Abstract

Human–machine interaction (HMI) refers to systems enabling communication between machines and humans. Systems for human–machine interfaces have advanced significantly in terms of materials, device design, and production methods. Energy supply units, logic circuits, sensors, and data storage units must be flexible, stretchable, undetectable, biocompatible, and self-healing to act as human–machine interfaces. This paper discusses the technologies for providing different haptic feedback of different natures. Notably, the physiological mechanisms behind touch perception are reported, along with a classification of the main haptic interfaces. Afterward, a comprehensive overview of wearable haptic interfaces is presented, comparing them in terms of cost, the number of integrated actuators and sensors, their main haptic feedback typology, and their future application. Additionally, a review of sensing systems that use haptic feedback technologies—specifically, smart gloves—is given by going through their fundamental technological specifications and key design requirements. Furthermore, useful insights related to the design of the next-generation HMI devices are reported. Lastly, a novel smart glove based on thin and conformable AlN (aluminum nitride) piezoelectric sensors is demonstrated. Specifically, the device acquires and processes the signal from the piezo sensors to classify performed gestures through an onboard machine learning (ML) algorithm. Then, the design and testing of the electronic conditioning section of AlN-based sensors integrated into the smart glove are shown. Finally, the architecture of a wearable visual-tactile recognition system is presented, combining visual data acquired by a micro-camera mounted on the user’s glass with the haptic ones provided by the piezoelectric sensors.

Suggested Citation

  • Roberto De Fazio & Vincenzo Mariano Mastronardi & Matteo Petruzzi & Massimo De Vittorio & Paolo Visconti, 2022. "Human–Machine Interaction through Advanced Haptic Sensors: A Piezoelectric Sensory Glove with Edge Machine Learning for Gesture and Object Recognition," Future Internet, MDPI, vol. 15(1), pages 1-42, December.
  • Handle: RePEc:gam:jftint:v:15:y:2022:i:1:p:14-:d:1016903
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/1999-5903/15/1/14/pdf
    Download Restriction: no

    File URL: https://www.mdpi.com/1999-5903/15/1/14/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Christian Janiesch & Patrick Zschech & Kai Heinrich, 2021. "Machine learning and deep learning," Electronic Markets, Springer;IIM University of St. Gallen, vol. 31(3), pages 685-695, September.
    2. Roberto De Fazio & Massimo De Vittorio & Paolo Visconti, 2022. "A BLE-Connected Piezoresistive and Inertial Chest Band for Remote Monitoring of the Respiratory Activity by an Android Application: Hardware Design and Software Optimization," Future Internet, MDPI, vol. 14(6), pages 1-27, June.
    3. Chad E. Bouton & Ammar Shaikhouni & Nicholas V. Annetta & Marcia A. Bockbrader & David A. Friedenberg & Dylan M. Nielson & Gaurav Sharma & Per B. Sederberg & Bradley C. Glenn & W. Jerry Mysiw & Austin, 2016. "Restoring cortical control of functional movement in a human with quadriplegia," Nature, Nature, vol. 533(7602), pages 247-250, May.
    4. Roberto de Fazio & Donato Cafagna & Giorgio Marcuccio & Alessandro Minerba & Paolo Visconti, 2020. "A Multi-Source Harvesting System Applied to Sensor-Based Smart Garments for Monitoring Workers’ Bio-Physical Parameters in Harsh Environments," Energies, MDPI, vol. 13(9), pages 1-33, May.
    5. Vito Cacucciolo & Jun Shintake & Yu Kuwajima & Shingo Maeda & Dario Floreano & Herbert Shea, 2019. "Stretchable pumps for soft machines," Nature, Nature, vol. 572(7770), pages 516-519, August.
    6. Bernardo Calabrese & Ramiro Velázquez & Carolina Del-Valle-Soto & Roberto de Fazio & Nicola Ivan Giannoccaro & Paolo Visconti, 2020. "Solar-Powered Deep Learning-Based Recognition System of Daily Used Objects and Human Faces for Assistance of the Visually Impaired," Energies, MDPI, vol. 13(22), pages 1-30, November.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jen-Yu Lee & Tien-Thinh Nguyen & Hong-Giang Nguyen & Jen-Yao Lee, 2022. "Towards Predictive Crude Oil Purchase: A Case Study in the USA and Europe," Energies, MDPI, vol. 15(11), pages 1-15, May.
    2. Eduard Hartwich & Alexander Rieger & Johannes Sedlmeir & Dominik Jurek & Gilbert Fridgen, 2023. "Machine economies," Electronic Markets, Springer;IIM University of St. Gallen, vol. 33(1), pages 1-13, December.
    3. Rainer Alt, 2021. "Electronic Markets on robotics," Electronic Markets, Springer;IIM University of St. Gallen, vol. 31(3), pages 465-471, September.
    4. Sijing Zhu & Zheng Fan & Baoquan Feng & Runze Shi & Zexin Jiang & Ying Peng & Jie Gao & Lei Miao & Kunihito Koumoto, 2022. "Review on Wearable Thermoelectric Generators: From Devices to Applications," Energies, MDPI, vol. 15(9), pages 1-27, May.
    5. Najla Alharbi & Bashayer Alkalifah & Ghaida Alqarawi & Murad A. Rassam, 2024. "Countering Social Media Cybercrime Using Deep Learning: Instagram Fake Accounts Detection," Future Internet, MDPI, vol. 16(10), pages 1-22, October.
    6. Abdulwahhab, Ali H. & Abdulaal, Alaa Hussein & Thary Al-Ghrairi, Assad H. & Mohammed, Ali Abdulwahhab & Valizadeh, Morteza, 2024. "Detection of epileptic seizure using EEG signals analysis based on deep learning techniques," Chaos, Solitons & Fractals, Elsevier, vol. 181(C).
    7. Abhirup Khanna & Bhawna Yadav Lamba & Sapna Jain & Vadim Bolshev & Dmitry Budnikov & Vladimir Panchenko & Alexandr Smirnov, 2023. "Biodiesel Production from Jatropha: A Computational Approach by Means of Artificial Intelligence and Genetic Algorithm," Sustainability, MDPI, vol. 15(12), pages 1-33, June.
    8. Rui Ma & Jia Wang & Wei Zhao & Hongjie Guo & Dongnan Dai & Yuliang Yun & Li Li & Fengqi Hao & Jinqiang Bai & Dexin Ma, 2022. "Identification of Maize Seed Varieties Using MobileNetV2 with Improved Attention Mechanism CBAM," Agriculture, MDPI, vol. 13(1), pages 1-16, December.
    9. Dylan Norbert Gono & Herlina Napitupulu & Firdaniza, 2023. "Silver Price Forecasting Using Extreme Gradient Boosting (XGBoost) Method," Mathematics, MDPI, vol. 11(18), pages 1-15, September.
    10. Cheng Yang & Fuhao Sun & Yujie Zou & Zhipeng Lv & Liang Xue & Chao Jiang & Shuangyu Liu & Bochao Zhao & Haoyang Cui, 2024. "A Survey of Photovoltaic Panel Overlay and Fault Detection Methods," Energies, MDPI, vol. 17(4), pages 1-37, February.
    11. Hong, Jichao & Li, Kerui & Liang, Fengwei & Yang, Haixu & Zhang, Chi & Yang, Qianqian & Wang, Jiegang, 2024. "A novel state of health prediction method for battery system in real-world vehicles based on gated recurrent unit neural networks," Energy, Elsevier, vol. 289(C).
    12. Shuai Sang & Lu Li, 2024. "A Novel Variant of LSTM Stock Prediction Method Incorporating Attention Mechanism," Mathematics, MDPI, vol. 12(7), pages 1-20, March.
    13. Vladimir Franki & Darin Majnarić & Alfredo Višković, 2023. "A Comprehensive Review of Artificial Intelligence (AI) Companies in the Power Sector," Energies, MDPI, vol. 16(3), pages 1-35, January.
    14. Chao Zhang & Zhuang Zhang & Yun Peng & Yanlin Zhang & Siqi An & Yunjie Wang & Zirui Zhai & Yan Xu & Hanqing Jiang, 2023. "Plug & play origami modules with all-purpose deformation modes," Nature Communications, Nature, vol. 14(1), pages 1-10, December.
    15. Thomas Grisold & Christian Janiesch & Maximilian Röglinger & Moe Thandar Wynn, 2022. "Call for Papers, Issue 5/2024," Business & Information Systems Engineering: The International Journal of WIRTSCHAFTSINFORMATIK, Springer;Gesellschaft für Informatik e.V. (GI), vol. 64(6), pages 841-843, December.
    16. Joshua Holstein & Max Schemmer & Johannes Jakubik & Michael Vössing & Gerhard Satzger, 2023. "Sanitizing data for analysis: Designing systems for data understanding," Electronic Markets, Springer;IIM University of St. Gallen, vol. 33(1), pages 1-18, December.
    17. Junwei Zhou & Yanguo Fan & Qingchun Guan & Guangyue Feng, 2024. "Research on Drought Monitoring Based on Deep Learning: A Case Study of the Huang-Huai-Hai Region in China," Land, MDPI, vol. 13(5), pages 1-20, May.
    18. Zhouheng Wang & Nanlin Shi & Yingchao Zhang & Ning Zheng & Haicheng Li & Yang Jiao & Jiahui Cheng & Yutong Wang & Xiaoqing Zhang & Ying Chen & Yihao Chen & Heling Wang & Tao Xie & Yijun Wang & Yinji M, 2023. "Conformal in-ear bioelectronics for visual and auditory brain-computer interfaces," Nature Communications, Nature, vol. 14(1), pages 1-11, December.
    19. Jonas Wanner & Lukas-Valentin Herm & Kai Heinrich & Christian Janiesch, 2022. "The effect of transparency and trust on intelligent system acceptance: Evidence from a user-based study," Electronic Markets, Springer;IIM University of St. Gallen, vol. 32(4), pages 2079-2102, December.
    20. Yuanyuan Yang & Md Muhie Menul Haque & Dongling Bai & Wei Tang, 2021. "Fault Diagnosis of Electric Motors Using Deep Learning Algorithms and Its Application: A Review," Energies, MDPI, vol. 14(21), pages 1-26, October.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jftint:v:15:y:2022:i:1:p:14-:d:1016903. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.