IDEAS home Printed from https://ideas.repec.org/a/nat/natcom/v13y2022i1d10.1038_s41467-022-33457-9.html
   My bibliography  Save this article

Ultrathin crystalline-silicon-based strain gauges with deep learning algorithms for silent speech interfaces

Author

Listed:
  • Taemin Kim

    (Yonsei University)

  • Yejee Shin

    (Yonsei University)

  • Kyowon Kang

    (Yonsei University)

  • Kiho Kim

    (Yonsei University)

  • Gwanho Kim

    (Yonsei University)

  • Yunsu Byeon

    (Yonsei University)

  • Hwayeon Kim

    (Yonsei University)

  • Yuyan Gao

    (The Pennsylvania State University)

  • Jeong Ryong Lee

    (Yonsei University)

  • Geonhui Son

    (Yonsei University)

  • Taeseong Kim

    (Yonsei University)

  • Yohan Jun

    (Yonsei University
    Massachusetts General Hospital
    Harvard Medical School)

  • Jihyun Kim

    (Yonsei University)

  • Jinyoung Lee

    (Yonsei University)

  • Seyun Um

    (Yonsei University)

  • Yoohwan Kwon

    (Yonsei University)

  • Byung Gwan Son

    (Yonsei University)

  • Myeongki Cho

    (Yonsei University)

  • Mingyu Sang

    (Yonsei University)

  • Jongwoon Shin

    (Yonsei University)

  • Kyubeen Kim

    (Yonsei University)

  • Jungmin Suh

    (Yonsei University)

  • Heekyeong Choi

    (Yonsei University)

  • Seokjun Hong

    (Yonsei University)

  • Huanyu Cheng

    (The Pennsylvania State University)

  • Hong-Goo Kang

    (Yonsei University)

  • Dosik Hwang

    (Yonsei University
    Yonsei University)

  • Ki Jun Yu

    (Yonsei University
    Yonsei University)

Abstract

A wearable silent speech interface (SSI) is a promising platform that enables verbal communication without vocalization. The most widely studied methodology for SSI focuses on surface electromyography (sEMG). However, sEMG suffers from low scalability because of signal quality-related issues, including signal-to-noise ratio and interelectrode interference. Hence, here, we present a novel SSI by utilizing crystalline-silicon-based strain sensors combined with a 3D convolutional deep learning algorithm. Two perpendicularly placed strain gauges with minimized cell dimension (

Suggested Citation

  • Taemin Kim & Yejee Shin & Kyowon Kang & Kiho Kim & Gwanho Kim & Yunsu Byeon & Hwayeon Kim & Yuyan Gao & Jeong Ryong Lee & Geonhui Son & Taeseong Kim & Yohan Jun & Jihyun Kim & Jinyoung Lee & Seyun Um , 2022. "Ultrathin crystalline-silicon-based strain gauges with deep learning algorithms for silent speech interfaces," Nature Communications, Nature, vol. 13(1), pages 1-12, December.
    • Handle: RePEc:nat:natcom:v:13:y:2022:i:1:d:10.1038_s41467-022-33457-9
    DOI: 10.1038/s41467-022-33457-9
    as

    Download full text from publisher

    File URL: https://www.nature.com/articles/s41467-022-33457-9
    File Function: Abstract
    Download Restriction: no
    File URL: https://libkey.io/10.1038/s41467-022-33457-9?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Young-Tae Kwon & Yun-Soung Kim & Shinjae Kwon & Musa Mahmood & Hyo-Ryoung Lim & Si-Woo Park & Sung-Oong Kang & Jeongmoon J. Choi & Robert Herbert & Young C. Jang & Yong-Ho Choa & Woon-Hong Yeo, 2020. "All-printed nanomembrane wireless bioelectronics using a biocompatible solderable graphene for multimodal human-machine interfaces," Nature Communications, Nature, vol. 11(1), pages 1-11, December.
    2. Yijia Lu & Han Tian & Jia Cheng & Fei Zhu & Bin Liu & Shanshan Wei & Linhong Ji & Zhong Lin Wang, 2022. "Decoding lip language using triboelectric sensors with deep learning," Nature Communications, Nature, vol. 13(1), pages 1-12, December.
    3. Jaemin Kim & Mincheol Lee & Hyung Joon Shim & Roozbeh Ghaffari & Hye Rim Cho & Donghee Son & Yei Hwan Jung & Min Soh & Changsoon Choi & Sungmook Jung & Kon Chu & Daejong Jeon & Soon-Tae Lee & Ji Hoon , 2014. "Stretchable silicon nanoribbon electronics for skin prosthesis," Nature Communications, Nature, vol. 5(1), pages 1-11, December.
    4. Wei Gao & Sam Emaminejad & Hnin Yin Yin Nyein & Samyuktha Challa & Kevin Chen & Austin Peck & Hossain M. Fahad & Hiroki Ota & Hiroshi Shiraki & Daisuke Kiriya & Der-Hsien Lien & George A. Brooks & Ron, 2016. "Fully integrated wearable sensor arrays for multiplexed in situ perspiration analysis," Nature, Nature, vol. 529(7587), pages 509-514, January.
    5. Gopala K. Anumanchipalli & Josh Chartier & Edward F. Chang, 2019. "Speech synthesis from neural decoding of spoken sentences," Nature, Nature, vol. 568(7753), pages 493-498, April.
    6. Timothée Proix & Jaime Delgado Saa & Andy Christen & Stephanie Martin & Brian N. Pasley & Robert T. Knight & Xing Tian & David Poeppel & Werner K. Doyle & Orrin Devinsky & Luc H. Arnal & Pierre Mégeva, 2022. "Imagined speech can be decoded from low- and cross-frequency intracranial EEG features," Nature Communications, Nature, vol. 13(1), pages 1-14, December.
    7. Sihong Wang & Jie Xu & Weichen Wang & Ging-Ji Nathan Wang & Reza Rastak & Francisco Molina-Lopez & Jong Won Chung & Simiao Niu & Vivian R. Feig & Jeffery Lopez & Ting Lei & Soon-Ki Kwon & Yeongin Kim , 2018. "Skin electronics from scalable fabrication of an intrinsically stretchable transistor array," Nature, Nature, vol. 555(7694), pages 83-88, March.
    8. Feng Wen & Zixuan Zhang & Tianyiyi He & Chengkuo Lee, 2021. "AI enabled sign language recognition and VR space bidirectional communication using triboelectric smart glove," Nature Communications, Nature, vol. 12(1), pages 1-13, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Kyowon Kang & Seongryeol Ye & Chanho Jeong & Jinmo Jeong & Yeong-sinn Ye & Jin-Young Jeong & Yu-Jin Kim & Selin Lim & Tae Hee Kim & Kyung Yeun Kim & Jong Uk Kim & Gwan In Kim & Do Hoon Chun & Kiho Kim, 2024. "Bionic artificial skin with a fully implantable wireless tactile sensory system for wound healing and restoring skin tactile function," Nature Communications, Nature, vol. 15(1), pages 1-15, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Zhao, Lin-Chuan & Zhou, Teng & Chang, Si-Deng & Zou, Hong-Xiang & Gao, Qiu-Hua & Wu, Zhi-Yuan & Yan, Ge & Wei, Ke-Xiang & Yeatman, Eric M. & Meng, Guang & Zhang, Wen-Ming, 2024. "A disposable cup inspired smart floor for trajectory recognition and human-interactive sensing," Applied Energy, Elsevier, vol. 357(C).
    2. Himchan Oh & Ji-Young Oh & Chan Woo Park & Jae-Eun Pi & Jong-Heon Yang & Chi-Sun Hwang, 2022. "High density integration of stretchable inorganic thin film transistors with excellent performance and reliability," Nature Communications, Nature, vol. 13(1), pages 1-9, December.
    3. Sean L. Metzger & Jessie R. Liu & David A. Moses & Maximilian E. Dougherty & Margaret P. Seaton & Kaylo T. Littlejohn & Josh Chartier & Gopala K. Anumanchipalli & Adelyn Tu-Chan & Karunesh Ganguly & E, 2022. "Generalizable spelling using a speech neuroprosthesis in an individual with severe limb and vocal paralysis," Nature Communications, Nature, vol. 13(1), pages 1-15, December.
    4. Sarah K. Wandelt & David A. Bjånes & Kelsie Pejsa & Brian Lee & Charles Liu & Richard A. Andersen, 2024. "Representation of internal speech by single neurons in human supramarginal gyrus," Nature Human Behaviour, Nature, vol. 8(6), pages 1136-1149, June.
    5. Seung-Han Kang & Jeong-Wan Jo & Jong Min Lee & Sanghee Moon & Seung Bum Shin & Su Bin Choi & Donghwan Byeon & Jaehyun Kim & Myung-Gil Kim & Yong-Hoon Kim & Jong-Woong Kim & Sung Kyu Park, 2024. "Full integration of highly stretchable inorganic transistors and circuits within molecular-tailored elastic substrates on a large scale," Nature Communications, Nature, vol. 15(1), pages 1-12, December.
    6. Yufei Zhang & Qiuchun Lu & Jiang He & Zhihao Huo & Runhui Zhou & Xun Han & Mengmeng Jia & Caofeng Pan & Zhong Lin Wang & Junyi Zhai, 2023. "Localizing strain via micro-cage structure for stretchable pressure sensor arrays with ultralow spatial crosstalk," Nature Communications, Nature, vol. 14(1), pages 1-11, December.
    7. You Wang & Ming Zhang & Ruifen Hu & Guang Li & Nan Li, 2020. "Silent Speech Recognition for BCI - A Review," Biomedical Journal of Scientific & Technical Research, Biomedical Research Network+, LLC, vol. 27(2), pages 20625-20627, April.
    8. Matthew S. Brown & Louis Somma & Melissa Mendoza & Yeonsik Noh & Gretchen J. Mahler & Ahyeon Koh, 2022. "Upcycling Compact Discs for Flexible and Stretchable Bioelectronic Applications," Nature Communications, Nature, vol. 13(1), pages 1-12, December.
    9. Xueguang Lu & Feilong Zhang & Liguo Zhu & Shan Peng & Jiazhen Yan & Qiwu Shi & Kefan Chen & Xue Chang & Hongfu Zhu & Cheng Zhang & Wanxia Huang & Qiang Cheng, 2024. "A terahertz meta-sensor array for 2D strain mapping," Nature Communications, Nature, vol. 15(1), pages 1-9, December.
    10. Yangshuang Bian & Mingliang Zhu & Chengyu Wang & Kai Liu & Wenkang Shi & Zhiheng Zhu & Mingcong Qin & Fan Zhang & Zhiyuan Zhao & Hanlin Wang & Yunqi Liu & Yunlong Guo, 2024. "A detachable interface for stable low-voltage stretchable transistor arrays and high-resolution X-ray imaging," Nature Communications, Nature, vol. 15(1), pages 1-9, December.
    11. Xiaoxiang Gao & Xiangjun Chen & Hongjie Hu & Xinyu Wang & Wentong Yue & Jing Mu & Zhiyuan Lou & Ruiqi Zhang & Keren Shi & Xue Chen & Muyang Lin & Baiyan Qi & Sai Zhou & Chengchangfeng Lu & Yue Gu & Xi, 2022. "A photoacoustic patch for three-dimensional imaging of hemoglobin and core temperature," Nature Communications, Nature, vol. 13(1), pages 1-13, December.
    12. Bekir Aksoy & Yufei Hao & Giulio Grasso & Krishna Manaswi Digumarti & Vito Cacucciolo & Herbert Shea, 2022. "Shielded soft force sensors," Nature Communications, Nature, vol. 13(1), pages 1-12, December.
    13. Yang Li & Nan Li & Wei Liu & Aleksander Prominski & Seounghun Kang & Yahao Dai & Youdi Liu & Huawei Hu & Shinya Wai & Shilei Dai & Zhe Cheng & Qi Su & Ping Cheng & Chen Wei & Lihua Jin & Jeffrey A. Hu, 2023. "Achieving tissue-level softness on stretchable electronics through a generalizable soft interlayer design," Nature Communications, Nature, vol. 14(1), pages 1-10, December.
    14. Jin Pyo Lee & Hanhyeok Jang & Yeonwoo Jang & Hyeonseo Song & Suwoo Lee & Pooi See Lee & Jiyun Kim, 2024. "Encoding of multi-modal emotional information via personalized skin-integrated wireless facial interface," Nature Communications, Nature, vol. 15(1), pages 1-13, December.
    15. Sangha Kim & Seongjin Park & Jina Choi & Wonseop Hwang & Sunho Kim & In-Suk Choi & Hyunjung Yi & Rhokyun Kwak, 2022. "An epifluidic electronic patch with spiking sweat clearance for event-driven perspiration monitoring," Nature Communications, Nature, vol. 13(1), pages 1-12, December.
    16. Jiayue Zhang & Yikui Gao & Di Liu & Jing-Shan Zhao & Jie Wang, 2023. "Discharge domains regulation and dynamic processes of direct-current triboelectric nanogenerator," Nature Communications, Nature, vol. 14(1), pages 1-10, December.
    17. Junfeng Lu & Yuanning Li & Zehao Zhao & Yan Liu & Yanming Zhu & Ying Mao & Jinsong Wu & Edward F. Chang, 2023. "Neural control of lexical tone production in human laryngeal motor cortex," Nature Communications, Nature, vol. 14(1), pages 1-12, December.
    18. Haojie Lu & Yong Zhang & Mengjia Zhu & Shuo Li & Huarun Liang & Peng Bi & Shuai Wang & Haomin Wang & Linli Gan & Xun-En Wu & Yingying Zhang, 2024. "Intelligent perceptual textiles based on ionic-conductive and strong silk fibers," Nature Communications, Nature, vol. 15(1), pages 1-9, December.
    19. Yijia Lu & Han Tian & Jia Cheng & Fei Zhu & Bin Liu & Shanshan Wei & Linhong Ji & Zhong Lin Wang, 2022. "Decoding lip language using triboelectric sensors with deep learning," Nature Communications, Nature, vol. 13(1), pages 1-12, December.
    20. Suseendrakumar Duraivel & Shervin Rahimpour & Chia-Han Chiang & Michael Trumpis & Charles Wang & Katrina Barth & Stephen C. Harward & Shivanand P. Lad & Allan H. Friedman & Derek G. Southwell & Saurab, 2023. "High-resolution neural recordings improve the accuracy of speech decoding," Nature Communications, Nature, vol. 14(1), pages 1-16, December.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nat:natcom:v:13:y:2022:i:1:d:10.1038_s41467-022-33457-9. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.nature.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.