IDEAS home Printed from https://ideas.repec.org/a/gam/jmathe/v11y2023i16p3514-d1217192.html
   My bibliography  Save this article

Hand-Guiding Gesture-Based Telemanipulation with the Gesture Mode Classification and State Estimation Using Wearable IMU Sensors

Author

Listed:
  • Haegyeom Choi

    (Mechanical Engineering Department, Soongsil University, Seoul 06978, Republic of Korea)

  • Haneul Jeon

    (Mechanical Engineering Department, Soongsil University, Seoul 06978, Republic of Korea)

  • Donghyeon Noh

    (Mechanical Engineering Department, Soongsil University, Seoul 06978, Republic of Korea)

  • Taeho Kim

    (Mechanical Engineering Department, Soongsil University, Seoul 06978, Republic of Korea)

  • Donghun Lee

    (Mechanical Engineering Department, Soongsil University, Seoul 06978, Republic of Korea)

Abstract

This study proposes a telemanipulation framework with two wearable IMU sensors without human skeletal kinematics. First, the states (intensity and direction) of spatial hand-guiding gestures are separately estimated through the proposed state estimator, and the states are also combined with the gesture’s mode (linear, angular, and via) obtained with the bi-directional LSTM-based mode classifier. The spatial pose of the 6-DOF manipulator’s end-effector (EEF) can be controlled by combining the spatial linear and angular motions based on integrating the gesture’s mode and state. To validate the significance of the proposed method, the teleoperation of the EEF to the designated target poses was conducted in the motion-capture space. As a result, it was confirmed that the mode could be classified with 84.5% accuracy in real time, even during the operator’s dynamic movement; the direction could be estimated with an error of less than 1 degree; and the intensity could be successfully estimated with the gesture speed estimator and finely tuned with the scaling factor. Finally, it was confirmed that a subject could place the EEF within the average range of 83 mm and 2.56 degrees in the target pose with only less than ten consecutive hand-guiding gestures and visual inspection in the first trial.

Suggested Citation

  • Haegyeom Choi & Haneul Jeon & Donghyeon Noh & Taeho Kim & Donghun Lee, 2023. "Hand-Guiding Gesture-Based Telemanipulation with the Gesture Mode Classification and State Estimation Using Wearable IMU Sensors," Mathematics, MDPI, vol. 11(16), pages 1-16, August.
    • Handle: RePEc:gam:jmathe:v:11:y:2023:i:16:p:3514-:d:1217192
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2227-7390/11/16/3514/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2227-7390/11/16/3514/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Haneul Jeon & Haegyeom Choi & Donghyeon Noh & Taeho Kim & Donghun Lee, 2022. "Wearable Inertial Sensor-Based Hand-Guiding Gestures Recognition Method Robust to Significant Changes in the Body-Alignment of Subject," Mathematics, MDPI, vol. 10(24), pages 1-13, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Donghyeon Noh & Haegyeom Choi & Haneul Jeon & Taeho Kim & Donghun Lee, 2024. "Upper Extremity Motion-Based Telemanipulation with Component-Wise Rescaling of Spatial Twist and Parameter-Invariant Skeletal Kinematics," Mathematics, MDPI, vol. 12(2), pages 1-18, January.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.

      Corrections

      All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jmathe:v:11:y:2023:i:16:p:3514-:d:1217192. See general information about how to correct material in RePEc.

      If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

      If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

      If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

      For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

      Please note that corrections may take a couple of weeks to filter through the various RePEc services.

      IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.