IDEAS home Printed from https://ideas.repec.org/a/igg/jmdem0/v13y2022i1p1-19.html
   My bibliography  Save this article

Multimodal Data Integration and User Interaction for Avatar Simulation in Augmented Reality

Author

Listed:
  • Anchen Sun

    (University of Miami, USA)

  • Yudong Tao

    (University of Miami, USA)

  • Mei-Ling Shyu

    (University of Miami, USA)

  • Angela Blizzard

    (University of Miami, USA)

  • William Andrew Rothenberg

    (University of Miami, USA)

  • Dainelys Garcia

    (University of Miami, USA)

  • Jason F. Jent

    (University of Miami, USA)

Abstract

Augmented Reality (AR) allows users to interact with the virtual world in the real-world environment. This paper proposes a child avatar simulation framework using multimodal data integration and user interaction in the AR environment. This framework generates the child avatar that can interact with the user and respond to his/her behaviors and consists of three subsystems: (1) the avatar interaction system scrutinizes user behaviors based on the user data, (2) the avatar action control system generates naturalistic avatar activities (actions and voices) according to the avatar internal status, and (3) the avatar display system renders the avatar through the AR interface. In addition, a child tantrum management training application has been built based on the proposed framework. And a light machine learning model has been integrated to enable efficient and effective speech emotion recognition. A sufficiently realistic child tantrum management training based on the evaluation of clinical child psychologists is enabled, which helps users get familiar with child tantrum management.

Suggested Citation

  • Anchen Sun & Yudong Tao & Mei-Ling Shyu & Angela Blizzard & William Andrew Rothenberg & Dainelys Garcia & Jason F. Jent, 2022. "Multimodal Data Integration and User Interaction for Avatar Simulation in Augmented Reality," International Journal of Multimedia Data Engineering and Management (IJMDEM), IGI Global, vol. 13(1), pages 1-19, January.
    • Handle: RePEc:igg:jmdem0:v:13:y:2022:i:1:p:1-19
    as

    Download full text from publisher

    File URL: http://services.igi-global.com/resolvedoi/resolve.aspx?doi=10.4018/IJMDEM.304391
    Download Restriction: no
    ---><---

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:igg:jmdem0:v:13:y:2022:i:1:p:1-19. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no bibliographic references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Journal Editor (email available below). General contact details of provider: https://www.igi-global.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.