IDEAS home Printed from https://ideas.repec.org/a/taf/tbitxx/v38y2019i4p375-383.html
   My bibliography  Save this article

A comparison of three think-aloud protocols used to evaluate a voice intelligent agent that expresses emotions

Author

Listed:
  • Xiang Ji
  • Pei-Luen Patrick Rau

Abstract

This study proposes two adapted think-aloud protocols for the evaluation of a voice intelligent agent. In the adapted retrospective think-aloud (RTA) protocol, users verbalise their thoughts based on the chat history after task-completion. In the adapted interactive think-aloud (ITA) protocol, users verbalise their thoughts regarding the intelligent agent being evaluated without the help of a facilitator. This study compares these two protocols with the classical think-aloud protocol (CTA) for evaluating an intelligent agent in terms of task time and verbal utterances. The influence of the intelligent agent’s emotional expression is also considered. The results suggest RTA is suitable for collecting user experience and causal explanation of utterances, CTA for collecting recommendation and prediction utterances, and ITA for collecting problem formulation and recommendation utterances. Furthermore, CTA and RTA can collect more total utterances, while CTA and ITA are influenced less the VIA’s emotional expression. This study provides guidelines by which future evaluators can choose suitable think-aloud protocols.

Suggested Citation

  • Xiang Ji & Pei-Luen Patrick Rau, 2019. "A comparison of three think-aloud protocols used to evaluate a voice intelligent agent that expresses emotions," Behaviour and Information Technology, Taylor & Francis Journals, vol. 38(4), pages 375-383, April.
    • Handle: RePEc:taf:tbitxx:v:38:y:2019:i:4:p:375-383
    DOI: 10.1080/0144929X.2018.1535621
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1080/0144929X.2018.1535621
    Download Restriction: Access to full text is restricted to subscribers.
    File URL: https://libkey.io/10.1080/0144929X.2018.1535621?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:taf:tbitxx:v:38:y:2019:i:4:p:375-383. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no bibliographic references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Longhurst (email available below). General contact details of provider: http://www.tandfonline.com/tbit .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.