IDEAS home Printed from https://ideas.repec.org/a/gam/jftint/v14y2022i2p62-d752435.html
   My bibliography  Save this article

A Performance Comparison of Different Cloud-Based Natural Language Understanding Services for an Italian e-Learning Platform

Author

Listed:
  • Matteo Zubani

    (Department of Information Engineering, University of Brescia, Via Branze 38, 25121 Brescia, Italy)

  • Luca Sigalini

    (Mega Italia Media, Via Roncadelle 70A, 25030 Castel Mella, Italy)

  • Ivan Serina

    (Department of Information Engineering, University of Brescia, Via Branze 38, 25121 Brescia, Italy)

  • Luca Putelli

    (Department of Information Engineering, University of Brescia, Via Branze 38, 25121 Brescia, Italy)

  • Alfonso E. Gerevini

    (Department of Information Engineering, University of Brescia, Via Branze 38, 25121 Brescia, Italy)

  • Mattia Chiari

    (Department of Information Engineering, University of Brescia, Via Branze 38, 25121 Brescia, Italy)

Abstract

During the COVID-19 pandemic, the corporate online training sector has increased exponentially and online course providers had to implement innovative solutions to be more efficient and provide a satisfactory service. This paper considers a real case study in implementing a chatbot, which answers frequently asked questions from learners on an Italian e-learning platform that provides workplace safety courses to several business customers. Having to respond quickly to the increase in the courses activated, the company decided to develop a chatbot using a cloud-based service currently available on the market. These services are based on Natural Language Understanding (NLU) engines, which deal with identifying information such as entities and intentions from the sentences provided as input. To integrate a chatbot in an e-learning platform, we studied the performance of the intent recognition task of the major NLU platforms available on the market with an in-depth comparison, using an Italian dataset provided by the owner of the e-learning platform. We focused on intent recognition, carried out several experiments and evaluated performance in terms of F-score, error rate, response time, and robustness of all the services selected. The chatbot is currently in production, therefore we present a description of the system implemented and its results on the original users’ requests.

Suggested Citation

  • Matteo Zubani & Luca Sigalini & Ivan Serina & Luca Putelli & Alfonso E. Gerevini & Mattia Chiari, 2022. "A Performance Comparison of Different Cloud-Based Natural Language Understanding Services for an Italian e-Learning Platform," Future Internet, MDPI, vol. 14(2), pages 1-20, February.
  • Handle: RePEc:gam:jftint:v:14:y:2022:i:2:p:62-:d:752435
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/1999-5903/14/2/62/pdf
    Download Restriction: no

    File URL: https://www.mdpi.com/1999-5903/14/2/62/
    Download Restriction: no
    ---><---

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jftint:v:14:y:2022:i:2:p:62-:d:752435. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no bibliographic references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.