IDEAS home Printed from https://ideas.repec.org/a/gam/jsusta/v13y2021i16p9364-d618500.html
   My bibliography  Save this article

Evaluation of an Algorithm for Automatic Grading of Forum Messages in MOOC Discussion Forums

Author

Listed:
  • Raquel L. Pérez-Nicolás

    (Department of Telematics Engineering, Universidad Carlos III de Madrid, E-28911 Leganés, Spain)

  • Carlos Alario-Hoyos

    (Department of Telematics Engineering, Universidad Carlos III de Madrid, E-28911 Leganés, Spain)

  • Iria Estévez-Ayres

    (Department of Telematics Engineering, Universidad Carlos III de Madrid, E-28911 Leganés, Spain)

  • Pedro Manuel Moreno-Marcos

    (Department of Telematics Engineering, Universidad Carlos III de Madrid, E-28911 Leganés, Spain)

  • Pedro J. Muñoz-Merino

    (Department of Telematics Engineering, Universidad Carlos III de Madrid, E-28911 Leganés, Spain)

  • Carlos Delgado Kloos

    (Department of Telematics Engineering, Universidad Carlos III de Madrid, E-28911 Leganés, Spain)

Abstract

Discussion forums are a valuable source of information in educational platforms such as Massive Open Online Courses (MOOCs), as users can exchange opinions or even help other students in an asynchronous way, contributing to the sustainability of MOOCs even with low interaction from the instructor. Therefore, the use of the forum messages to get insights about students’ performance in a course is interesting. This article presents an automatic grading approach that can be used to assess learners through their interactions in the forum. The approach is based on the combination of three dimensions: (1) the quality of the content of the interactions, (2) the impact of the interactions, and (3) the user’s activity in the forum. The evaluation of the approach compares the assessment by experts with the automatic assessment obtaining a high accuracy of 0.8068 and Normalized Root Mean Square Error (NRMSE) of 0.1799, which outperforms previous existing approaches. Future research work can try to improve the automatic grading by the training of the indicators of the approach depending on the MOOCs or the combination with text mining techniques.

Suggested Citation

  • Raquel L. Pérez-Nicolás & Carlos Alario-Hoyos & Iria Estévez-Ayres & Pedro Manuel Moreno-Marcos & Pedro J. Muñoz-Merino & Carlos Delgado Kloos, 2021. "Evaluation of an Algorithm for Automatic Grading of Forum Messages in MOOC Discussion Forums," Sustainability, MDPI, vol. 13(16), pages 1-18, August.
    • Handle: RePEc:gam:jsusta:v:13:y:2021:i:16:p:9364-:d:618500
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2071-1050/13/16/9364/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2071-1050/13/16/9364/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Pradeep Kumar Roy & Zishan Ahmad & Jyoti Prakash Singh & Mohammad Abdallah Ali Alryalat & Nripendra P. Rana & Yogesh K. Dwivedi, 2018. "Finding and Ranking High-Quality Answers in Community Question Answering Sites," Global Journal of Flexible Systems Management, Springer;Global Institute of Flexible Systems Management, vol. 19(1), pages 53-68, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Lei Li & Xue Song & Shujun Liu & Kun Huang, 2021. "Defining High-Quality Answers on a Chinese Tourism Q&A Platform in Terms of Information Needs," Sustainability, MDPI, vol. 13(24), pages 1-21, December.
    2. Samer Ali Al-shami & Salem Aldahmani & Massila Kamalrudin & Nabil Hasan Al-Kumaim & Abdullah Al Mamun & Mohammed Al-shami & Mustafa Musa Jaber, 2022. "A Model of Motivational and Technological Factors Influencing Massive Open Online Courses’ Continuous Intention to Use," Sustainability, MDPI, vol. 14(15), pages 1-23, July.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Roy, Pradeep K. & Singh, Jyoti P. & Baabdullah, Abdullah M. & Kizgin, Hatice & Rana, Nripendra P., 2018. "Identifying reputation collectors in community question answering (CQA) sites: Exploring the dark side of social media," International Journal of Information Management, Elsevier, vol. 42(C), pages 25-35.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jsusta:v:13:y:2021:i:16:p:9364-:d:618500. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.