IDEAS home Printed from https://ideas.repec.org/a/gam/jsusta/v14y2022i6p3392-d770684.html
   My bibliography  Save this article

Design and Validation of a Novel Tool to Assess Citizens’ Netiquette and Information and Data Literacy Using Interactive Simulations

Author

Listed:
  • Juan Bartolomé

    (TECNALIA, Basque Research and Technology Alliance (BRTA), 48160 Derio Bizkaia, Spain)

  • Pablo Garaizar

    (Engineering Faculty, University of Deusto, 48007 Bilbao, Spain)

Abstract

Until recently, most of the digital literacy frameworks have been based on assessment frameworks used by commercial entities. The release of the DigComp framework has allowed the development of tailored implementations for the evaluation of digital competence. However, the majority of these digital literacy frameworks are based on self-assessments, measuring only low-order cognitive skills. This paper reports on a study to develop and validate an assessment instrument, including interactive simulations to assess citizens’ digital competence. These formats are particularly important for the evaluation of complex cognitive constructs such as digital competence. Additionally, we selected two different approaches for designing the tests based on their scope, at the competence or competence area level. Their overall and dimensional validity and reliability were analysed. We summarise the issues addressed in each phase and key points to consider in new implementations. For both approaches, items present satisfactory difficulty and discrimination indicators. Validity was ensured through expert validation, and the Rasch analysis revealed good EAP/PV reliabilities. Therefore, the tests have sound psychometric properties that make them reliable and valid instruments for measuring digital competence. This paper contributes to an increasing number of tools designed to evaluate digital competence and highlights the necessity of measuring higher-order cognitive skills.

Suggested Citation

  • Juan Bartolomé & Pablo Garaizar, 2022. "Design and Validation of a Novel Tool to Assess Citizens’ Netiquette and Information and Data Literacy Using Interactive Simulations," Sustainability, MDPI, vol. 14(6), pages 1-28, March.
    • Handle: RePEc:gam:jsusta:v:14:y:2022:i:6:p:3392-:d:770684
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2071-1050/14/6/3392/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2071-1050/14/6/3392/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. David Thissen, 1982. "Marginal maximum likelihood estimation for the one-parameter logistic model," Psychometrika, Springer;The Psychometric Society, vol. 47(2), pages 175-186, June.
    2. Ana Isabel Santos & Sandro Serpa, 2017. "The Importance of Promoting Digital Literacy in Higher Education," International Journal of Social Science Studies, Redfame publishing, vol. 5(6), pages 90-93, June.
    3. Rafael Saltos-Rivas & Pavel Novoa-Hernández & Rocío Serrano Rodríguez, 2021. "On the quality of quantitative instruments to measure digital competence in higher education: A systematic mapping study," PLOS ONE, Public Library of Science, vol. 16(9), pages 1-27, September.
    4. Kopecky, Kamil & Szotkowski, Rene & Aznar-Díaz, Inmaculada & Romero-Rodríguez, José-María, 2020. "The phenomenon of sharenting and its risks in the online environment. Experiences from Czech Republic and Spain," Children and Youth Services Review, Elsevier, vol. 110(C).
    5. Rotimi Abidoye & Benson Teck Heng Lim & Yu-Cheng Lin & Junge Ma, 2022. "Equipping Property Graduates for the Digital Age," Sustainability, MDPI, vol. 14(2), pages 1-23, January.
    6. Katriona O’Sullivan & Serena Clark & Kevin Marshall & Malcolm MacLachlan, 2021. "A Just Digital framework to ensure equitable achievement of the Sustainable Development Goals," Nature Communications, Nature, vol. 12(1), pages 1-4, December.
    7. Javier Portillo & Urtza Garay & Eneko Tejada & Naiara Bilbao, 2020. "Self-Perception of the Digital Competence of Educators during the COVID-19 Pandemic: A Cross-Analysis of Different Educational Stages," Sustainability, MDPI, vol. 12(23), pages 1-13, December.
    8. Stefano Kluzer & Laia Pujol Priego, 2018. "DigComp into action: Get inspired, make it happen. A user guide to the European Digital Competence Framework," JRC Research Reports JRC110624, Joint Research Centre.
    9. Ana Pérez-Escoda & Luis Miguel Pedrero-Esteban & Juana Rubio-Romero & Carlos Jiménez-Narros, 2021. "Fake News Reaching Young People on Social Networks: Distrust Challenging Media Literacy," Publications, MDPI, vol. 9(2), pages 1-16, June.
    10. Stephanie Carretero & Riina Vuorikari & Yves Punie, 2017. "DigComp 2.1: The Digital Competence Framework for Citizens with eight proficiency levels and examples of use," JRC Research Reports JRC106281, Joint Research Centre.
    11. Alexander J.A.M. Van Deursen & Ellen J. Helsper & Rebecca Eynon, 2016. "Development and validation of the Internet Skills Scale (ISS)," LSE Research Online Documents on Economics 64485, London School of Economics and Political Science, LSE Library.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Marcelo Rioseco-Pais & Juan Silva-Quiroz & Rodrigo Vargas-Vitoria, 2024. "Digital Competences and Years of Access to Technologies Among Chilean University Students: An Analysis Based on the DIGCOMP Framework," Sustainability, MDPI, vol. 16(22), pages 1-14, November.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yu Zhao & María Cruz Sánchez Gómez & Ana María Pinto Llorente & Liping Zhao, 2021. "Digital Competence in Higher Education: Students’ Perception and Personal Factors," Sustainability, MDPI, vol. 13(21), pages 1-17, November.
    2. Núñez-Canal, Margarita & de Obesso, Mª de las Mercedes & Pérez-Rivero, Carlos Alberto, 2022. "New challenges in higher education: A study of the digital competence of educators in Covid times," Technological Forecasting and Social Change, Elsevier, vol. 174(C).
    3. Junfeng Yang & Ahmed Tlili & Ronghuai Huang & Rongxia Zhuang & Kaushal Kumar Bhagat, 2021. "Development and Validation of a Digital Learning Competence Scale: A Comprehensive Review," Sustainability, MDPI, vol. 13(10), pages 1-14, May.
    4. Cordella, Antonio & Gualdi, Francesco & van de Laar, Mindel, 2024. "Digital skills within the public sector: a missing link to achieve the Sustainable Development Goals (SDGs)," LSE Research Online Documents on Economics 122256, London School of Economics and Political Science, LSE Library.
    5. Maria José Sá & Ana Isabel Santos & Sandro Serpa & Carlos Miguel Ferreira, 2021. "Digitainability—Digital Competences Post-COVID-19 for a Sustainable Society," Sustainability, MDPI, vol. 13(17), pages 1-22, August.
    6. Alberto Maydeu-Olivares & Rosa Montaño, 2013. "How Should We Assess the Fit of Rasch-Type Models? Approximating the Power of Goodness-of-Fit Statistics in Categorical Data Analysis," Psychometrika, Springer;The Psychometric Society, vol. 78(1), pages 116-133, January.
    7. Trung Tran & Manh-Toan Ho & Thanh-Hang Pham & Minh-Hoang Nguyen & Khanh-Linh P. Nguyen & Thu-Trang Vuong & Thanh-Huyen T. Nguyen & Thanh-Dung Nguyen & Thi-Linh Nguyen & Quy Khuc & Viet-Phuong La & Qua, 2020. "How Digital Natives Learn and Thrive in the Digital Age: Evidence from an Emerging Economy," Sustainability, MDPI, vol. 12(9), pages 1-24, May.
    8. Haris Krijestorac & Rajiv Garg & Prabhudev Konana, 2021. "Decisions Under the Illusion of Objectivity: Digital Embeddedness and B2B Purchasing," Production and Operations Management, Production and Operations Management Society, vol. 30(7), pages 2232-2251, July.
    9. Cees Glas, 1988. "The derivation of some tests for the rasch model from the multinomial distribution," Psychometrika, Springer;The Psychometric Society, vol. 53(4), pages 525-546, December.
    10. Haruhiko Ogasawara, 2013. "Asymptotic properties of the Bayes modal estimators of item parameters in item response theory," Computational Statistics, Springer, vol. 28(6), pages 2559-2583, December.
    11. Lamberti, Giuseppe & Lopez-Sintas, Jordi & Sukphan, Jakkapong, 2021. "The social process of internet appropriation: Living in a digitally advanced country benefits less well-educated Europeans," Telecommunications Policy, Elsevier, vol. 45(1).
    12. Olivanti, Francesco & Gastaldi, Luca, 2024. "Evaluating digital skills policies: Assessing the potential impact of outreach programs in Italy," Telecommunications Policy, Elsevier, vol. 48(8).
    13. Artur Santoalha & Davide Consoli & Fulvio Castellacci, 2019. "Do digital skills foster green diversification? A study of European regions," Working Papers on Innovation Studies 20191029, Centre for Technology, Innovation and Culture, University of Oslo.
    14. Li Cai, 2010. "A Two-Tier Full-Information Item Factor Analysis Model with Applications," Psychometrika, Springer;The Psychometric Society, vol. 75(4), pages 581-612, December.
    15. David Thissen & Howard Wainer, 1982. "Some standard errors in item response theory," Psychometrika, Springer;The Psychometric Society, vol. 47(4), pages 397-412, December.
    16. César Torres Martín & Christian Acal & Mohammed El Homrani & Ángel Custodio Mingorance Estrada, 2021. "Impact on the Virtual Learning Environment Due to COVID-19," Sustainability, MDPI, vol. 13(2), pages 1-16, January.
    17. Godbersen, Hendrik & Hausinger, Lena, 2022. "Die Travel Journey von Digital Natives und Digital Immigrants: Eine vergleichende Analyse der digitalen und analogen Customer Touchpoints im Buchungsprozess von Urlaubsreisen," KCMS Schriftenreihe, FOM Hochschule für Oekonomie & Management, KCMS KompetenzCentrum für Marketing & Sales Management, volume 6, number 6 edited by FOM Hochschule für Oekonomie & Management, KCMS KompetenzCentrum für Marketing & Sales Management, July.
    18. Romero-Rodríguez, José-M. & Kopecký, Kamil & García-González, Abel & Gómez-García, Gerardo, 2022. "Sharing images or videos of minors online: Validation of the Sharenting Evaluation Scale (SES)," Children and Youth Services Review, Elsevier, vol. 136(C).
    19. Sandro Serpa & Ana Isabel Santos & Carlos Miguel Ferreira, 2020. "Contributions of Ivan Illich to Education in a Digital Society," Academic Journal of Interdisciplinary Studies, Richtmann Publishing Ltd, vol. 9, March.
    20. Lorena Martín-Párraga & Carmen Llorente-Cejudo & Julio Barroso-Osuna, 2023. "Self-Perception of Digital Competence in University Lecturers: A Comparative Study between Universities in Spain and Peru According to the DigCompEdu Model," Societies, MDPI, vol. 13(6), pages 1-14, June.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jsusta:v:14:y:2022:i:6:p:3392-:d:770684. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.