IDEAS home Printed from https://ideas.repec.org/a/inm/orisre/v34y2023i1p27-49.html
   My bibliography  Save this article

Could Gamification Designs Enhance Online Learning Through Personalization? Lessons from a Field Experiment

Author

Listed:
  • Alvin Chung Man Leung

    (Department of Information Systems, College of Business, City University of Hong Kong, Hong Kong)

  • Radhika Santhanam

    (Division of MIS, Price College of Business, University of Oklahoma, Norman, Oklahoma 73019)

  • Ron Chi-Wai Kwok

    (Department of Information Systems, College of Business, City University of Hong Kong, Hong Kong)

  • Wei Thoo Yue

    (Department of Information Systems, College of Business, City University of Hong Kong, Hong Kong)

Abstract

The COVID-19 pandemic has highlighted the importance of online learning. As learner autonomy is relatively high in online environments, learners must engage in self-regulated learning (SRL) to achieve optimal learning outcomes. Because most learners are unable to consistently engage in SRL, gamification interventions are being implemented to improve SRL engagement; however, mixed results cast doubt on the efficacy of this approach. Massively open online courses (MOOCs), a type of online learning environment, are currently experiencing rapid growth due to widespread adoption by many institutions. In MOOCs, there is no instructor intervention; hence, students have difficulty regulating their own learning and are easily distracted. Therefore, this study investigates whether mixed-research results regarding the efficacy of gamification can be attributed to lack of attention to individual learner traits during design. For this purpose, the study analyzes MOOCs as an instance of online learning by applying SRL theory and gamification principles. We altered a traditional MOOC platform to provide different types of gamified performance feedback to facilitate learners’ SRL engagement. We then examined whether this matched with goal orientation, an individual learner trait to influence SRL and learning outcomes. Using learning-analytics tools, we tracked 760 college students’ SRL engagement on a MOOC platform over five weeks. As theorized, SRL engagement and learning outcomes of participants who had a strong performance-avoidance goal orientation increased with positively framed performance feedback that involved no social comparisons; however, the same feedback had a negative impact on participants with a strong mastery goal orientation. Our findings add to SRL theory by demonstrating that gamification designs can enhance SRL engagement and learning outcomes in online learning, but with a caveat—this occurs only when there is a match with learner traits—confirming the gamification principle stating that task improvements and meaningful engagement can only occur through thoughtful gamification design.

Suggested Citation

  • Alvin Chung Man Leung & Radhika Santhanam & Ron Chi-Wai Kwok & Wei Thoo Yue, 2023. "Could Gamification Designs Enhance Online Learning Through Personalization? Lessons from a Field Experiment," Information Systems Research, INFORMS, vol. 34(1), pages 27-49, March.
    • Handle: RePEc:inm:orisre:v:34:y:2023:i:1:p:27-49
    DOI: 10.1287/isre.2022.1123
    as

    Download full text from publisher

    File URL: http://dx.doi.org/10.1287/isre.2022.1123
    Download Restriction: no
    File URL: https://libkey.io/10.1287/isre.2022.1123?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Radhika Santhanam & De Liu & Wei-Cheng Milton Shen, 2016. "Research Note—Gamification of Technology-Mediated Training: Not All Competitions Are the Same," Information Systems Research, INFORMS, vol. 27(2), pages 453-465, June.
    2. Jean Tirole & Roland Bénabou, 2006. "Incentives and Prosocial Behavior," American Economic Review, American Economic Association, vol. 96(5), pages 1652-1678, December.
    3. Radhika Santhanam & Sharath Sasidharan & Jane Webster, 2008. "Using Self-Regulatory Learning to Enhance E-Learning-Based Information Technology Training," Information Systems Research, INFORMS, vol. 19(1), pages 26-47, March.
    4. Amrit Tiwana & Benn Konsynski & Ashley A. Bush, 2010. "Research Commentary ---Platform Evolution: Coevolution of Platform Architecture, Governance, and Environmental Dynamics," Information Systems Research, INFORMS, vol. 21(4), pages 675-687, December.
    5. Panigrahi, Ritanjali & Srivastava, Praveen Ranjan & Sharma, Dheeraj, 2018. "Online learning: Adoption, continuance, and learning outcome—A review of literature," International Journal of Information Management, Elsevier, vol. 43(C), pages 1-14.
    6. Gabriele Piccoli & Joaquin Rodriguez & Biagio Palese & Marcin Lukasz Bartosiak, 2020. "Feedback at scale: designing for accurate and timely practical digital skills evaluation," European Journal of Information Systems, Taylor & Francis Journals, vol. 29(2), pages 114-133, March.
    7. Koivisto, Jonna & Hamari, Juho, 2019. "The rise of motivational information systems: A review of gamification research," International Journal of Information Management, Elsevier, vol. 45(C), pages 191-210.
    8. Bandura, Albert, 1991. "Social cognitive theory of self-regulation," Organizational Behavior and Human Decision Processes, Elsevier, vol. 50(2), pages 248-287, December.
    9. Rajeev H. Dehejia & Sadek Wahba, 2002. "Propensity Score-Matching Methods For Nonexperimental Causal Studies," The Review of Economics and Statistics, MIT Press, vol. 84(1), pages 151-161, February.
    10. Maryam Alavi & Dorothy E. Leidner, 2001. "Research Commentary: Technology-Mediated Learning—A Call for Greater Depth and Breadth of Research," Information Systems Research, INFORMS, vol. 12(1), pages 1-10, March.
    11. Peter C. Austin, 2017. "A Tutorial on Multilevel Survival Analysis: Methods, Models and Applications," International Statistical Review, International Statistical Institute, vol. 85(2), pages 185-203, August.
    12. Van de Velde, Liesbeth & Verbeke, Wim & Popp, Michael & Van Huylenbroeck, Guido, 2010. "The importance of message framing for providing information about sustainability and environmental aspects of energy," Energy Policy, Elsevier, vol. 38(10), pages 5541-5549, October.
    13. Krishnamurthy, Parthasarathy & Carter, Patrick & Blair, Edward, 2001. "Attribute Framing and Goal Framing Effects in Health Decisions," Organizational Behavior and Human Decision Processes, Elsevier, vol. 85(2), pages 382-399, July.
    14. Mun Y. Yi & Fred D. Davis, 2003. "Developing and Validating an Observational Learning Model of Computer Software Training and Skill Acquisition," Information Systems Research, INFORMS, vol. 14(2), pages 146-169, June.
    15. Shuk Ying Ho & David Bodoff & Kar Yan Tam, 2011. "Timing of Adaptive Web Personalization and Its Effects on Online Consumer Behavior," Information Systems Research, INFORMS, vol. 22(3), pages 660-679, September.
    16. Levin, Irwin P. & Schneider, Sandra L. & Gaeth, Gary J., 1998. "All Frames Are Not Created Equal: A Typology and Critical Analysis of Framing Effects," Organizational Behavior and Human Decision Processes, Elsevier, vol. 76(2), pages 149-188, November.
    17. Ni Huang & Jiayin Zhang & Gordon Burtch & Xitong Li & Peiyu Chen, 2021. "Combating Procrastination on Massive Online Open Courses via Optimal Calls to Action," Information Systems Research, INFORMS, vol. 32(2), pages 301-317, June.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Amalia R. Miller & Kamalini Ramdas & Alp Sungu, 2023. "Browsers Don’t Lie? Gender Differences in the Effects of the Indian COVID-19 Lockdown on Digital Activity and Time Use," NBER Working Papers 31919, National Bureau of Economic Research, Inc.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Radhika Santhanam & De Liu & Wei-Cheng Milton Shen, 2016. "Research Note—Gamification of Technology-Mediated Training: Not All Competitions Are the Same," Information Systems Research, INFORMS, vol. 27(2), pages 453-465, June.
    2. Radhika Santhanam & Sharath Sasidharan & Jane Webster, 2008. "Using Self-Regulatory Learning to Enhance E-Learning-Based Information Technology Training," Information Systems Research, INFORMS, vol. 19(1), pages 26-47, March.
    3. Saurabh Gupta & Robert Bostrom, 2013. "Research Note ---An Investigation of the Appropriation of Technology-Mediated Training Methods Incorporating Enactive and Collaborative Learning," Information Systems Research, INFORMS, vol. 24(2), pages 454-469, June.
    4. Ellingsen, Tore & Johannesson, Magnus & Mollerstrom, Johanna & Munkhammar, Sara, 2012. "Social framing effects: Preferences or beliefs?," Games and Economic Behavior, Elsevier, vol. 76(1), pages 117-130.
    5. repec:cup:judgdm:v:13:y:2018:i:6:p:529-546 is not listed on IDEAS
    6. Jain, Gaurav & Gaeth, Gary J. & Nayakankuppam, Dhananjay & Levin, Irwin P., 2020. "Revisiting attribute framing: The impact of number roundedness on framing," Organizational Behavior and Human Decision Processes, Elsevier, vol. 161(C), pages 109-119.
    7. Reitmann, Ann-Kristin & Goedhuys, Micheline & Grimm, Michael & Nillesen, Eleonora E.M., 2020. "Gender attitudes in the Arab region – The role of framing and priming effects," Journal of Economic Psychology, Elsevier, vol. 80(C).
    8. Van de Velde, Liesbeth & Verbeke, Wim & Popp, Michael & Van Huylenbroeck, Guido, 2010. "The importance of message framing for providing information about sustainability and environmental aspects of energy," Energy Policy, Elsevier, vol. 38(10), pages 5541-5549, October.
    9. Freling, Traci H. & Vincent, Leslie H. & Henard, David H., 2014. "When not to accentuate the positive: Re-examining valence effects in attribute framing," Organizational Behavior and Human Decision Processes, Elsevier, vol. 124(2), pages 95-109.
    10. Dimant, Eugen & van Kleef, Gerben A. & Shalvi, Shaul, 2020. "Requiem for a Nudge: Framing effects in nudging honesty," Journal of Economic Behavior & Organization, Elsevier, vol. 172(C), pages 247-266.
    11. Boun My, Kene & Ouvrard, Benjamin, 2019. "Nudge and tax in an environmental public goods experiment: Does environmental sensitivity matter?," Resource and Energy Economics, Elsevier, vol. 55(C), pages 24-48.
    12. Håkan J. Holm & Victor Nee & Sonja Opper, 2020. "Strategic decisions: behavioral differences between CEOs and others," Experimental Economics, Springer;Economic Science Association, vol. 23(1), pages 154-180, March.
    13. Laura Abrardi, 2019. "Behavioral barriers and the energy efficiency gap: a survey of the literature," Economia e Politica Industriale: Journal of Industrial and Business Economics, Springer;Associazione Amici di Economia e Politica Industriale, vol. 46(1), pages 25-43, March.
    14. Wang, Jianming & Li, Yongqiang & He, Zhengxia & Gao, Jian & Wang, Jianguo, 2022. "Scale framing, benefit framing and their interaction effects on energy-saving behaviors: Evidence from urban residents of China," Energy Policy, Elsevier, vol. 166(C).
    15. Martin Adam & Konstantin Roethke & Alexander Benlian, 2022. "Gamblified digital product offerings: an experimental study of loot box menu designs," Electronic Markets, Springer;IIM University of St. Gallen, vol. 32(2), pages 971-986, June.
    16. Michael Kühnen & Samanthi Silva & Rüdiger Hahn, 2022. "From negative to positive sustainability performance measurement and assessment? A qualitative inquiry drawing on framing effects theory," Business Strategy and the Environment, Wiley Blackwell, vol. 31(5), pages 1985-2001, July.
    17. Siqi Dai & Kai Chen & Rui Jin, 2022. "The effect of message framing and language intensity on green consumption behavior willingness," Environment, Development and Sustainability: A Multidisciplinary Approach to the Theory and Practice of Sustainable Development, Springer, vol. 24(2), pages 2432-2452, February.
    18. Juin-Ming Tsai & Shiu-Wan Hung & Guan-Ting Lin, 2022. "Continued usage of smart wearable devices (SWDs): cross-level analysis of gamification and network externality," Electronic Markets, Springer;IIM University of St. Gallen, vol. 32(3), pages 1661-1676, September.
    19. Cinzia Castiglioni & Edoardo Lozza & Eric Dijk & Wilco W. Dijk, 2019. "Two sides of the same coin? An investigation of the effects of frames on tax compliance and charitable giving," Palgrave Communications, Palgrave Macmillan, vol. 5(1), pages 1-10, December.
    20. Tore Ellingsen & Magnus Johannesson, 2008. "Pride and Prejudice: The Human Side of Incentive Theory," American Economic Review, American Economic Association, vol. 98(3), pages 990-1008, June.
    21. Ni Huang & Lingli Wang & Yili Hong & Lihui Lin & Xunhua Guo & Guoqing Chen, 2024. "When the Clock Strikes: A Multimethod Investigation of On-the-Hour Effects in Online Learning," Information Systems Research, INFORMS, vol. 35(2), pages 766-782, June.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:orisre:v:34:y:2023:i:1:p:27-49. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.