IDEAS home Printed from https://ideas.repec.org/a/eee/jeborg/v153y2018icp293-321.html
   My bibliography  Save this article

Can behavioral tools improve online student outcomes? Experimental evidence from a massive open online course

Author

Listed:
  • Patterson, Richard W.

Abstract

In order to address poor outcomes for online students, I leverage insights from behavioral economics to design three software tools including (1) a commitment device, (2) an alert tool, and (3) a distraction blocking tool. I test the impact of these tools in a massive open online course (MOOC). Relative to students in the control group, students in the commitment device treatment spend 24% more time working on the course, receive course grades that are 0.29 standard deviations higher, and are 40% more likely to complete the course. In contrast, outcomes for students in the alert and distraction blocking treatments are statistically indistinguishable from the control.

Suggested Citation

  • Patterson, Richard W., 2018. "Can behavioral tools improve online student outcomes? Experimental evidence from a massive open online course," Journal of Economic Behavior & Organization, Elsevier, vol. 153(C), pages 293-321.
    • Handle: RePEc:eee:jeborg:v:153:y:2018:i:c:p:293-321
    DOI: 10.1016/j.jebo.2018.06.017
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0167268118301768
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.jebo.2018.06.017?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Emre Ozdenoren & Stephen W. Salant & Dan Silverman, 2012. "Willpower And The Optimal Control Of Visceral Urges," Journal of the European Economic Association, European Economic Association, vol. 10(2), pages 342-368, April.
    2. Richard H. Thaler & Shlomo Benartzi, 2004. "Save More Tomorrow (TM): Using Behavioral Economics to Increase Employee Saving," Journal of Political Economy, University of Chicago Press, vol. 112(S1), pages 164-187, February.
    3. David K. Levine & Drew Fudenberg, 2006. "A Dual-Self Model of Impulse Control," American Economic Review, American Economic Association, vol. 96(5), pages 1449-1476, December.
    4. Matthew Rabin & Ted O'Donoghue, 1999. "Doing It Now or Later," American Economic Review, American Economic Association, vol. 89(1), pages 103-124, March.
    5. Dean Karlan & Sendhil Mullainathan & Margaret McConnell & Jonathan Zinman, 2010. "Getting to theTop of Mind: How Reminders Increase Saving," Working Papers id:2587, eSocialSciences.
    6. Houser, Daniel & Schunk, Daniel & Winter, Joachim & Xiao, Erte, 2018. "Temptation and commitment in the laboratory," Games and Economic Behavior, Elsevier, vol. 107(C), pages 329-344.
    7. Drew Fudenberg & David K. Levine, 2012. "Timing and Self‐Control," Econometrica, Econometric Society, vol. 80(1), pages 1-42, January.
    8. Steven D. Levitt & John A. List & Susanne Neckermann & Sally Sadoff, 2016. "The Behavioralist Goes to School: Leveraging Behavioral Economics to Improve Educational Performance," American Economic Journal: Economic Policy, American Economic Association, vol. 8(4), pages 183-219, November.
    9. Alberto Abadie & Matthew M. Chingos & Martin R. West, 2018. "Endogenous Stratification in Randomized Experiments," The Review of Economics and Statistics, MIT Press, vol. 100(4), pages 567-580, October.
    10. Nava Ashraf & Dean Karlan & Wesley Yin, 2006. "Tying Odysseus to the Mast: Evidence From a Commitment Savings Product in the Philippines," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 121(2), pages 635-672.
    11. Keith M. Marzilli Ericson, 2011. "Forgetting We Forget: Overconfidence And Memory," Journal of the European Economic Association, European Economic Association, vol. 9(1), pages 43-60, February.
    12. Brigitte C. Madrian & Dennis F. Shea, 2001. "The Power of Suggestion: Inertia in 401(k) Participation and Savings Behavior," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 116(4), pages 1149-1187.
    13. Raj Chetty & Adam Looney & Kory Kroft, 2009. "Salience and Taxation: Theory and Evidence," American Economic Review, American Economic Association, vol. 99(4), pages 1145-1177, September.
    14. Xavier Giné & Dean Karlan & Jonathan Zinman, 2010. "Put Your Money Where Your Butt Is: A Commitment Contract for Smoking Cessation," American Economic Journal: Applied Economics, American Economic Association, vol. 2(4), pages 213-235, October.
    15. Abhijit V. Banerjee & Esther Duflo, 2014. "(Dis)organization and Success in an Economics MOOC," American Economic Review, American Economic Association, vol. 104(5), pages 514-518, May.
    16. Ximena Cadena & Antoinette Schoar, 2011. "Remembering to Pay? Reminders vs. Financial Incentives for Loan Payments," NBER Working Papers 17020, National Bureau of Economic Research, Inc.
    17. David Laibson, 1997. "Golden Eggs and Hyperbolic Discounting," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 112(2), pages 443-478.
    18. Keith M. Marzilli Ericson, 2014. "On the Interaction of Memory and Procrastination: Implications for Reminders," NBER Working Papers 20381, National Bureau of Economic Research, Inc.
    19. Felipe Kast & Stephan Meier & Dina Pomeranz, 2012. "Under-Savers Anonymous: Evidence on Self-Help Groups and Peer Pressure as a Savings Commitment Device," NBER Working Papers 18417, National Bureau of Economic Research, Inc.
    20. Daniel Zizzo, 2010. "Experimenter demand effects in economic experiments," Experimental Economics, Springer;Economic Science Association, vol. 13(1), pages 75-98, March.
    21. Calzolari, Giacomo & Nardotto, Mattia, 2011. "Nudging with information: a randomized field experiment on reminders and feedback," CEPR Discussion Papers 8571, C.E.P.R. Discussion Papers.
    22. Romano, Joseph P. & Wolf, Michael, 2016. "Efficient computation of adjusted p-values for resampling-based stepdown multiple testing," Statistics & Probability Letters, Elsevier, vol. 113(C), pages 38-40.
    23. Sendhil Mullainathan, 2002. "A Memory-Based Model of Bounded Rationality," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 117(3), pages 735-774.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Hardt, David & Nagler, Markus & Rincke, Johannes, 2023. "Tutoring in (online) higher education: Experimental evidence," Economics of Education Review, Elsevier, vol. 92(C).
    2. Raphael Brade & Oliver Himmler & Robert Jaeckle & Philipp Weinschenk, 2024. "Helping Students to Succeed – The Long-Term Effects of Soft Commitments and Reminders," CESifo Working Paper Series 11001, CESifo.
    3. Damon Clark & David Gill & Victoria Prowse & Mark Rush, 2020. "Using Goals to Motivate College Students: Theory and Evidence From Field Experiments," The Review of Economics and Statistics, MIT Press, vol. 102(4), pages 648-663, October.
    4. Damgaard, Mette Trier & Nielsen, Helena Skyt, 2018. "Nudging in education," Economics of Education Review, Elsevier, vol. 64(C), pages 313-342.
    5. Raphael Brade & Oliver Himmler & Robert Jaeckle, 2023. "Relative Performance Feedback and Long-Term Tasks – Experimental Evidence from Higher Education," CESifo Working Paper Series 10346, CESifo.
    6. Andor, Mark A. & Fels, Katja M. & Renz, Jan & Rzepka, Sylvi, 2018. "Do planning prompts increase educational success? Evidence from randomized controlled trials in MOOCs," Ruhr Economic Papers 790, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
    7. Bulman, George & Fairlie, Robert W, 2015. "Technology and Education: Computers, Software, and the Internet," Santa Cruz Department of Economics, Working Paper Series qt5265z87t, Department of Economics, UC Santa Cruz.
    8. Nicholas A. Wright & Puneet Arora & Jesse Wright, 2024. "I Promise to Work Hard: The Impact of a Non-Binding Commitment Pledge on Academic Performance," Working Papers 2411, Florida International University, Department of Economics.
    9. Emily Beam & Priya Mukherjee & Laia Navarro-Sola, 2022. "Lowering Barriers to Remote Education: Experimental Impacts on Parental Responses and Learning," Working Papers 2022-030, Human Capital and Economic Opportunity Working Group.
    10. Hardt, David & Nagler, Markus & Rincke, Johannes, 2022. "Can peer mentoring improve online teaching effectiveness? An RCT during the COVID-19 pandemic," Labour Economics, Elsevier, vol. 78(C).
    11. Novella, Rafael & Rosas-Shady, David & Freund, Richard, 2024. "Is online job training for all? Experimental evidence on the effects of a Coursera program in Costa Rica," Journal of Development Economics, Elsevier, vol. 169(C).
    12. Oliver Himmler & Robert Jäckle & Philipp Weinschenk, 2019. "Soft Commitments, Reminders, and Academic Performance," American Economic Journal: Applied Economics, American Economic Association, vol. 11(2), pages 114-142, April.
    13. Pugatch, Todd & Schroeder, Elizabeth & Wilson, Nicholas, 2022. "Study More Tomorrow," GLO Discussion Paper Series 1115, Global Labor Organization (GLO).
    14. Rachel Baker & Brent Evans & Qiujie Li & Bianca Cung, 2019. "Does Inducing Students to Schedule Lecture Watching in Online Classes Improve Their Academic Performance? An Experimental Analysis of a Time Management Intervention," Research in Higher Education, Springer;Association for Institutional Research, vol. 60(4), pages 521-552, June.
    15. Gong, Jie & Liu, Tracy Xiao & Tang, Jie, 2021. "How monetary incentives improve outcomes in MOOCs: Evidence from a field experiment," Journal of Economic Behavior & Organization, Elsevier, vol. 190(C), pages 905-921.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Dean Karlan & Margaret McConnell & Sendhil Mullainathan & Jonathan Zinman, 2016. "Getting to the Top of Mind: How Reminders Increase Saving," Management Science, INFORMS, vol. 62(12), pages 3393-3411, December.
    2. Sebastian Vollmer & Juditha Wójcik, 2017. "The long-term consequences of the global 1918 influenza pandemic: A systematic analysis of 117 IPUMS international census data sets," Courant Research Centre: Poverty, Equity and Growth - Discussion Papers 242, Courant Research Centre PEG.
    3. Houser, Daniel & Schunk, Daniel & Winter, Joachim & Xiao, Erte, 2018. "Temptation and commitment in the laboratory," Games and Economic Behavior, Elsevier, vol. 107(C), pages 329-344.
    4. Uhr, Charline & Meyer, Steffen & Hackethal, Andreas, 2021. "Smoking hot portfolios? Trading behavior, investment biases, and self-control failure," Journal of Empirical Finance, Elsevier, vol. 63(C), pages 73-95.
    5. Laureti, Carolina & Szafarz, Ariane, 2023. "Banking regulation and costless commitment contracts for time-inconsistent agents," Economic Modelling, Elsevier, vol. 129(C).
    6. Stefano DellaVigna, 2009. "Psychology and Economics: Evidence from the Field," Journal of Economic Literature, American Economic Association, vol. 47(2), pages 315-372, June.
    7. Anett John, 2020. "When Commitment Fails: Evidence from a Field Experiment," Management Science, INFORMS, vol. 66(2), pages 503-529, February.
    8. Derksen, Laura & Kerwin, Jason Theodore & Reynoso, Natalia Ordaz & Sterck, Olivier, 2021. "Appointments: A More Effective Commitment Device for Health Behaviors," SocArXiv y8gh7, Center for Open Science.
    9. Dean Karlan & Aishwarya Lakshmi Ratan & Jonathan Zinman, 2014. "Savings by and for the Poor: A Research Review and Agenda," Review of Income and Wealth, International Association for Research in Income and Wealth, vol. 60(1), pages 36-78, March.
    10. Brigitte C. Madrian, 2014. "Applying Insights from Behavioral Economics to Policy Design," Annual Review of Economics, Annual Reviews, vol. 6(1), pages 663-688, August.
    11. Tasoff, Joshua & Letzler, Robert, 2014. "Everyone believes in redemption: Nudges and overoptimism in costly task completion," Journal of Economic Behavior & Organization, Elsevier, vol. 107(PA), pages 107-122.
    12. Bisin, Alberto & Hyndman, Kyle, 2020. "Present-bias, procrastination and deadlines in a field experiment," Games and Economic Behavior, Elsevier, vol. 119(C), pages 339-357.
    13. Houser, Daniel & Liu, Jia & Reiley, David H. & Urbancic, Michael B., 2021. "Checking out temptation: A natural experiment with purchases at the grocery register," Journal of Economic Behavior & Organization, Elsevier, vol. 191(C), pages 39-50.
    14. B. Douglas Bernheim & Debraj Ray & Şevin Yeltekin, 2015. "Poverty and Self‐Control," Econometrica, Econometric Society, vol. 83(5), pages 1877-1911, September.
    15. Xavier Gabaix, 2017. "Behavioral Inattention," NBER Working Papers 24096, National Bureau of Economic Research, Inc.
    16. Kohei Kubota & Mototsugu Fukushige, 2016. "Rational Consumers," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 57(1), pages 231-254, February.
    17. Emma Boswell Dean & Frank Schilbach & Heather Schofield, 2017. "Poverty and Cognitive Function," NBER Chapters, in: The Economics of Poverty Traps, pages 57-118, National Bureau of Economic Research, Inc.
    18. Christine L. Exley & Jeffrey K. Naecker, 2017. "Observability Increases the Demand for Commitment Devices," Management Science, INFORMS, vol. 63(10), pages 3262-3267, October.
    19. Altmann, Steffen & Traxler, Christian, 2014. "Nudges at the dentist," European Economic Review, Elsevier, vol. 72(C), pages 19-38.
    20. Kyle Hyndman & Alberto Bisin, 2022. "Procrastination, self-imposed deadlines and other commitment devices," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 74(3), pages 871-897, October.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:jeborg:v:153:y:2018:i:c:p:293-321. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/jebo .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.