IDEAS home Printed from https://ideas.repec.org/a/inm/ormsom/v19y2017i3p347-367.html
   My bibliography  Save this article

Does Social Interaction Improve Learning Outcomes? Evidence from Field Experiments on Massive Open Online Courses

Author

Listed:
  • Dennis J. Zhang

    (Olin Business School, Washington University in St. Louis, St. Louis, Missouri 63130)

  • Gad Allon

    (Wharton School, University of Pennsylvania, Philadelphia, Pennsylvania 19104)

  • Jan A. Van Mieghem

    (Kellogg School of Management, Northwestern University, Evanston, Illinois 60208)

Abstract

This paper studies how service providers can design social interaction among participants and quantify the causal impact of that interaction on service quality. We focus on education and analyze whether encouraging social interaction among students improves learning outcomes in massive open online courses (MOOCs), which are a new service delivery channel with universal access at reduced, if not zero, cost. We analyze three randomized experiments in a MOOC with more than 30,317 students from 183 countries. Two experiments study large-group interaction by encouraging a random subset of students to visit the course discussion board. The majority of students treated in these experiments had higher social engagement, higher quiz completion rates, and higher course grades. Using these treatments as instrumental variables, we estimate that one additional board visit causally increases the probability that a student finishes the quiz in the subsequent week by up to 4.3%. The third experiment studies small-group interaction by encouraging a random subset of students to conduct one-on-one synchronous discussions. Students who followed through and actually conducted pairwise discussions increased their quiz completion rates and quiz scores by 10% in the subsequent week. Combining results from these three experiments, we provide recommendations for designing social interaction mechanisms to improve service quality.

Suggested Citation

  • Dennis J. Zhang & Gad Allon & Jan A. Van Mieghem, 2017. "Does Social Interaction Improve Learning Outcomes? Evidence from Field Experiments on Massive Open Online Courses," Manufacturing & Service Operations Management, INFORMS, vol. 19(3), pages 347-367, July.
  • Handle: RePEc:inm:ormsom:v:19:y:2017:i:3:p:347-367
    DOI: 10.1287/msom.2016.0615
    as

    Download full text from publisher

    File URL: https://doi.org/10.1287/msom.2016.0615
    Download Restriction: no

    File URL: https://libkey.io/10.1287/msom.2016.0615?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Bing Jing, 2011. "Social Learning and Dynamic Pricing of Durable Goods," Marketing Science, INFORMS, vol. 30(5), pages 851-865, September.
    2. Mary A. Burke & Tim R. Sass, 2013. "Classroom Peer Effects and Student Achievement," Journal of Labor Economics, University of Chicago Press, vol. 31(1), pages 51-82.
    3. Yiangos Papanastasiou & Nicos Savva, 2017. "Dynamic Pricing in the Presence of Social Learning and Strategic Consumers," Management Science, INFORMS, vol. 63(4), pages 919-939, April.
    4. Gad Allon & Achal Bassamboo & Itai Gurvich, 2011. "“We Will Be Right with You”: Managing Customer Expectations with Vague Promises and Cheap Talk," Operations Research, INFORMS, vol. 59(6), pages 1382-1394, December.
    5. Armin Falk & Andrea Ichino, 2006. "Clean Evidence on Peer Effects," Journal of Labor Economics, University of Chicago Press, vol. 24(1), pages 39-58, January.
    6. Duflo, Esther & Glennerster, Rachel & Kremer, Michael, 2008. "Using Randomization in Development Economics Research: A Toolkit," Handbook of Development Economics, in: T. Paul Schultz & John A. Strauss (ed.), Handbook of Development Economics, edition 1, volume 4, chapter 61, pages 3895-3962, Elsevier.
    7. Imbens, Guido W & Angrist, Joshua D, 1994. "Identification and Estimation of Local Average Treatment Effects," Econometrica, Econometric Society, vol. 62(2), pages 467-475, March.
    8. Zeynep Akşin & Barış Ata & Seyed Morteza Emadi & Che-Lin Su, 2013. "Structural Estimation of Callers' Delay Sensitivity in Call Centers," Management Science, INFORMS, vol. 59(12), pages 2727-2746, December.
    9. David J. Zimmerman, 2003. "Peer Effects in Academic Outcomes: Evidence from a Natural Experiment," The Review of Economics and Statistics, MIT Press, vol. 85(1), pages 9-23, February.
    10. Victor Lavy & Analia Schlosser, 2011. "Mechanisms and Impacts of Gender Peer Effects at School," American Economic Journal: Applied Economics, American Economic Association, vol. 3(2), pages 1-33, April.
    11. Scott A. Imberman & Adriana D. Kugler & Bruce I. Sacerdote, 2012. "Katrina's Children: Evidence on the Structure of Peer Effects from Hurricane Evacuees," American Economic Review, American Economic Association, vol. 102(5), pages 2048-2082, August.
    12. Qiuping Yu & Gad Allon & Achal Bassamboo, 2017. "How Do Delay Announcements Shape Customer Behavior? An Empirical Study," Management Science, INFORMS, vol. 63(1), pages 1-20, January.
    13. Epple, Dennis & Romano, Richard E, 1998. "Competition between Private and Public Schools, Vouchers, and Peer-Group Effects," American Economic Review, American Economic Association, vol. 88(1), pages 33-62, March.
    14. Antoni Calvó-Armengol & Eleonora Patacchini & Yves Zenou, 2009. "Peer Effects and Social Networks in Education," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 76(4), pages 1239-1267.
    15. Senthil K. Veeraraghavan & Laurens G. Debo, 2011. "Herding in Queues with Waiting Costs: Rationality and Regret," Manufacturing & Service Operations Management, INFORMS, vol. 13(3), pages 329-346, July.
    16. Ozan Candogan & Kostas Bimpikis & Asuman Ozdaglar, 2012. "Optimal Pricing in Networks with Externalities," Operations Research, INFORMS, vol. 60(4), pages 883-905, August.
    17. David S. Lyle, 2007. "Estimating and Interpreting Peer and Role Model Effects from Randomly Assigned Social Groups at West Point," The Review of Economics and Statistics, MIT Press, vol. 89(2), pages 289-299, May.
    18. Naor, P, 1969. "The Regulation of Queue Size by Levying Tolls," Econometrica, Econometric Society, vol. 37(1), pages 15-24, January.
    19. Bruce Sacerdote, 2001. "Peer Effects with Random Assignment: Results for Dartmouth Roommates," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 116(2), pages 681-704.
    20. Caroline Hoxby, 2000. "Peer Effects in the Classroom: Learning from Gender and Race Variation," NBER Working Papers 7867, National Bureau of Economic Research, Inc.
    21. Victor Lavy & Analía Schlosser, 2011. "Corrigendum: Mechanisms and Impacts of Gender Peer Effects at School," American Economic Journal: Applied Economics, American Economic Association, vol. 3(3), pages 268-268, July.
    22. Jane Cooley Fruehwirth, 2014. "Can Achievement Peer Effect Estimates Inform Policy? A View from Inside the Black Box," The Review of Economics and Statistics, MIT Press, vol. 96(3), pages 514-523, July.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Friess, Svenja & Rosendahl Huber, Laura, 2024. "Breaking The Ice: Can Initially Active Peers Improve Platform Engagement And Persistence?," VfS Annual Conference 2024 (Berlin): Upcoming Labor Market Challenges 302443, Verein für Socialpolitik / German Economic Association.
    2. Xiaoyan Chen & Wei Geng, 2022. "Enroll now, pay later: optimal pricing and nudge efforts for massive-online-open-courses providers," Electronic Markets, Springer;IIM University of St. Gallen, vol. 32(2), pages 1003-1018, June.
    3. Andrew M. Davis & Vishal Gaur & Dayoung Kim, 2021. "Consumer Learning from Own Experience and Social Information: An Experimental Study," Management Science, INFORMS, vol. 67(5), pages 2924-2943, May.
    4. Dennis J. Zhang & Hengchen Dai & Lingxiu Dong & Qian Wu & Lifan Guo & Xiaofei Liu, 2019. "The Value of Pop-Up Stores on Retailing Platforms: Evidence from a Field Experiment with Alibaba," Management Science, INFORMS, vol. 65(11), pages 5142-5151, November.
    5. Marshall Fisher & Marcelo Olivares & Bradley R. Staats, 2020. "Why Empirical Research Is Good for Operations Management, and What Is Good Empirical Operations Management?," Manufacturing & Service Operations Management, INFORMS, vol. 22(1), pages 170-178, January.
    6. Jingchuan Pu, 2022. "Do Online Communities Improve Job Performance in the Geographically Dispersed Organization?," Production and Operations Management, Production and Operations Management Society, vol. 31(2), pages 403-421, February.
    7. Pnina Feldman & Yiangos Papanastasiou & Ella Segev, 2019. "Social Learning and the Design of New Experience Goods," Management Science, INFORMS, vol. 65(5), pages 1502-1519, April.
    8. Dennis J. Zhang & Hengchen Dai & Lingxiu Dong & Fangfang Qi & Nannan Zhang & Xiaofei Liu & Zhongyi Liu & Jiang Yang, 2020. "The Long-term and Spillover Effects of Price Promotions on Retailing Platforms: Evidence from a Large Randomized Experiment on Alibaba," Management Science, INFORMS, vol. 66(6), pages 2589-2609, June.
    9. Gong, Jie & Liu, Tracy Xiao & Tang, Jie, 2021. "How monetary incentives improve outcomes in MOOCs: Evidence from a field experiment," Journal of Economic Behavior & Organization, Elsevier, vol. 190(C), pages 905-921.
    10. Christian Terwiesch & Marcelo Olivares & Bradley R. Staats & Vishal Gaur, 2020. "OM Forum—A Review of Empirical Operations Management over the Last Two Decades," Manufacturing & Service Operations Management, INFORMS, vol. 22(4), pages 656-668, July.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Gad Allon & Jan Van Mieghem & Dennis Zhang, 2016. "Does Social Interaction Improve Learning Outcomes? Field Evidence from Massive Open Online Education," Natural Field Experiments 00574, The Field Experiments Website.
    2. Silvia Mendolia & Alfredo R Paloyo & Ian Walker, 2018. "Heterogeneous effects of high school peers on educational outcomes," Oxford Economic Papers, Oxford University Press, vol. 70(3), pages 613-634.
    3. Wang, Haining & Cheng, Zhiming & Smyth, Russell, 2018. "Do migrant students affect local students’ academic achievements in urban China?," Economics of Education Review, Elsevier, vol. 63(C), pages 64-77.
    4. Alexandra de Gendre & Nicolás Salamanca, 2020. "On the Mechanisms of Ability Peer Effects," Melbourne Institute Working Paper Series wp2020n19, Melbourne Institute of Applied Economic and Social Research, The University of Melbourne.
    5. Michela Maria Tincani, 2017. "Heterogeneous Peer Effects and Rank Concerns: Theory and Evidence," CESifo Working Paper Series 6331, CESifo.
    6. Stephen Gibbons & Shqiponja Telhaj, 2016. "Peer Effects: Evidence from Secondary School Transition in England," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 78(4), pages 548-575, August.
    7. Ozkan Eren, 2016. "Differential Peer Effects, Student Achievement, and Student Absenteeism: Evidence from a Large Scale Randomized Experiment," Departmental Working Papers 2016-01, Department of Economics, Louisiana State University.
    8. Ozkan Eren, 2017. "Differential Peer Effects, Student Achievement, and Student Absenteeism: Evidence From a Large-Scale Randomized Experiment," Demography, Springer;Population Association of America (PAA), vol. 54(2), pages 745-773, April.
    9. Berlinski, Samuel & Busso, Matias & Giannola, Michele, 2023. "Helping struggling students and benefiting all: Peer effects in primary education," Journal of Public Economics, Elsevier, vol. 224(C).
    10. Michela Tincani, 2017. "Heterogeneous Peer Effects and Rank Concerns: Theory and Evidence," Working Papers 2017-006, Human Capital and Economic Opportunity Working Group.
    11. Zhao, Liqiu & Zhao, Zhong, 2021. "Disruptive Peers in the Classroom and Students’ Academic Outcomes: Evidence and Mechanisms," Labour Economics, Elsevier, vol. 68(C).
    12. Wennberg, Karl & Norgren, Axel, 2021. "Models of Peer Effects in Education," Working Papers 21/3, Stockholm School of Economics, Center for Educational Leadership and Excellence.
    13. Clifton-Sprigg, Joanna, 2015. "Educational spillovers and parental migration," Labour Economics, Elsevier, vol. 34(C), pages 64-75.
    14. Li, Tao & Han, Li & Zhang, Linxiu & Rozelle, Scott, 2014. "Encouraging classroom peer interactions: Evidence from Chinese migrant schools," Journal of Public Economics, Elsevier, vol. 111(C), pages 29-45.
    15. Zhang, Hongliang, 2016. "The role of testing noise in the estimation of achievement-based peer effects," Economics of Education Review, Elsevier, vol. 54(C), pages 113-123.
    16. Yi Cao & Tao Zhou & Jian Gao, 2024. "Heterogeneous peer effects of college roommates on academic performance," Nature Communications, Nature, vol. 15(1), pages 1-11, December.
    17. Dewan, Prerna & Ray, Tridip & Roy Chaudhuri, Arka & Tater, Kirti, 2024. "Gender peer effects in high schools: Evidence from India," Journal of Economic Behavior & Organization, Elsevier, vol. 220(C), pages 470-494.
    18. Pan, Zheng & Lien, Donald & Wang, Hao, 2022. "Peer effects and shadow education," Economic Modelling, Elsevier, vol. 111(C).
    19. Dimant, Eugen, 2015. "On Peer Effects: Behavioral Contagion of (Un)Ethical Behavior and the Role of Social Identity," MPRA Paper 68732, University Library of Munich, Germany.
    20. Auestad, May Linn, 2018. "The effect of low-achieving peers," Labour Economics, Elsevier, vol. 55(C), pages 178-214.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:ormsom:v:19:y:2017:i:3:p:347-367. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.