IDEAS home Printed from https://ideas.repec.org/a/spr/reihed/v62y2021i3d10.1007_s11162-021-09633-z.html
   My bibliography  Save this article

Are Artificially Intelligent Conversational Chatbots Uniformly Effective in Reducing Summer Melt? Evidence from a Randomized Controlled Trial

Author

Listed:
  • Aizat Nurshatayeva

    (University of Pittsburgh)

  • Lindsay C. Page

    (University of Pittsburgh)

  • Carol C. White

    (Mississippi State University)

  • Hunter Gehlbach

    (Johns Hopkins University)

Abstract

Our field experiment extends prior work on college matriculation by testing the extent to which an artificially intelligent (AI) chatbot’s outreach and support to college students (N = 4442) reduced summer melt and improved first-year college enrollment at a 4-year university. Specifically, we investigate which students the intervention proves most effective for. We find that the AI chatbot increased overall success with navigating financial aid processes, such that student take up of educational loans increased by four percentage points. This financial aid effect was concentrated among would-be first-generation college goers, for whom loan acceptances increased by eight percentage points. In addition, the outreach increased first-generation students’ success with course registration and fall semester enrollment each by three percentage points. Our findings suggest that proactive chatbot outreach to students is likely to be most successful in reducing summer melt among those who may need the chatbot support the most.

Suggested Citation

  • Aizat Nurshatayeva & Lindsay C. Page & Carol C. White & Hunter Gehlbach, 2021. "Are Artificially Intelligent Conversational Chatbots Uniformly Effective in Reducing Summer Melt? Evidence from a Randomized Controlled Trial," Research in Higher Education, Springer;Association for Institutional Research, vol. 62(3), pages 392-402, May.
    • Handle: RePEc:spr:reihed:v:62:y:2021:i:3:d:10.1007_s11162-021-09633-z
    DOI: 10.1007/s11162-021-09633-z
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11162-021-09633-z
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11162-021-09633-z?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Bird, Kelli A. & Castleman, Benjamin L. & Denning, Jeffrey T. & Goodman, Joshua & Lamberton, Cait & Rosinger, Kelly Ochs, 2021. "Nudging at scale: Experimental evidence from FAFSA completion campaigns," Journal of Economic Behavior & Organization, Elsevier, vol. 183(C), pages 105-128.
    2. Castleman, Benjamin L. & Page, Lindsay C., 2015. "Summer nudging: Can personalized text messages and peer mentor outreach increase college going among low-income high school graduates?," Journal of Economic Behavior & Organization, Elsevier, vol. 115(C), pages 144-160.
    3. Benjamin M. Marx & Lesley J. Turner, 2019. "Student Loan Nudges: Experimental Evidence on Borrowing and Educational Attainment," American Economic Journal: Economic Policy, American Economic Association, vol. 11(2), pages 108-141, May.
    4. Jeffrey T. Denning & Todd R. Jones, 2021. "Maxed Out?: The Effect of Larger Student Loan Limits on Borrowing and Education Outcomes," Journal of Human Resources, University of Wisconsin Press, vol. 56(4), pages 1113-1140.
    5. Oreopoulos, Philip & Petronijevic, Uros, 2019. "The Remarkable Unresponsiveness of College Students to Nudging and What We Can Learn from It," IZA Discussion Papers 12460, Institute of Labor Economics (IZA).
    6. Castleman, Benjamin L. & Owen, Laura & Page, Lindsay C., 2015. "Stay late or start early? Experimental evidence on the benefits of college matriculation support from high schools versus colleges," Economics of Education Review, Elsevier, vol. 47(C), pages 168-179.
    7. Benjamin L. Castleman & Lindsay C. Page, 2014. "A Trickle or a Torrent? Understanding the Extent of Summer “Melt” Among College-Intending High School Graduates," Social Science Quarterly, Southwestern Social Science Association, vol. 95(1), pages 202-220, March.
    8. Benjamin L. Castleman & Lindsay C. Page, 2016. "Freshman Year Financial Aid Nudges: An Experiment to Increase FAFSA Renewal and College Persistence," Journal of Human Resources, University of Wisconsin Press, vol. 51(2), pages 389-415.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Alina Martinez & Tamara Linkow & Hannah Miller & Amanda Parsad & Cristofer Price, "undated". "Study of College Transition Messaging in GEAR UP: Impacts on Enrolling and Staying in College," Mathematica Policy Research Reports c6113480c8134325ac685ddd9, Mathematica Policy Research.
    2. Oded Gurantz & Jessica Howell & Michael Hurwitz & Cassandra Larson & Matea Pender & Brooke White, 2021. "A National‐Level Informational Experiment to Promote Enrollment in Selective Colleges," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 40(2), pages 453-479, March.
    3. Bird, Kelli A. & Castleman, Benjamin L. & Denning, Jeffrey T. & Goodman, Joshua & Lamberton, Cait & Rosinger, Kelly Ochs, 2021. "Nudging at scale: Experimental evidence from FAFSA completion campaigns," Journal of Economic Behavior & Organization, Elsevier, vol. 183(C), pages 105-128.
    4. Avery, Christopher & Castleman, Benjamin L. & Hurwitz, Michael & Long, Bridget Terry & Page, Lindsay C., 2021. "Digital messaging to improve college enrollment and success," Economics of Education Review, Elsevier, vol. 84(C).
    5. Lindsay C. Page & Judith Scott-Clayton, 2015. "Improving College Access in the United States: Barriers and Policy Responses," NBER Working Papers 21781, National Bureau of Economic Research, Inc.
    6. Christa Deneault, 2023. "College Enrollment and Mandatory FAFSA Applications: Evidence from Louisiana," American Economic Journal: Economic Policy, American Economic Association, vol. 15(3), pages 465-494, August.
    7. Oreopoulos, Philip, 2020. "Promises and Limitations of Nudging in Education," IZA Discussion Papers 13718, Institute of Labor Economics (IZA).
    8. Justin C. Ortagus & Melvin J. Tanner & Isaac McFarlin Jr., 2020. "Can Re-Enrollment Campaigns Help Dropouts Return to College? Evidence from Florida Community Colleges," NBER Working Papers 26649, National Bureau of Economic Research, Inc.
    9. Eric P. Bettinger & Benjamin L. Castleman & Alice Choe & Zachary Mabel, 2022. "Finishing the Last Lap: Experimental Evidence on Strategies to Increase Attainment for Students Near College Completion," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 41(4), pages 1040-1059, September.
    10. Kelly Ochs Rosinger, 2019. "Can Simplifying Financial Aid Offers Impact College Enrollment and Borrowing? Experimental and Quasi-Experimental Evidence," Education Finance and Policy, MIT Press, vol. 14(4), pages 601-626, Fall.
    11. Benjamin L. Castleman & Francis X. Murphy & Richard W. Patterson & William L. Skimmyhorn, 2021. "Nudges Don't Work When the Benefits Are Ambiguous: Evidence from a High‐Stakes Education Program," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 40(4), pages 1230-1248, September.
    12. Kramer, Dennis A. & Lamb, Christina & Page, Lindsay C., 2021. "The effects of default choice on student loan borrowing: Experimental evidence from a public research university," Journal of Economic Behavior & Organization, Elsevier, vol. 189(C), pages 470-489.
    13. Rodríguez-Planas, Núria & Secor, Alan & De Balanzó Joue, Rafael, 2023. "Resilience-Thinking Training for College Students: Evidence from a Randomized Trial," IZA Discussion Papers 16627, Institute of Labor Economics (IZA).
    14. Bergman, Peter & Denning, Jeffrey T. & Manoli, Dayanand, 2017. "Broken Tax Breaks? Evidence from a Tax Credit Information Experiment with 1,000,000 Students," IZA Discussion Papers 10997, Institute of Labor Economics (IZA).
    15. Todd R. Jones & Daniel Kreisman & Ross Rubenstein & Cynthia Searcy & Rachana Bhatt, 2022. "The Effects of Financial Aid Loss on Persistence and Graduation: A Multi-Dimensional Regression Discontinuity Approach," Education Finance and Policy, MIT Press, vol. 17(2), pages 206-231, Spring.
    16. Marx, Benjamin M. & Turner, Lesley J., 2020. "Paralysis by analysis? Effects of information on student loan take-up," Economics of Education Review, Elsevier, vol. 77(C).
    17. French, Robert & Oreopoulos, Philip, 2017. "Behavioral barriers transitioning to college," Labour Economics, Elsevier, vol. 47(C), pages 48-63.
    18. Adam M. Lavecchia & Heidi Liu & Philip Oreopoulos, 2014. "Behavioral Economics of Education: Progress and Possibilities," NBER Working Papers 20609, National Bureau of Economic Research, Inc.
    19. Scott Carrell & Bruce Sacerdote, 2017. "Why Do College-Going Interventions Work?," American Economic Journal: Applied Economics, American Economic Association, vol. 9(3), pages 124-151, July.
    20. Page, Lindsay C. & Scott-Clayton, Judith, 2016. "Improving college access in the United States: Barriers and policy responses," Economics of Education Review, Elsevier, vol. 51(C), pages 4-22.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:reihed:v:62:y:2021:i:3:d:10.1007_s11162-021-09633-z. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.