IDEAS home Printed from https://ideas.repec.org/p/pra/mprapa/103934.html
   My bibliography  Save this paper

Experimental Evidence on Artificial Intelligence in the Classroom

Author

Listed:
  • Ferman, Bruno
  • Lima, Lycia
  • Riva, Flavio

Abstract

This paper investigates how technologies that use different combinations of artificial and human intelligence are incorporated into classroom instruction, and how they ultimately affect students' outcomes. We conducted a field experiment to study two technologies that allow teachers to outsource grading and feedback tasks on writing practices. The first technology is a fully automated evaluation system that provides instantaneous scores and feedback. The second one uses human graders as an additional resource to enhance grading and feedback quality in aspects in which the automated system arguably falls short. Both technologies significantly improved students' essay scores, and the additional inputs from human graders did not improve effectiveness. Furthermore, the technologies similarly helped teachers engage more frequently on nonroutine tasks that supported the individualization of pedagogy. Our results are informative about the potential of artificial intelligence to expand the set of tasks that can be automated, and on how advances in artificial intelligence may relocate human labor to tasks that remain out of reach of automation.

Suggested Citation

  • Ferman, Bruno & Lima, Lycia & Riva, Flavio, 2020. "Experimental Evidence on Artificial Intelligence in the Classroom," MPRA Paper 103934, University Library of Munich, Germany.
    • Handle: RePEc:pra:mprapa:103934
    as

    Download full text from publisher

    File URL: https://mpra.ub.uni-muenchen.de/103934/1/MPRA_paper_103934.pdf
    File Function: original version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Anderson, Michael L., 2008. "Multiple Inference and Gender Differences in the Effects of Early Intervention: A Reevaluation of the Abecedarian, Perry Preschool, and Early Training Projects," Journal of the American Statistical Association, American Statistical Association, vol. 103(484), pages 1481-1495.
    2. Christopher J. Doss & Erin M. Fahle & Susanna Loeb & Benjamin N. York, 2018. "More than Just a Nudge: Supporting Kindergarten Parents with Differentiated and Personalized Text-Messages," NBER Working Papers 24450, National Bureau of Economic Research, Inc.
    3. Ferman, Bruno & Finamor, Lucas & Lima, Lycia, 2019. "Are Public Schools Ready to Integrate Math Classes with Khan Academy?," MPRA Paper 94736, University Library of Munich, Germany.
    4. George Bulman & Robert W. Fairlie, 2015. "Technology and Education: Computers, Software, and the Internet," CESifo Working Paper Series 5570, CESifo.
    5. Karthik Muralidharan & Abhijeet Singh & Alejandro J. Ganimian, 2019. "Disrupting Education? Experimental Evidence on Technology-Aided Instruction in India," American Economic Review, American Economic Association, vol. 109(4), pages 1426-1460, April.
    6. Esther Duflo & Pascaline Dupas & Michael Kremer, 2011. "Peer Effects, Teacher Incentives, and the Impact of Tracking: Evidence from a Randomized Evaluation in Kenya," American Economic Review, American Economic Association, vol. 101(5), pages 1739-1774, August.
    7. David H. Autor & Frank Levy & Richard J. Murnane, 2003. "The Skill Content of Recent Technological Change: An Empirical Exploration," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 118(4), pages 1279-1333.
    8. David S. Lee, 2009. "Training, Wages, and Sample Selection: Estimating Sharp Bounds on Treatment Effects," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 76(3), pages 1071-1102.
    9. Bulman, G. & Fairlie, R.W., 2016. "Technology and Education," Handbook of the Economics of Education,, Elsevier.
    10. Eric Bettinger & Robert Fairlie & Anastasia Kapuza & Elena Kardanova & Prashant Loyalka & Andrey Zakharov, 2023. "Diminishing Marginal Returns to Computer‐Assisted Learning," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 42(2), pages 552-570, March.
    11. Acemoglu, Daron & Autor, David, 2011. "Skills, Tasks and Technologies: Implications for Employment and Earnings," Handbook of Labor Economics, in: O. Ashenfelter & D. Card (ed.), Handbook of Labor Economics, edition 1, volume 4, chapter 12, pages 1043-1171, Elsevier.
    12. Ferman, Bruno & Ponczek, Vladimir, 2017. "Should we drop covariate cells with attrition problems?," MPRA Paper 80686, University Library of Munich, Germany.
    13. Benjamin N. York & Susanna Loeb, 2014. "One Step at a Time: The Effects of an Early Literacy Text Messaging Program for Parents of Preschoolers," NBER Working Papers 20659, National Bureau of Economic Research, Inc.
    14. Joel Mokyr & Chris Vickers & Nicolas L. Ziebarth, 2015. "The History of Technological Anxiety and the Future of Economic Growth: Is This Time Different?," Journal of Economic Perspectives, American Economic Association, vol. 29(3), pages 31-50, Summer.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ferman, Bruno & Lima, Lycia & Riva, Flávio, 2021. "Artificial Intelligence, Teacher Tasks and Individualized Pedagogy," SocArXiv qw249, Center for Open Science.
    2. Derksen, Laura & Michaud-Leclerc, Catherine & Souza, Pedro C.L., 2022. "Restricted access: How the internet can be used to promote reading and learning," Journal of Development Economics, Elsevier, vol. 155(C).
    3. Derksen, Laura & Leclerc, Catherine Michaud & Souza, Pedro CL, 2019. "Searching for Answers : The Impact of Student Access to Wikipedia," The Warwick Economics Research Paper Series (TWERPS) 1236, University of Warwick, Department of Economics.
    4. Falck, Oliver & Heimisch-Roecker, Alexandra & Wiederhold, Simon, 2021. "Returns to ICT skills," Research Policy, Elsevier, vol. 50(7).
    5. Naik, Gopal & Chitre, Chetan & Bhalla, Manaswini & Rajan, Jothsna, 2020. "Impact of use of technology on student learning outcomes: Evidence from a large-scale experiment in India," World Development, Elsevier, vol. 127(C).
    6. Díaz, Carlos & Dodel, Matías & Menese, Pablo, 2022. "Can one laptop per child reduce digital inequalities? ICT household access patterns under Plan Ceibal," Telecommunications Policy, Elsevier, vol. 46(9).
    7. Karthik Muralidharan & Abhijeet Singh & Alejandro J. Ganimian, 2019. "Disrupting Education? Experimental Evidence on Technology-Aided Instruction in India," American Economic Review, American Economic Association, vol. 109(4), pages 1426-1460, April.
    8. Derksen, Laura & Leclerc, Catherine Michaud & Souza, Pedro CL, 2019. "Searching for Answers: The Impact of Student Access to Wikipedia," CAGE Online Working Paper Series 450, Competitive Advantage in the Global Economy (CAGE).
    9. Perera, Marcelo & Aboal, Diego, 2019. "The impact of a mathematics computer-assisted learning platform on students' mathematics test scores," MERIT Working Papers 2019-007, United Nations University - Maastricht Economic and Social Research Institute on Innovation and Technology (MERIT).
    10. So Yoon Ahn & Youjin Hahn & Semee Yoon, 2021. "Can New Learning Opportunities Reshape Gender Attitudes for Girls?: Field Evidence from Tanzania," Working Papers 2021-046, Human Capital and Economic Opportunity Working Group.
    11. Ferman, Bruno & Finamor, Lucas & Lima, Lycia, 2019. "Are Public Schools Ready to Integrate Math Classes with Khan Academy?," MPRA Paper 94736, University Library of Munich, Germany.
    12. Islam, Asad & Kwon, Sungoh & Masood, Eema & Prakash, Nishith & Sabarwal, Shwetlena & Saraswat, Deepak, 2020. "When Goal-Setting Forges Ahead but Stops Short," GLO Discussion Paper Series 526, Global Labor Organization (GLO).
    13. Bianchi, Nicola & Lu, Yi & Song, Hong, 2022. "The effect of computer-assisted learning on students’ long-term development," Journal of Development Economics, Elsevier, vol. 158(C).
    14. Eric Bettinger & Robert Fairlie & Anastasia Kapuza & Elena Kardanova & Prashant Loyalka & Andrey Zakharov, 2023. "Diminishing Marginal Returns to Computer‐Assisted Learning," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 42(2), pages 552-570, March.
    15. Rosa Sanchis-Guarner & José Montalbán & Felix Weinhardt, 2021. "Home Broadband and Human Capital Formation," CESifo Working Paper Series 8846, CESifo.
    16. Briglauer, Wolfgang & Dürr, Niklas S. & Falck, Oliver & Hüschelrath, Kai, 2019. "Does state aid for broadband deployment in rural areas close the digital and economic divide?," Information Economics and Policy, Elsevier, vol. 46(C), pages 68-85.
    17. Johnston, Jamie & Ksoll, Christopher, 2022. "Effectiveness of interactive satellite-transmitted instruction: Experimental evidence from Ghanaian primary schools," Economics of Education Review, Elsevier, vol. 91(C).
    18. Per-Anders Edin & Peter Fredriksson & Martin Nybom & Björn Öckert, 2022. "The Rising Return to Noncognitive Skill," American Economic Journal: Applied Economics, American Economic Association, vol. 14(2), pages 78-100, April.
    19. Arntz, Melanie & Gregory, Terry & Zierahn, Ulrich, 2019. "Digitalization and the Future of Work: Macroeconomic Consequences," IZA Discussion Papers 12428, Institute of Labor Economics (IZA).
    20. Daron Acemoglu & Pascual Restrepo, 2020. "The wrong kind of AI? Artificial intelligence and the future of labour demand," Cambridge Journal of Regions, Economy and Society, Cambridge Political Economy Society, vol. 13(1), pages 25-35.

    More about this item

    Keywords

    artificial intelligence; technological change; automated writing evaluation; routine and nonroutine tasks;
    All these keywords.

    JEL classification:

    • I21 - Health, Education, and Welfare - - Education - - - Analysis of Education
    • I22 - Health, Education, and Welfare - - Education - - - Educational Finance; Financial Aid
    • I25 - Health, Education, and Welfare - - Education - - - Education and Economic Development

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:pra:mprapa:103934. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Joachim Winter (email available below). General contact details of provider: https://edirc.repec.org/data/vfmunde.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.