IDEAS home Printed from https://ideas.repec.org/a/eee/deveco/v170y2024ics0304387824000567.html
   My bibliography  Save this article

Using technology to prevent fraud in high stakes national school examinations: Evidence from Indonesia

Author

Listed:
  • Berkhout, Emilie
  • Pradhan, Menno
  • Rahmawati,
  • Suryadarma, Daniel
  • Swarnata, Arya

Abstract

Cheating reduces the signaling value of examinations. It also shifts the focus of teachers and students away from learning. Combating widespread cheating is difficult as students, teachers, and bureaucrats all benefit from high reported grades. We evaluate the impact of computer-based testing (CBT), an at-scale policy implemented by the Indonesian government to reduce widespread cheating in the national examinations. Exploiting the phased roll-out of the program from 2015 to 2019, we find that test scores declined dramatically, by 0.5 standard deviations, after the introduction of CBT. Schools with response patterns that indicated cheating prior to CBT adoption experienced a steeper decline. The effect is similar between schools with and without access to a computer lab, indicating that the reduction in the opportunity to cheat is the main reason for the test score decline. In districts with high adoption of CBT, schools that still used paper-based exams cheated less and scored lower, indicating spillovers of CBT. The results highlight the potential role of technology in improving the effectiveness in efforts to overcome collusive behavior in the education sector.

Suggested Citation

  • Berkhout, Emilie & Pradhan, Menno & Rahmawati, & Suryadarma, Daniel & Swarnata, Arya, 2024. "Using technology to prevent fraud in high stakes national school examinations: Evidence from Indonesia," Journal of Development Economics, Elsevier, vol. 170(C).
  • Handle: RePEc:eee:deveco:v:170:y:2024:i:c:s0304387824000567
    DOI: 10.1016/j.jdeveco.2024.103307
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0304387824000567
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.jdeveco.2024.103307?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Roberto Galbiati & Emeric Henry & Nicolas Jacquemet & Max Lobeck, 2021. "How laws affect the perception of norms: Empirical evidence from the lockdown," PLOS ONE, Public Library of Science, vol. 16(9), pages 1-14, September.
    2. Clément de Chaisemartin & Xavier D'Haultfœuille, 2020. "Two-Way Fixed Effects Estimators with Heterogeneous Treatment Effects," American Economic Review, American Economic Association, vol. 110(9), pages 2964-2996, September.
    3. Derek Neal, 2013. "The Consequences of Using One Assessment System To Pursue Two Objectives," NBER Working Papers 19214, National Bureau of Economic Research, Inc.
    4. Brian A. Jacob & Steven D. Levitt, 2003. "Rotten Apples: An Investigation of the Prevalence and Predictors of Teacher Cheating," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 118(3), pages 843-877.
    5. Battistin, Erich & De Nadai, Michele & Vuri, Daniela, 2017. "Counting rotten apples: Student achievement and score manipulation in Italian elementary Schools," Journal of Econometrics, Elsevier, vol. 200(2), pages 344-362.
    6. Oana Borcan & Mikael Lindahl & Andreea Mitrut, 2017. "Fighting Corruption in Education: What Works and Who Benefits?," American Economic Journal: Economic Policy, American Economic Association, vol. 9(1), pages 180-209, February.
    7. Goodman-Bacon, Andrew, 2021. "Difference-in-differences with variation in treatment timing," Journal of Econometrics, Elsevier, vol. 225(2), pages 254-277.
    8. Karthik Muralidharan & Paul Niehaus, 2017. "Experimentation at Scale," Journal of Economic Perspectives, American Economic Association, vol. 31(4), pages 103-124, Fall.
    9. Noam Angrist & David K. Evans & Deon Filmer & Rachel Glennerster & F. Halsey Rogers & Shwetlena Sabarwal, 2020. "How to Improve Education Outcomes Most Efficiently? A Comparison of 150 Interventions Using the New Learning-Adjusted Years of Schooling Metric," Working Papers 558, Center for Global Development.
    10. César Martinelli & Susan W. Parker & Ana Cristina Pérez-Gea & Rodimiro Rodrigo, 2018. "Cheating and Incentives: Learning from a Policy Experiment," American Economic Journal: Economic Policy, American Economic Association, vol. 10(1), pages 298-325, February.
    11. Claudio Lucifora & Marco Tonello, 2020. "Monitoring and Sanctioning Cheating at School: What Works? Evidence from a National Evaluation Program," Journal of Human Capital, University of Chicago Press, vol. 14(4), pages 584-616.
    12. Sun, Liyang & Abraham, Sarah, 2021. "Estimating dynamic treatment effects in event studies with heterogeneous treatment effects," Journal of Econometrics, Elsevier, vol. 225(2), pages 175-199.
    13. Jacob, Brian A., 2005. "Accountability, incentives and behavior: the impact of high-stakes testing in the Chicago Public Schools," Journal of Public Economics, Elsevier, vol. 89(5-6), pages 761-796, June.
    14. Thomas S. Dee & Will Dobbie & Brian A. Jacob & Jonah Rockoff, 2019. "The Causes and Consequences of Test Score Manipulation: Evidence from the New York Regents Examinations," American Economic Journal: Applied Economics, American Economic Association, vol. 11(3), pages 382-423, July.
    15. Bertoni, Marco & Brunello, Giorgio & Rocco, Lorenzo, 2013. "When the cat is near, the mice won't play: The effect of external examiners in Italian schools," Journal of Public Economics, Elsevier, vol. 104(C), pages 65-77.
    16. Joshua Angrist & Victor Lavy, 2009. "The Effects of High Stakes High School Achievement Awards: Evidence from a Randomized Trial," American Economic Review, American Economic Association, vol. 99(4), pages 1384-1414, September.
    17. Joshua D. Angrist & Erich Battistin & Daniela Vuri, 2017. "In a Small Moment: Class Size and Moral Hazard in the Italian Mezzogiorno," American Economic Journal: Applied Economics, American Economic Association, vol. 9(4), pages 216-249, October.
    18. Daniel Rodriguez-Segura, 2022. "EdTech in Developing Countries: A Review of the Evidence [From Proof of Concept to Scalable Policies: Challenges and Solutions, with an Application]," The World Bank Research Observer, World Bank, vol. 37(2), pages 171-203.
    19. Callaway, Brantly & Sant’Anna, Pedro H.C., 2021. "Difference-in-Differences with multiple time periods," Journal of Econometrics, Elsevier, vol. 225(2), pages 200-230.
    20. Derek Neal, 2013. "The Consequences of Using one Assessment System to Pursue two Objectives," The Journal of Economic Education, Taylor & Francis Journals, vol. 44(4), pages 339-352, October.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Bertoni, Marco & Brunello, Giorgio & De Benedetto, Marco Alberto & De Paola, Maria, 2019. "External Monitors and Score Manipulation in Italian Schools: Symptomatic Treatment or Cure?," IZA Discussion Papers 12591, Institute of Labor Economics (IZA).
    2. Sergio Longobardi & Patrizia Falzetti & Margherita Maria Pagliuca, 2018. "Quis custiodet ipsos custodes? How to detect and correct teacher cheating in Italian student data," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 27(3), pages 515-543, August.
    3. Joshua D. Angrist & Erich Battistin & Daniela Vuri, 2017. "In a Small Moment: Class Size and Moral Hazard in the Italian Mezzogiorno," American Economic Journal: Applied Economics, American Economic Association, vol. 9(4), pages 216-249, October.
    4. Leschnig, Lisa & Schwerdt, Guido & Zigova, Katarina, 2022. "Central exams and adult skills: Evidence from PIAAC," Economics of Education Review, Elsevier, vol. 90(C).
    5. Cuesta, José Ignacio & González, Felipe & Larroulet Philippi, Cristian, 2020. "Distorted quality signals in school markets," Journal of Development Economics, Elsevier, vol. 147(C).
    6. Erich Battistin, 2016. "How manipulating test scores affects school accountability and student achievement," IZA World of Labor, Institute of Labor Economics (IZA), pages 295-295, September.
    7. repec:ags:aaea22:335793 is not listed on IDEAS
    8. Morton, Emily & Thompson, Paul N. & Kuhfeld, Megan, 2024. "A multi-state, student-level analysis of the effects of the four-day school week on student achievement and growth," Economics of Education Review, Elsevier, vol. 100(C).
    9. Bas Scheer & Wiljan van den Berge & Maarten Goos & Alan Manning & Anna Salomons, 2022. "Alternative Work Arrangements and Worker Outcomes: Evidence from Payrolling," CPB Discussion Paper 435, CPB Netherlands Bureau for Economic Policy Analysis.
    10. Luo, Lianfa & Cheng, Zhiming & Ye, Qingqing & Cheng, Yanjun & Smyth, Russell & Yang, Zhiqing & Zhang, Le, 2024. "Nonmonetary awards and innovation: Evidence from winning China's Top Brand Contest," China Economic Review, Elsevier, vol. 86(C).
    11. Jaraitė, Jūratė & Kurtyka, Oliwia & Ollivier, Hélène, 2022. "Take a ride on the (not so) green side: How do CDM projects affect Indian manufacturing firms’ environmental performance?," Journal of Environmental Economics and Management, Elsevier, vol. 114(C).
    12. Simon Freyaldenhoven & Christian Hansen & Jorge Perez Perez & Jesse Shapiro, 2021. "Visualization, Identification, and stimation in the Linear Panel Event-Study Design," Working Papers 21-44, Federal Reserve Bank of Philadelphia.
    13. Mikhail Mamonov & Anna Pestova & Steven Ongena, 2023. "“Crime and Punishment”? How Banks Anticipate and Propagate Global Financial Sanctions," CERGE-EI Working Papers wp753, The Center for Economic Research and Graduate Education - Economics Institute, Prague.
    14. Arenas-Arroyo, Esther & Schmidpeter, Bernhard, 2022. "Spillover effects of immigration policies on children's human capital," Ruhr Economic Papers 974, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
    15. Kayaoglu, Aysegul, 2022. "Do refugees cause crime?," World Development, Elsevier, vol. 154(C).
    16. Simon Bensnes & Ingrid Huitfeldt & Edwin Leuven, 2023. "Reconciling estimates of the long-term earnings effect of fertility," Discussion Papers 1004, Statistics Norway, Research Department.
    17. Freise, Diana & Schmitz, Hendrik & Westphal, Matthias, 2022. "Late-career unemployment and cognitive abilities," Journal of Health Economics, Elsevier, vol. 86(C).
    18. Alexander Karaivanov & Dongwoo Kim & Shih En Lu & Hitoshi Shigeoka, 2022. "COVID-19 vaccination mandates and vaccine uptake," Nature Human Behaviour, Nature, vol. 6(12), pages 1615-1624, December.
    19. Davidson, Carl & Heyman, Fredrik & Matusz, Steven & Sjöholm, Fredrik & Chun Zhu, Susan, 2022. "How International Experience Helps Shape Labor Market Outcomes," Working Paper Series 1453, Research Institute of Industrial Economics, revised 09 Jun 2024.
    20. Cornelisz, Ilja & Meeter, Martijn & van Klaveren, Chris, 2019. "Educational equity and teacher discretion effects in high stake exams," Economics of Education Review, Elsevier, vol. 73(C).
    21. Francesconi, Marco & Sonedda, Daniela, 2024. "Does Weaker Employment Protection Lower the Cost of Job Loss?," IZA Discussion Papers 17374, Institute of Labor Economics (IZA).

    More about this item

    Keywords

    Education; National examinations; Cheating; Education technology; Indonesia;
    All these keywords.

    JEL classification:

    • H52 - Public Economics - - National Government Expenditures and Related Policies - - - Government Expenditures and Education
    • I21 - Health, Education, and Welfare - - Education - - - Analysis of Education
    • I28 - Health, Education, and Welfare - - Education - - - Government Policy

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:deveco:v:170:y:2024:i:c:s0304387824000567. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/devec .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.