IDEAS home Printed from https://ideas.repec.org/a/eee/ecoedu/v68y2019icp89-103.html
   My bibliography  Save this article

Is the pen mightier than the keyboard? The effect of online testing on measured student achievement

Author

Listed:
  • Backes, Ben
  • Cowan, James

Abstract

Nearly two dozen states now administer online exams to deliver testing to K-12 students. These tests have real consequences: their results feed into accountability systems, which have been used for more than a decade to hold schools and districts accountable for their students’ learning. We examine the rollout of computer-based testing in Massachusetts over 2 years to investigate test mode effects. Crucial to the study design is the state administering the same exam (PARCC) in online and offline formats each year during the transitional period. We find an online test penalty of about 0.10 standard deviations in math and 0.25 standard deviations in English language arts (ELA), which partially but not fully fades out in the second year of online testing.

Suggested Citation

  • Backes, Ben & Cowan, James, 2019. "Is the pen mightier than the keyboard? The effect of online testing on measured student achievement," Economics of Education Review, Elsevier, vol. 68(C), pages 89-103.
    • Handle: RePEc:eee:ecoedu:v:68:y:2019:i:c:p:89-103
    DOI: 10.1016/j.econedurev.2018.12.007
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0272775718305119
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.econedurev.2018.12.007?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Richard K. Crump & V. Joseph Hotz & Guido W. Imbens & Oscar A. Mitnik, 2009. "Dealing with limited overlap in estimation of average treatment effects," Biometrika, Biometrika Trust, vol. 96(1), pages 187-199.
    2. Sergio Firpo, 2007. "Efficient Semiparametric Estimation of Quantile Treatment Effects," Econometrica, Econometric Society, vol. 75(1), pages 259-276, January.
    3. Elizabeth U. Cascio & Douglas O. Staiger, 2012. "Knowledge, Tests, and Fadeout in Educational Interventions," NBER Working Papers 18038, National Bureau of Economic Research, Inc.
    4. Black, Sandra E. & Machin, Stephen, 2011. "Housing Valuations of School Performance," Handbook of the Economics of Education, in: Erik Hanushek & Stephen Machin & Ludger Woessmann (ed.), Handbook of the Economics of Education, edition 1, volume 3, chapter 10, pages 485-519, Elsevier.
    5. Backes, Ben & Cowan, James & Goldhaber, Dan & Koedel, Cory & Miller, Luke C. & Xu, Zeyu, 2018. "The common core conundrum: To what extent should we worry that changes to assessments will affect test-based measures of teacher performance?," Economics of Education Review, Elsevier, vol. 62(C), pages 48-65.
    6. Eric A. Hanushek & Steven G. Rivkin, 2010. "Generalizations about Using Value-Added Measures of Teacher Quality," American Economic Review, American Economic Association, vol. 100(2), pages 267-271, May.
    7. Markus Frolich & Blaise Melly, 2010. "Estimation of quantile treatment effects with Stata," Stata Journal, StataCorp LP, vol. 10(3), pages 423-457, September.
    8. Kim, Jaehoon & Kim, Sangsin, 2015. "2012년 국회법 개정의 효과 연구 [A Study on the Effect of the 2012 National Assembly Act Amendment]," KDI Research Monographs, Korea Development Institute (KDI), volume 127, number v:2015-03(k):y:2015:p:1-1.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Donna Feir & Kelly Foley & Maggie E. C. Jones, 2021. "The Distributional Impacts of Active Labor Market Programs for Indigenous Populations," AEA Papers and Proceedings, American Economic Association, vol. 111, pages 216-220, May.
    2. Victor Chernozhukov & Iván Fernández‐Val & Blaise Melly, 2013. "Inference on Counterfactual Distributions," Econometrica, Econometric Society, vol. 81(6), pages 2205-2268, November.
    3. M. Fort, 2012. "Unconditional and Conditional Quantile Treatment Effect: Identification Strategies and Interpretations," Working Papers wp857, Dipartimento Scienze Economiche, Universita' di Bologna.
    4. Evangelista, Rui & Silva, João Andrade e & Ramalho, Esmeralda A., 2022. "How heterogeneous is the impact of energy efficiency on dwelling prices? Evidence from the application of the unconditional quantile hedonic model to the Portuguese residential market," Energy Economics, Elsevier, vol. 109(C).
    5. Laetitia Duval & François-Charles Wolff, 2013. "The consumption-enhancing effect of remittances: Evidence from Kosovo," wiiw Balkan Observatory Working Papers 107, The Vienna Institute for International Economic Studies, wiiw.
    6. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    7. Dambala Gelo & Steven F. Koch & Edwin Muchapondwah, 2013. "Do the Poor Benefit from Devolution Policies? Evidences from Quantile Treatment Effect Evaluation of Joint Forest Management," Working Papers 201388, University of Pretoria, Department of Economics.
    8. Rui Evangelista & João Andrade E Silva & Esmeralda A. Ramalho, 2021. "How heterogeneous is the impact of energy efficiency on dwelling prices? Evidence from the application of the unconditional quantile hedonic model to the Portuguese residential market," Working Papers REM 2021/0186, ISEG - Lisbon School of Economics and Management, REM, Universidade de Lisboa.
    9. Julie Berry Cullen & Cory Koedel & Eric Parsons, 2021. "The Compositional Effect of Rigorous Teacher Evaluation on Workforce Quality," Education Finance and Policy, MIT Press, vol. 16(1), pages 7-41, Winter.
    10. Francesca Caselli & Mr. Philippe Wingender, 2018. "Bunching at 3 Percent: The Maastricht Fiscal Criterion and Government Deficits," IMF Working Papers 2018/182, International Monetary Fund.
    11. Alex Coad & Julian Frankish & Paul Nightingale & Richard Roberts, 2014. "Business experience and start-up size: Buying more lottery tickets next time around?," Small Business Economics, Springer, vol. 43(3), pages 529-547, October.
    12. Zongwu Cai & Ying Fang & Ming Lin & Shengfang Tang, 2021. "A Nonparametric Test for Testing Heterogeneity in Conditional Quantile Treatment Effects," WORKING PAPERS SERIES IN THEORETICAL AND APPLIED ECONOMICS 202117, University of Kansas, Department of Economics, revised Aug 2021.
    13. Daly, Moira, 2015. "The long term returns of attempting self-employment with regular employment as a fall back option," Labour Economics, Elsevier, vol. 35(C), pages 26-52.
    14. Swaibu, Mbowa & Tonny, Odokonyero & Ezra, Munyambonera, 2014. "The potential of coffee to uplift people out of poverty in Northern Uganda," Research Reports 206171, Economic Policy Research Centre (EPRC).
    15. Damian Clarke & Manuel Llorca Jaña & Daniel Pailañir, 2023. "The use of quantile methods in economic history," Historical Methods: A Journal of Quantitative and Interdisciplinary History, Taylor & Francis Journals, vol. 56(2), pages 115-132, April.
    16. Samuel Bazzi & Sudarno Sumarto & Asep Suryahadi, "undated". "It’s All in the Timing: Household Expenditure and Labor Supply Responses to Unconditional Cash Transfers," Working Papers 280, Communications Section.
    17. Schiltz, Fritz & Sestito, Paolo & Agasisti, Tommaso & De Witte, Kristof, 2018. "The added value of more accurate predictions for school rankings," Economics of Education Review, Elsevier, vol. 67(C), pages 207-215.
    18. Borgen, Nicolai T. & Haupt, Andreas & Wiborg, Øyvind N., 2021. "Flexible and fast estimation of quantile treatment effects: The rqr and rqrplot commands," SocArXiv 4vquh, Center for Open Science.
    19. Schwerdt, Guido & West, Martin R. & Winters, Marcus A., 2017. "The effects of test-based retention on student outcomes over time: Regression discontinuity evidence from Florida," Journal of Public Economics, Elsevier, vol. 152(C), pages 154-169.
    20. Simone Balestra & Uschi Backes-Gellner, 2014. "Heterogeneous effects of pupil-to-teacher ratio policies - A look at class size reduction and teacher aide," Economics of Education Working Paper Series 0102, University of Zurich, Department of Business Administration (IBW), revised Apr 2017.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:ecoedu:v:68:y:2019:i:c:p:89-103. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/econedurev .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.