IDEAS home Printed from https://ideas.repec.org/a/eee/ecoedu/v68y2019icp89-103.html
   My bibliography  Save this article

Is the pen mightier than the keyboard? The effect of online testing on measured student achievement

Author

Listed:
  • Backes, Ben
  • Cowan, James

Abstract

Nearly two dozen states now administer online exams to deliver testing to K-12 students. These tests have real consequences: their results feed into accountability systems, which have been used for more than a decade to hold schools and districts accountable for their students’ learning. We examine the rollout of computer-based testing in Massachusetts over 2 years to investigate test mode effects. Crucial to the study design is the state administering the same exam (PARCC) in online and offline formats each year during the transitional period. We find an online test penalty of about 0.10 standard deviations in math and 0.25 standard deviations in English language arts (ELA), which partially but not fully fades out in the second year of online testing.

Suggested Citation

  • Backes, Ben & Cowan, James, 2019. "Is the pen mightier than the keyboard? The effect of online testing on measured student achievement," Economics of Education Review, Elsevier, vol. 68(C), pages 89-103.
    • Handle: RePEc:eee:ecoedu:v:68:y:2019:i:c:p:89-103
    DOI: 10.1016/j.econedurev.2018.12.007
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0272775718305119
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.econedurev.2018.12.007?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Richard K. Crump & V. Joseph Hotz & Guido W. Imbens & Oscar A. Mitnik, 2009. "Dealing with limited overlap in estimation of average treatment effects," Biometrika, Biometrika Trust, vol. 96(1), pages 187-199.
    2. Sergio Firpo, 2007. "Efficient Semiparametric Estimation of Quantile Treatment Effects," Econometrica, Econometric Society, vol. 75(1), pages 259-276, January.
    3. Elizabeth U. Cascio & Douglas O. Staiger, 2012. "Knowledge, Tests, and Fadeout in Educational Interventions," NBER Working Papers 18038, National Bureau of Economic Research, Inc.
    4. Black, Sandra E. & Machin, Stephen, 2011. "Housing Valuations of School Performance," Handbook of the Economics of Education, in: Erik Hanushek & Stephen Machin & Ludger Woessmann (ed.), Handbook of the Economics of Education, edition 1, volume 3, chapter 10, pages 485-519, Elsevier.
    5. Backes, Ben & Cowan, James & Goldhaber, Dan & Koedel, Cory & Miller, Luke C. & Xu, Zeyu, 2018. "The common core conundrum: To what extent should we worry that changes to assessments will affect test-based measures of teacher performance?," Economics of Education Review, Elsevier, vol. 62(C), pages 48-65.
    6. Eric A. Hanushek & Steven G. Rivkin, 2010. "Generalizations about Using Value-Added Measures of Teacher Quality," American Economic Review, American Economic Association, vol. 100(2), pages 267-271, May.
    7. Markus Frolich & Blaise Melly, 2010. "Estimation of quantile treatment effects with Stata," Stata Journal, StataCorp LP, vol. 10(3), pages 423-457, September.
    8. Kim, Jaehoon & Kim, Sangsin, 2015. "2012년 국회법 개정의 효과 연구 [A Study on the Effect of the 2012 National Assembly Act Amendment]," KDI Research Monographs, Korea Development Institute (KDI), volume 127, number v:2015-03(k):y:2015:p:1-1.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Donna Feir & Kelly Foley & Maggie E. C. Jones, 2021. "The Distributional Impacts of Active Labor Market Programs for Indigenous Populations," AEA Papers and Proceedings, American Economic Association, vol. 111, pages 216-220, May.
    2. Victor Chernozhukov & Iván Fernández‐Val & Blaise Melly, 2013. "Inference on Counterfactual Distributions," Econometrica, Econometric Society, vol. 81(6), pages 2205-2268, November.
    3. Daniel Burkhard & Christian P. R. Schmid & Kaspar Wüthrich, 2019. "Financial incentives and physician prescription behavior: Evidence from dispensing regulations," Health Economics, John Wiley & Sons, Ltd., vol. 28(9), pages 1114-1129, September.
    4. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    5. Schiele, Valentin & Schmitz, Hendrik, 2016. "Quantile treatment effects of job loss on health," Journal of Health Economics, Elsevier, vol. 49(C), pages 59-69.
    6. George Adu & Franklin Amuakwa-Mensah & George Marbuah & Justice Tei Mensah, 2016. "Effect of gold mining on income distribution in Ghana," Working Papers 2016.23, FAERE - French Association of Environmental and Resource Economists.
    7. M. Fort, 2012. "Unconditional and Conditional Quantile Treatment Effect: Identification Strategies and Interpretations," Working Papers wp857, Dipartimento Scienze Economiche, Universita' di Bologna.
    8. Evangelista, Rui & Silva, João Andrade e & Ramalho, Esmeralda A., 2022. "How heterogeneous is the impact of energy efficiency on dwelling prices? Evidence from the application of the unconditional quantile hedonic model to the Portuguese residential market," Energy Economics, Elsevier, vol. 109(C).
    9. Laetitia Duval & François-Charles Wolff, 2013. "The consumption-enhancing effect of remittances: Evidence from Kosovo," wiiw Balkan Observatory Working Papers 107, The Vienna Institute for International Economic Studies, wiiw.
    10. Damian Clarke & Manuel Llorca Jaña & Daniel Pailañir, 2023. "The use of quantile methods in economic history," Historical Methods: A Journal of Quantitative and Interdisciplinary History, Taylor & Francis Journals, vol. 56(2), pages 115-132, April.
    11. Dambala Gelo & Steven F. Koch & Edwin Muchapondwah, 2013. "Do the Poor Benefit from Devolution Policies? Evidences from Quantile Treatment Effect Evaluation of Joint Forest Management," Working Papers 201388, University of Pretoria, Department of Economics.
    12. Oketch, Moses & Rolleston, Caine & Rossiter, Jack, 2021. "Diagnosing the learning crisis: What can value-added analysis contribute?," International Journal of Educational Development, Elsevier, vol. 87(C).
    13. Gustavo A. Crespi & Alessandro Maffioli & Pierre Mohnen & Gonzalo Vázquez, 2011. "Evaluating the Impact of Science, Technology and Innovation Programs: a Methodological Toolkit," SPD Working Papers 1104, Inter-American Development Bank, Office of Strategic Planning and Development Effectiveness (SPD).
    14. Rui Evangelista & João Andrade E Silva & Esmeralda A. Ramalho, 2021. "How heterogeneous is the impact of energy efficiency on dwelling prices? Evidence from the application of the unconditional quantile hedonic model to the Portuguese residential market," Working Papers REM 2021/0186, ISEG - Lisbon School of Economics and Management, REM, Universidade de Lisboa.
    15. Sean Corcoran & Dan Goldhaber, 2013. "Value Added and Its Uses: Where You Stand Depends on Where You Sit," Education Finance and Policy, MIT Press, vol. 8(3), pages 418-434, July.
    16. Liao, Wen-Chi & Zhao, Daxuan, 2019. "The selection and quantile treatment effects on the economic returns of green buildings," Regional Science and Urban Economics, Elsevier, vol. 74(C), pages 38-48.
    17. Julie Berry Cullen & Cory Koedel & Eric Parsons, 2021. "The Compositional Effect of Rigorous Teacher Evaluation on Workforce Quality," Education Finance and Policy, MIT Press, vol. 16(1), pages 7-41, Winter.
    18. Francesca Caselli & Mr. Philippe Wingender, 2018. "Bunching at 3 Percent: The Maastricht Fiscal Criterion and Government Deficits," IMF Working Papers 2018/182, International Monetary Fund.
    19. Yuta Kikuchi & Ryo Nakajima, 2016. "Evaluating Professor Value-added: Evidence from Professor and Student Matching in Physics," Keio-IES Discussion Paper Series 2016-002, Institute for Economics Studies, Keio University.
    20. Alex Coad & Julian Frankish & Paul Nightingale & Richard Roberts, 2014. "Business experience and start-up size: Buying more lottery tickets next time around?," Small Business Economics, Springer, vol. 43(3), pages 529-547, October.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:ecoedu:v:68:y:2019:i:c:p:89-103. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/econedurev .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.