IDEAS home Printed from https://ideas.repec.org/a/sae/evarev/v39y2015i5p501-524.html
   My bibliography  Save this article

Failed Warnings

Author

Listed:
  • Brian G. Moss
  • William H. Yeaton

Abstract

Background: Academic probation is a nearly universal but underresearched policy practiced at most postsecondary institutions. Objectives: To evaluate the impact of probation warning letters on students’ academic performance. Research design: Employing the inferentially strong regression discontinuity design, we evaluated the impact of two versions of warning letters (U.S. mail and email), noting their impact on next semester grades. Subjects: Probation and nonprobation students at a large, Midwestern college enrolled during two, successive fall–winter semester pairs ( n > 17,000, for each pair). Measures: Fall and winter grade point averages (GPAs) were identified for each individual student in the study sample. Results: Using both parametric and nonparametric analyses, we found that neither delivery method, paper or electronic, had a consistent, significant impact on subsequent GPA or odds of a GPA ≥ 2.0 during the next semester. Four of the eight measures of effect for GPA were small and positive (0.02–0.15; one significant positive outcome), and four were negative (−0.02 to −0.08; one significant negative result). A separate set of analyses that excluded students who took a single course led to further inconsistency in results. Supplementary analyses that excluded students scoring far from the cut-point yielded results consistent with a no-difference conclusion. Our findings also indicated that, after being placed on probation, only a small percentage of students were able to avoid movement to the next stage of academic sanction. Conclusions: Warning letters notifying students of probation status, lacking a staff-focused intervention, had little impact on academic performance.

Suggested Citation

  • Brian G. Moss & William H. Yeaton, 2015. "Failed Warnings," Evaluation Review, , vol. 39(5), pages 501-524, October.
  • Handle: RePEc:sae:evarev:v:39:y:2015:i:5:p:501-524
    DOI: 10.1177/0193841X15610192
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/0193841X15610192
    Download Restriction: no

    File URL: https://libkey.io/10.1177/0193841X15610192?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. repec:mpr:mprres:6683 is not listed on IDEAS
    2. Philip Oreopoulos & Daniel Lang & Joshua Angrist, 2009. "Incentives and Services for College Achievement: Evidence from a Randomized Trial," American Economic Journal: Applied Economics, American Economic Association, vol. 1(1), pages 136-163, January.
    3. Judith Scott-Clayton & Olga Rodriguez, 2014. "Development, Discouragement, or Diversion? New Evidence on the Effects of College Remediation Policy," Education Finance and Policy, MIT Press, vol. 10(1), pages 4-45, November.
    4. Jason M. Lindo & Nicholas J. Sanders & Philip Oreopoulos, 2010. "Ability, Gender, and Performance Standards: Evidence from Academic Probation," American Economic Journal: Applied Economics, American Economic Association, vol. 2(2), pages 95-117, April.
    5. Guido Imbens & Karthik Kalyanaraman, 2012. "Optimal Bandwidth Choice for the Regression Discontinuity Estimator," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 79(3), pages 933-959.
    6. West, S.G. & Duan, N. & Pequegnat, W. & Gaist, P. & Des Jarlais, D.C. & Holtgrave, D. & Szapocznik, J. & Fishbein, M. & Rapkin, B. & Clatts, M. & Mullen, P.D., 2008. "Alternatives to the randomized controlled trial," American Journal of Public Health, American Public Health Association, vol. 98(8), pages 1359-1366.
    7. Imbens, Guido W. & Lemieux, Thomas, 2008. "Regression discontinuity designs: A guide to practice," Journal of Econometrics, Elsevier, vol. 142(2), pages 615-635, February.
    8. Andrew Gelman & Guido Imbens, 2019. "Why High-Order Polynomials Should Not Be Used in Regression Discontinuity Designs," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 37(3), pages 447-456, July.
    9. McCrary, Justin, 2008. "Manipulation of the running variable in the regression discontinuity design: A density test," Journal of Econometrics, Elsevier, vol. 142(2), pages 698-714, February.
    10. Rebecca A. Maynard & Kenneth A. Couch & Coady Wing & Thomas D. Cook, 2013. "Strengthening The Regression Discontinuity Design Using Additional Design Elements: A Within‐Study Comparison," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 32(4), pages 853-877, September.
    11. Vivian C. Wong & Thomas D. Cook & W. Steven Barnett & Kwanghee Jung, 2008. "An effectiveness-based evaluation of five state pre-kindergarten programs," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 27(1), pages 122-154.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Adam C. Sales & Ben B. Hansen, 2020. "Limitless Regression Discontinuity," Journal of Educational and Behavioral Statistics, , vol. 45(2), pages 143-174, April.
    2. Ari Hyytinen & Jaakko Meriläinen & Tuukka Saarimaa & Otto Toivanen & Janne Tukiainen, 2018. "When does regression discontinuity design work? Evidence from random election outcomes," Quantitative Economics, Econometric Society, vol. 9(2), pages 1019-1051, July.
    3. Lucia Rizzica, 2018. "Raising aspirations and higher education: evidence from the UK’s Widening Participation policy," Temi di discussione (Economic working papers) 1188, Bank of Italy, Economic Research and International Relations Area.
    4. Thushyanthan Baskaran & Sonia Bhalotra & Brian Min & Yogesh Uppal, 2024. "Women legislators and economic performance," Journal of Economic Growth, Springer, vol. 29(2), pages 151-214, June.
    5. Koichiro Ito & Shuang Zhang, 2020. "Willingness to Pay for Clean Air: Evidence from Air Purifier Markets in China," Journal of Political Economy, University of Chicago Press, vol. 128(5), pages 1627-1672.
    6. Erasmo Giambona & Rafael P. Ribas, 2023. "Unveiling the Price of Obscenity: Evidence From Closing Prostitution Windows in Amsterdam," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 42(3), pages 677-705, June.
    7. Claire MacPherson & Olivier Sterck, 2019. "Humanitarian vs. Development Aid for Refugees: Evidence from a Regression Discontinuity Design," CSAE Working Paper Series 2019-15, Centre for the Study of African Economies, University of Oxford.
    8. María F. Prada & Graciana Rucci & Sergio S. Urzúa, 2015. "The Effect of Mandated Child Care on Female Wages in Chile," NBER Working Papers 21080, National Bureau of Economic Research, Inc.
    9. Adrien Montalbo, 2020. "Education supply and economic growth in nineteenth-century France," Working Papers halshs-02482643, HAL.
    10. Jain, Chandan & Kashyap, Shagun & Lahoti, Rahul & Sahoo, Soham, 2022. "Do Educated Leaders Affect Economic Development? Evidence from India," IZA Discussion Papers 15278, Institute of Labor Economics (IZA).
    11. Daniel Burkhard, 2015. "Consumption smoothing at retirement: average and quantile treatment effects in the regression discontinuity design," Diskussionsschriften dp1512, Universitaet Bern, Departement Volkswirtschaft.
    12. Li, Yunfei & Babazono, Akira & Jamal, Aziz & Liu, Ning & Fujita, Takako & Zhao, Rui & Maeno, Yukari & Su, Ya & Liang, Lifan & Yao, Lan, 2022. "The impact of lifestyle guidance intervention on health outcomes among Japanese middle-aged population with metabolic syndrome: A regression discontinuity study," Social Science & Medicine, Elsevier, vol. 314(C).
    13. Decio Coviello & Andrea Guglielmo & Giancarlo Spagnolo, 2015. "The Effect of Discretion on Procurement Performance," CEIS Research Paper 361, Tor Vergata University, CEIS, revised 17 Nov 2015.
    14. Steven F. Koch & Jeffrey S. Racine, 2016. "Healthcare facility choice and user fee abolition: regression discontinuity in a multinomial choice setting," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 179(4), pages 927-950, October.
    15. Mauricio Villamizar‐Villegas & Freddy A. Pinzon‐Puerto & Maria Alejandra Ruiz‐Sanchez, 2022. "A comprehensive history of regression discontinuity designs: An empirical survey of the last 60 years," Journal of Economic Surveys, Wiley Blackwell, vol. 36(4), pages 1130-1178, September.
    16. Decio Coviello & Andrea Guglielmo & Giancarlo Spagnolo, 2018. "The Effect of Discretion on Procurement Performance," Management Science, INFORMS, vol. 64(2), pages 715-738, February.
    17. Prakash, Nishith & Rockmore, Marc & Uppal, Yogesh, 2019. "Do criminally accused politicians affect economic outcomes? Evidence from India," Journal of Development Economics, Elsevier, vol. 141(C).
    18. Wang, Yadong & Wang, Delu & Shi, Xunpeng, 2022. "Exploring the multidimensional effects of China's coal de-capacity policy: A regression discontinuity design," Resources Policy, Elsevier, vol. 75(C).
    19. Satyakti, Yayan, 2020. "Do Human Restriction Mobility Policy in Indonesia effectively reduce The Spread of COVID-19," MPRA Paper 101911, University Library of Munich, Germany.
    20. Gabrielle Fack & Julien Grenet, 2015. "Improving College Access and Success for Low-Income Students: Evidence from a Large Need-Based Grant Program," American Economic Journal: Applied Economics, American Economic Association, vol. 7(2), pages 1-34, April.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:evarev:v:39:y:2015:i:5:p:501-524. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.