IDEAS home Printed from https://ideas.repec.org/p/iza/izadps/dp17636.html
   My bibliography  Save this paper

Examiner and Judge Designs in Economics: A Practitioner's Guide

Author

Listed:
  • Chyn, Eric

    (University of Texas at Austin)

  • Frandsen, Brigham R.

    (Brigham Young University)

  • Leslie, Emily

    (Brigham Young University)

Abstract

This article provides empirical researchers with an introduction and guide to research designs based on variation in judge and examiner tendencies to administer treatments or other interventions. We review the basic theory behind the research design, outline the assumptions under which the design identifies causal effects, describe empirical tests of the conditions for identification, and discuss tradeoffs associated with choices researchers must make for estimation. We demonstrate concepts and best practices concretely in an empirical case study that uses an examiner tendency research design to study the effects of pre-trial detention.

Suggested Citation

  • Chyn, Eric & Frandsen, Brigham R. & Leslie, Emily, 2025. "Examiner and Judge Designs in Economics: A Practitioner's Guide," IZA Discussion Papers 17636, Institute of Labor Economics (IZA).
    • Handle: RePEc:iza:izadps:dp17636
    as

    Download full text from publisher

    File URL: https://docs.iza.org/dp17636.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Will Dobbie & Jae Song, 2015. "Debt Relief and Debtor Outcomes: Measuring the Effects of Consumer Bankruptcy Protection," American Economic Review, American Economic Association, vol. 105(3), pages 1272-1311, March.
    2. Robert Collinson & John Eric Humphries & Nicholas Mader & Davin Reed & Daniel Tannenbaum & Winnie van Dijk, 2024. "Eviction and Poverty in American Cities," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 139(1), pages 57-120.
    3. Manudeep Bhuller & Gordon B. Dahl & Katrine V. Løken & Magne Mogstad, 2020. "Incarceration, Recidivism, and Employment," Journal of Political Economy, University of Chicago Press, vol. 128(4), pages 1269-1324.
    4. David Arnold & Will Dobbie & Crystal S Yang, 2018. "Racial Bias in Bail Decisions," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 133(4), pages 1885-1932.
    5. Will Dobbie & Hans Grönqvist & Susan Niknami & Mårten Palme & Mikael Priks, 2018. "The Intergenerational Effects of Parental Incarceration," NBER Working Papers 24186, National Bureau of Economic Research, Inc.
    6. Bhuller, Manudeep & Khoury, Laura & Løken, Katrine V., 2021. "Prison, Mental Health and Family Spillovers," Discussion Paper Series in Economics 19/2021, Norwegian School of Economics, Department of Economics.
    7. Will Dobbie & Jacob Goldin & Crystal S. Yang, 2018. "The Effects of Pretrial Detention on Conviction, Future Crime, and Employment: Evidence from Randomly Assigned Judges," American Economic Review, American Economic Association, vol. 108(2), pages 201-240, February.
    8. Eric Helland & Minjae Yun, 2023. "More Talk, Less Conflict: Evidence from Requiring Informal Discovery Conferences," American Law and Economics Review, American Law and Economics Association, vol. 25(1), pages 129-189.
    9. Diana Bonfim & Gil Nogueira, 2024. "Corporate Reorganization and the Reallocation of Labor in Bankruptcy," Working Papers w202409, Banco de Portugal, Economics and Research Department.
    10. White, Ariel, 2019. "Misdemeanor Disenfranchisement? The Demobilizing Effects of Brief Jail Spells on Potential Voters," American Political Science Review, Cambridge University Press, vol. 113(2), pages 311-324, May.
    11. Elsa Augustine & Johanna Lacoe & Steven Raphael & Alissa Skog, 2022. "The Impact of Felony Diversion in San Francisco," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 41(3), pages 683-709, June.
    12. William Arbour, 2021. "Can Recidivism be Prevented from Behind Bars? Evidence from a Behavioral Program," Working Papers tecipa-683, University of Toronto, Department of Economics.
    13. Doyle, Joseph J., 2013. "Causal effects of foster care: An instrumental-variables approach," Children and Youth Services Review, Elsevier, vol. 35(7), pages 1143-1151.
    14. Ozkan Eren & Naci Mocan, 2021. "Juvenile Punishment, High School Graduation, and Adult Crime: Evidence from Idiosyncratic Judge Harshness," The Review of Economics and Statistics, MIT Press, vol. 103(1), pages 34-47, March.
    15. Rachel Nesbit, 2022. "The Role of Mandated Mental Health Treatment in the Criminal Justice System," Papers 2212.06736, arXiv.org, revised Nov 2023.
    16. Galasso, Alberto & Schankerman, Mark, 2015. "Patents and cumulative innovation: causal evidence from the courts," LSE Research Online Documents on Economics 61614, London School of Economics and Political Science, LSE Library.
    17. Mariyana Zapryanova, 2020. "The Effects of Time in Prison and Time on Parole on Recidivism," Journal of Law and Economics, University of Chicago Press, vol. 63(4), pages 699-727.
    18. Eric French & Jae Song, 2014. "The Effect of Disability Insurance Receipt on Labor Supply," American Economic Journal: Economic Policy, American Economic Association, vol. 6(2), pages 291-337, May.
    19. Andrew Jordan & Ezra Karger & Derek Neal, 2021. "Heterogeneous Impacts of Sentencing Decisions," Working Paper Series WP 2022-02, Federal Reserve Bank of Chicago, revised 12 Feb 2022.
    20. Patrick Gaulé, 2018. "Patents and the Success of Venture‐Capital Backed Startups: Using Examiner Assignment to Estimate Causal Effects," Journal of Industrial Economics, Wiley Blackwell, vol. 66(2), pages 350-376, June.
    21. Nicole Maestas & Kathleen J. Mullen & Alexander Strand, 2013. "Does Disability Insurance Receipt Discourage Work? Using Examiner Assignment to Estimate Causal Effects of SSDI Receipt," American Economic Review, American Economic Association, vol. 103(5), pages 1797-1829, August.
    22. Paul Beaudry & Tim Willems, 2022. "On the Macroeconomic Consequences of Over-Optimism," American Economic Journal: Macroeconomics, American Economic Association, vol. 14(1), pages 38-59, January.
    23. Anthony Bald & Eric Chyn & Justine Hastings & Margarita Machelett, 2022. "The Causal Impact of Removing Children from Abusive and Neglectful Homes," Journal of Political Economy, University of Chicago Press, vol. 130(7), pages 1919-1962.
    24. Emily Leslie & Nolan G. Pope, 2017. "The Unintended Impact of Pretrial Detention on Case Outcomes: Evidence from New York City Arraignments," Journal of Law and Economics, University of Chicago Press, vol. 60(3), pages 529-557.
    25. Megan T Stevenson, 2018. "Distortion of Justice: How the Inability to Pay Bail Affects Case Outcomes," The Journal of Law, Economics, and Organization, Oxford University Press, vol. 34(4), pages 511-542.
    26. Arni, Patrick & Schiprowski, Amelie, 2019. "Job search requirements, effort provision and labor market outcomes," Journal of Public Economics, Elsevier, vol. 169(C), pages 65-88.
    27. Henri Fraisse, 2017. "Households Debt Restructuring: The Re-default Effects of a Debt Suspension," The Journal of Law, Economics, and Organization, Oxford University Press, vol. 33(4), pages 686-717.
    28. Jessica W. Gillooly, 2022. "“Lights and Sirens”: Variation in 911 Call‐Taker Risk Appraisal and its Effects on Police Officer Perceptions at the Scene," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 41(3), pages 762-786, June.
    29. Jeffrey R. Kling, 2006. "Incarceration Length, Employment, and Earnings," American Economic Review, American Economic Association, vol. 96(3), pages 863-876, June.
    30. Nicolás Grau & Gonzalo Marivil & Jorge Rivera, 2019. "The Effect of Pretrial Detention on Labor Market Outcomes," Working Papers wp488, University of Chile, Department of Economics.
    31. Honigsberg, Colleen & Jacob, Matthew, 2021. "Deleting misconduct: The expungement of BrokerCheck records," Journal of Financial Economics, Elsevier, vol. 139(3), pages 800-831.
    32. Eichmeyer, Sarah & Zhang, Jonathan, 2023. "Primary care providers’ influence on opioid use and its adverse consequences," Journal of Public Economics, Elsevier, vol. 217(C).
    33. Felipe M. Gonçalves & Steven Mello, 2023. "Police Discretion and Public Safety," NBER Working Papers 31678, National Bureau of Economic Research, Inc.
    34. Bas van der Klaauw & Heike Vethaak, 2022. "Empirical Evaluation of Broader Job Search Requirements for Unemployed Workers," Tinbergen Institute Discussion Papers 22-083/V, Tinbergen Institute.
    35. Jenny Williams & Don Weatherburn, 2022. "Can Electronic Monitoring Reduce Reoffending?," The Review of Economics and Statistics, MIT Press, vol. 104(2), pages 232-245, May.
    36. E. Jason Baron & Max Gross, 2022. "Is There a Foster Care-To-Prison Pipeline? Evidence from Quasi-Randomly Assigned Investigators," NBER Working Papers 29922, National Bureau of Economic Research, Inc.
    37. Henri Fraisse, 2017. "Households Debt Restructuring: The Re-default Effects of a Debt Suspension," The Journal of Law, Economics, and Organization, Oxford University Press, vol. 33(4), pages 686-717.
    38. Bonfim, Diana & Nogueira, Gil, 2024. "Corporate reorganization and the reallocation of labor in bankruptcy," Working Paper Series 2974, European Central Bank.
    39. Joseph Doyle & John Graves & Jonathan Gruber, 2019. "Evaluating Measures of Hospital Quality: Evidence from Ambulance Referral Patterns," The Review of Economics and Statistics, MIT Press, vol. 101(5), pages 841-852, December.
    40. Emely Ek Blæhr & Rikke Søgaard, 2021. "Instrumental variable‐based assessment of the effect of psychotherapy on suicide attempts, health, and economic outcomes in schizophrenia," Health Economics, John Wiley & Sons, Ltd., vol. 30(4), pages 903-914, April.
    41. Daniel Herbst, 2023. "The Impact of Income-Driven Repayment on Student Borrower Outcomes," American Economic Journal: Applied Economics, American Economic Association, vol. 15(1), pages 1-25, January.
    42. Samuel Norris & Matthew Pecenco & Jeffrey Weaver, 2021. "The Effects of Parental and Sibling Incarceration: Evidence from Ohio," American Economic Review, American Economic Association, vol. 111(9), pages 2926-2963, September.
    43. Will Dobbie & Paul Goldsmith-Pinkham & Crystal S. Yang, 2017. "Consumer Bankruptcy and Financial Health," The Review of Economics and Statistics, MIT Press, vol. 99(5), pages 853-869, December.
    44. Aurelien Quignon, 2023. "Crowd-based feedback and early-stage entrepreneurial performance: Evidence from a digital platform," Post-Print hal-04701800, HAL.
    45. Alberto Galasso & Mark Schankerman, 2018. "Patent rights, innovation, and firm exit," RAND Journal of Economics, RAND Corporation, vol. 49(1), pages 64-86, March.
    46. Logan M. Lee, 2023. "Halfway Home? Residential Housing and Reincarceration," American Economic Journal: Applied Economics, American Economic Association, vol. 15(3), pages 117-149, July.
    47. Anna Aizer & Joseph J. Doyle, 2015. "Juvenile Incarceration, Human Capital, and Future Crime: Evidence from Randomly Assigned Judges," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 130(2), pages 759-803.
    48. Arpit Gupta & Christopher Hansman & Ethan Frenchman, 2016. "The Heavy Costs of High Bail: Evidence from Judge Randomization," The Journal of Legal Studies, University of Chicago Press, vol. 45(2), pages 471-505.
    49. Morten Grindaker & Andreas R. Kostøl & Kasper Roszbach, 2021. "Executive Labor Market Frictions, Corporate Bankruptcy and CEO Careers," Working Paper 2021/15, Norges Bank.
    50. Ing-Haw Cheng & Felipe Severino & Richard R Townsend & Stijn Van Nieuwerburgh, 2021. "How Do Consumers Fare When Dealing with Debt Collectors? Evidence from Out-of-Court Settlements [Why don’t lenders renegotiate more home mortgages? Redefaults, self-cures and securitization]," The Review of Financial Studies, Society for Financial Studies, vol. 34(4), pages 1617-1660.
    51. David C. Chan & David Card & Lowell Taylor, 2023. "Is There a VA Advantage? Evidence from Dually Eligible Veterans," American Economic Review, American Economic Association, vol. 113(11), pages 3003-3043, November.
    52. Antill, Samuel, 2022. "Do the right firms survive bankruptcy?," Journal of Financial Economics, Elsevier, vol. 144(2), pages 523-546.
    53. Okudaira, Hiroko, 2018. "The economic costs of court decisions concerning dismissals in Japan: Identification by judge transfers," International Review of Law and Economics, Elsevier, vol. 53(C), pages 60-75.
    54. Joseph J. Doyle Jr., 2008. "Child Protection and Adult Crime: Using Investigator Assignment to Estimate Causal Effects of Foster Care," Journal of Political Economy, University of Chicago Press, vol. 116(4), pages 746-770, August.
    55. Sarah Eichmeyer & Jonathan Zhang, 2022. "Pathways into Opioid Dependence: Evidence from Practice Variation in Emergency Departments," American Economic Journal: Applied Economics, American Economic Association, vol. 14(4), pages 271-300, October.
    56. Alexeev, Sergey & Weatherburn, Don, 2022. "Fines for illicit drug use do not prevent future crime: evidence from randomly assigned judges," Journal of Economic Behavior & Organization, Elsevier, vol. 200(C), pages 555-575.
    57. Ofek-Shanny, Yuval & Strulov-Shlain, Avner & Zeltzer, Dan, 2023. "Impacts of Home-Care Subsidies: Evidence from Quasi-Random Assignment," IZA Discussion Papers 16551, Institute of Labor Economics (IZA).
    58. Bakx, Pieter & Wouterse, Bram & van Doorslaer, Eddy & Wong, Albert, 2020. "Better off at home? Effects of nursing home eligibility on costs, hospitalizations and survival," Journal of Health Economics, Elsevier, vol. 73(C).
    59. Michelle Yin & Garima Siwach & Dajun Lin, 2023. "Vocational Rehabilitation Services and Labor Market Outcomes for Transition‐Age Youth with Disabilities in Maine," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 42(1), pages 166-197, January.
    60. Sendhil Mullainathan & Ziad Obermeyer, 2022. "Diagnosing Physician Error: A Machine Learning Approach to Low-Value Health Care [“The Determinants of Productivity in Medical Testing: Intensity and Allocation of Care,”]," The Quarterly Journal of Economics, Oxford University Press, vol. 137(2), pages 679-727.
    61. Meier, Armando N. & Levav, Jonathan & Meier, Stephan, 2020. "Early Release and Recidivism," IZA Discussion Papers 13035, Institute of Labor Economics (IZA).
    62. Quignon, Aurelien, 2023. "Crowd-based feedback and early-stage entrepreneurial performance: Evidence from a digital platform," Research Policy, Elsevier, vol. 52(7).
    63. Jacelly C. Cespedes & Carlos R. Parra & Clemens Sialm, 2021. "The Effect of Principal Reduction on Household Distress: Evidence from Mortgage Cramdown," NBER Working Papers 28900, National Bureau of Economic Research, Inc.
    64. Alberto Galasso & Mark Schankerman, 2015. "Patents and Cumulative Innovation: Causal Evidence from the Courts," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 130(1), pages 317-369.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Jonathan Cohen & Geoffrey C. Schnorr, 2024. "Efficiency Costs of Unemployment Insurance Denial: Evidence from Randomly Assigned Examiners," Upjohn Working Papers 24-404, W.E. Upjohn Institute for Employment Research.
    2. Jason B. Cook & Chloe N. East, 2023. "The Effect of Means-Tested Transfers on Work: Evidence from Quasi-Randomly Assigned SNAP Caseworkers," NBER Working Papers 31307, National Bureau of Economic Research, Inc.
    3. Dennis Lim & Wenjie Wang & Yichong Zhang, 2024. "A Dimension-Agnostic Bootstrap Anderson-Rubin Test For Instrumental Variable Regressions," Papers 2412.01603, arXiv.org.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Johannes W. Ligtenberg & Tiemen Woutersen, 2024. "Multidimensional clustering in judge designs," Papers 2406.09473, arXiv.org.
    2. Brigham Frandsen & Lars Lefgren & Emily Leslie, 2023. "Judging Judge Fixed Effects," American Economic Review, American Economic Association, vol. 113(1), pages 253-277, January.
    3. William Arbour & Steeve Marchand, 2022. "Parole, Recidivism, and the Role of Supervised Transition," Working Papers tecipa-725, University of Toronto, Department of Economics.
    4. Ivan A Canay & Magne Mogstad & Jack Mount, 2024. "On the Use of Outcome Tests for Detecting Bias in Decision Making," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 91(4), pages 2135-2167.
    5. Robert Collinson & John Eric Humphries & Nicholas Mader & Davin Reed & Daniel Tannenbaum & Winnie van Dijk, 2024. "Eviction and Poverty in American Cities," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 139(1), pages 57-120.
    6. Michelle Yin & Garima Siwach & Dajun Lin, 2023. "Vocational Rehabilitation Services and Labor Market Outcomes for Transition‐Age Youth with Disabilities in Maine," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 42(1), pages 166-197, January.
    7. Sofia Amaral & Gordon B. Dahl & Victoria Endl-Geyer & Timo Hener & Helmut Rainer, 2023. "Deterrence or Backlash? Arrests and the Dynamics of Domestic Violence," NBER Working Papers 30855, National Bureau of Economic Research, Inc.
    8. Bhuller, Manudeep & Dahl, Gordon B & Løken, Katrine V. & Mogstad, Magne, 2018. "Incarceration Spillovers in Criminal and Family Networks," Discussion Paper Series in Economics 15/2018, Norwegian School of Economics, Department of Economics.
    9. Elsa Augustine & Johanna Lacoe & Steven Raphael & Alissa Skog, 2022. "The Impact of Felony Diversion in San Francisco," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 41(3), pages 683-709, June.
    10. Araujo, Aloisio & Ferreira, Rafael & Lagaras, Spyridon & Moraes, Flavio & Ponticelli, Jacopo & Tsoutsoura, Margarita, 2023. "The labor effects of judicial bias in bankruptcy," Journal of Financial Economics, Elsevier, vol. 150(2).
    11. Will Dobbie & Jacob Goldin & Crystal S. Yang, 2018. "The Effects of Pretrial Detention on Conviction, Future Crime, and Employment: Evidence from Randomly Assigned Judges," American Economic Review, American Economic Association, vol. 108(2), pages 201-240, February.
    12. Manudeep Bhuller & Gordon B. Dahl & Katrine V. Løken & Magne Mogstad, 2020. "Incarceration, Recidivism, and Employment," Journal of Political Economy, University of Chicago Press, vol. 128(4), pages 1269-1324.
    13. Amanda Agan & Jennifer Doleac & Anna Harvey, 2021. "Misdemeanor Prosecution," Working Papers 2021-014, Human Capital and Economic Opportunity Working Group.
    14. Anthony Bald & Eric Chyn & Justine Hastings & Margarita Machelett, 2022. "The Causal Impact of Removing Children from Abusive and Neglectful Homes," Journal of Political Economy, University of Chicago Press, vol. 130(7), pages 1919-1962.
    15. Carolina Arteaga, 2021. "Parental Incarceration and Children's Educational Attainment," Working Papers tecipa-703, University of Toronto, Department of Economics.
    16. Patrick Gaulé, 2018. "Patents and the Success of Venture‐Capital Backed Startups: Using Examiner Assignment to Estimate Causal Effects," Journal of Industrial Economics, Wiley Blackwell, vol. 66(2), pages 350-376, June.
    17. Julien Grenet & Hans Grönqvist & Susan Niknami, 2024. "The effects of electronic monitoring on offenders and their families," PSE-Ecole d'économie de Paris (Postprint) halshs-04630125, HAL.
    18. John Eric Humphries & Nicholas Mader & Daniel Tannenbaum & Winnie van Dijk, 2019. "Does Eviction Cause Poverty? Quasi-Experimental Evidence from Cook County, IL," CESifo Working Paper Series 7800, CESifo.
    19. Julien Grenet & Hans Grönqvist & Susan Niknami, 2024. "The effects of electronic monitoring on offenders and their families," Post-Print halshs-04630125, HAL.
    20. Hirvonen, Johannes & Kässi, Otto & Ropponen, Olli, 2023. "Jobs, Workers, and Firms: Dissecting the Labour Market Effects of Finland’s COVID-19 Subsidy Program," ETLA Working Papers 111, The Research Institute of the Finnish Economy.

    More about this item

    Keywords

    judges; examiners; judge fixed effects; applied econometrics;
    All these keywords.

    JEL classification:

    • C21 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Cross-Sectional Models; Spatial Models; Treatment Effect Models
    • C26 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Instrumental Variables (IV) Estimation
    • C31 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables - - - Cross-Sectional Models; Spatial Models; Treatment Effect Models; Quantile Regressions; Social Interaction Models
    • K14 - Law and Economics - - Basic Areas of Law - - - Criminal Law
    • C54 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Quantitative Policy Modeling

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:iza:izadps:dp17636. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Holger Hinte (email available below). General contact details of provider: https://edirc.repec.org/data/izaaade.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.