IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2406.09473.html
   My bibliography  Save this paper

Multidimensional clustering in judge designs

Author

Listed:
  • Johannes W. Ligtenberg
  • Tiemen Woutersen

Abstract

Estimates in judge designs run the risk of being biased due to the many judge identities that are implicitly or explicitly used as instrumental variables. The usual method to analyse judge designs, via a leave-out mean instrument, eliminates this many instrument bias only in case the data are clustered in at most one dimension. What is left out in the mean defines this clustering dimension. How most judge designs cluster their standard errors, however, implies that there are additional clustering dimensions, which makes that a many instrument bias remains. We propose two estimators that are many instrument bias free, also in multidimensional clustered judge designs. The first generalises the one dimensional cluster jackknife instrumental variable estimator, by removing from this estimator the additional bias terms due to the extra dependence in the data. The second models all but one clustering dimensions by fixed effects and we show how these numerous fixed effects can be removed without introducing extra bias. A Monte-Carlo experiment and the revisitation of two judge designs show the empirical relevance of properly accounting for multidimensional clustering in estimation.

Suggested Citation

  • Johannes W. Ligtenberg & Tiemen Woutersen, 2024. "Multidimensional clustering in judge designs," Papers 2406.09473, arXiv.org.
  • Handle: RePEc:arx:papers:2406.09473
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2406.09473
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Joan Farre‐Mensa & Deepak Hegde & Alexander Ljungqvist, 2020. "What Is a Patent Worth? Evidence from the U.S. Patent “Lottery”," Journal of Finance, American Finance Association, vol. 75(2), pages 639-682, April.
    2. Will Dobbie & Jae Song, 2015. "Debt Relief and Debtor Outcomes: Measuring the Effects of Consumer Bankruptcy Protection," American Economic Review, American Economic Association, vol. 105(3), pages 1272-1311, March.
    3. Robert Collinson & John Eric Humphries & Nicholas Mader & Davin Reed & Daniel Tannenbaum & Winnie van Dijk, 2024. "Eviction and Poverty in American Cities," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 139(1), pages 57-120.
    4. Joseph J. Doyle Jr., 2007. "Child Protection and Child Outcomes: Measuring the Effects of Foster Care," American Economic Review, American Economic Association, vol. 97(5), pages 1583-1610, December.
    5. Manudeep Bhuller & Gordon B. Dahl & Katrine V. Løken & Magne Mogstad, 2020. "Incarceration, Recidivism, and Employment," Journal of Political Economy, University of Chicago Press, vol. 128(4), pages 1269-1324.
    6. Jeffrey R. Kling, 2006. "Incarceration Length, Employment, and Earnings," American Economic Review, American Economic Association, vol. 96(3), pages 863-876, June.
    7. Ozkan Eren & Naci Mocan, 2021. "Juvenile Punishment, High School Graduation, and Adult Crime: Evidence from Idiosyncratic Judge Harshness," The Review of Economics and Statistics, MIT Press, vol. 103(1), pages 34-47, March.
    8. David Arnold & Will Dobbie & Crystal S Yang, 2018. "Racial Bias in Bail Decisions," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 133(4), pages 1885-1932.
    9. David Autor & Andreas Kostøl & Magne Mogstad & Bradley Setzler, 2019. "Disability Benefits, Consumption Insurance, and Household Labor Supply," American Economic Review, American Economic Association, vol. 109(7), pages 2613-2654, July.
    10. Samuel Norris & Matthew Pecenco & Jeffrey Weaver, 2021. "The Effects of Parental and Sibling Incarceration: Evidence from Ohio," American Economic Review, American Economic Association, vol. 111(9), pages 2926-2963, September.
    11. Crudu, Federico & Mellace, Giovanni & Sándor, Zsolt, 2021. "Inference In Instrumental Variable Models With Heteroskedasticity And Many Instruments," Econometric Theory, Cambridge University Press, vol. 37(2), pages 281-310, April.
    12. Gordon B. Dahl & Andreas Ravndal Kostøl & Magne Mogstad, 2014. "Family Welfare Cultures," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 129(4), pages 1711-1752.
    13. Black, Bernard & French, Eric & McCauley, Jeremy & Song, Jae, 2024. "The effect of disability insurance receipt on mortality," Journal of Public Economics, Elsevier, vol. 229(C).
    14. Honigsberg, Colleen & Jacob, Matthew, 2021. "Deleting misconduct: The expungement of BrokerCheck records," Journal of Financial Economics, Elsevier, vol. 139(3), pages 800-831.
    15. Max Gross & E. Jason Baron, 2022. "Temporary Stays and Persistent Gains: The Causal Effects of Foster Care," American Economic Journal: Applied Economics, American Economic Association, vol. 14(2), pages 170-199, April.
    16. Anna Mikusheva & Liyang Sun, 2022. "Inference with Many Weak Instruments [Specification Testing in Models with Many Instruments]," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 89(5), pages 2663-2686.
    17. Sarah Eichmeyer & Jonathan Zhang, 2022. "Pathways into Opioid Dependence: Evidence from Practice Variation in Emergency Departments," American Economic Journal: Applied Economics, American Economic Association, vol. 14(4), pages 271-300, October.
    18. A. Colin Cameron & Jonah B. Gelbach & Douglas L. Miller, 2011. "Robust Inference With Multiway Clustering," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 29(2), pages 238-249, April.
    19. Will Dobbie & Jacob Goldin & Crystal S. Yang, 2018. "The Effects of Pretrial Detention on Conviction, Future Crime, and Employment: Evidence from Randomly Assigned Judges," American Economic Review, American Economic Association, vol. 108(2), pages 201-240, February.
    20. Rafael Di Tella & Ernesto Schargrodsky, 2013. "Criminal Recidivism after Prison and Electronic Monitoring," Journal of Political Economy, University of Chicago Press, vol. 121(1), pages 28-73.
    21. Eric French & Jae Song, 2014. "The Effect of Disability Insurance Receipt on Labor Supply," American Economic Journal: Economic Policy, American Economic Association, vol. 6(2), pages 291-337, May.
    22. MacKinnon, James G. & Nielsen, Morten Ørregaard & Webb, Matthew D., 2023. "Cluster-robust inference: A guide to empirical practice," Journal of Econometrics, Elsevier, vol. 232(2), pages 272-299.
    23. Felipe M. Gonçalves & Steven Mello, 2023. "Police Discretion and Public Safety," NBER Working Papers 31678, National Bureau of Economic Research, Inc.
    24. White, Ariel, 2019. "Misdemeanor Disenfranchisement? The Demobilizing Effects of Brief Jail Spells on Potential Voters," American Political Science Review, Cambridge University Press, vol. 113(2), pages 311-324, May.
    25. Alberto Abadie & Susan Athey & Guido W. Imbens & Jeffrey M. Wooldridge, 2020. "Sampling‐Based versus Design‐Based Uncertainty in Regression Analysis," Econometrica, Econometric Society, vol. 88(1), pages 265-296, January.
    26. Alexeev, Sergey & Weatherburn, Don, 2022. "Fines for illicit drug use do not prevent future crime: evidence from randomly assigned judges," Journal of Economic Behavior & Organization, Elsevier, vol. 200(C), pages 555-575.
    27. Blomquist, Soren & Dahlberg, Matz, 1999. "Small Sample Properties of LIML and Jackknife IV Estimators: Experiments with Weak Instruments," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 14(1), pages 69-88, Jan.-Feb..
    28. Elsa Augustine & Johanna Lacoe & Steven Raphael & Alissa Skog, 2022. "The Impact of Felony Diversion in San Francisco," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 41(3), pages 683-709, June.
    29. Alberto Abadie & Susan Athey & Guido W Imbens & Jeffrey M Wooldridge, 2023. "When Should You Adjust Standard Errors for Clustering?," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 138(1), pages 1-35.
    30. William Arbour, 2021. "Can Recidivism be Prevented from Behind Bars? Evidence from a Behavioral Program," Working Papers tecipa-683, University of Toronto, Department of Economics.
    31. Doyle, Joseph J., 2013. "Causal effects of foster care: An instrumental-variables approach," Children and Youth Services Review, Elsevier, vol. 35(7), pages 1143-1151.
    32. Will Dobbie & Paul Goldsmith-Pinkham & Crystal S. Yang, 2017. "Consumer Bankruptcy and Financial Health," The Review of Economics and Statistics, MIT Press, vol. 99(5), pages 853-869, December.
    33. Christian Dippel & Dustin Frye & Bryan Leonard, 2020. "Property Rights without Transfer Rights: A Study of Indian Land Allotment," NBER Working Papers 27479, National Bureau of Economic Research, Inc.
    34. Bhaven Sampat & Heidi L. Williams, 2019. "How Do Patents Affect Follow-On Innovation? Evidence from the Human Genome," American Economic Review, American Economic Association, vol. 109(1), pages 203-236, January.
    35. Stephanie Holmes Didwania, 2020. "The Immediate Consequences of Federal Pretrial Detention," American Law and Economics Review, American Law and Economics Association, vol. 22(1), pages 24-74.
    36. Chao, John C. & Swanson, Norman R. & Woutersen, Tiemen, 2023. "Jackknife estimation of a cluster-sample IV regression model with many weak instruments," Journal of Econometrics, Elsevier, vol. 235(2), pages 1747-1769.
    37. Elior Cohen, 2024. "Housing the Homeless: The Effect of Placing Single Adults Experiencing Homelessness in Housing Programs on Future Homelessness and Socioeconomic Outcomes," American Economic Journal: Applied Economics, American Economic Association, vol. 16(2), pages 130-175, April.
    38. Galasso, Alberto & Schankerman, Mark, 2015. "Patents and cumulative innovation: causal evidence from the courts," LSE Research Online Documents on Economics 61614, London School of Economics and Political Science, LSE Library.
    39. Jerry A. Hausman & Whitney K. Newey & Tiemen Woutersen & John C. Chao & Norman R. Swanson, 2012. "Instrumental variable estimation with heteroskedasticity and many instruments," Quantitative Economics, Econometric Society, vol. 3(2), pages 211-255, July.
    40. Alberto Galasso & Mark Schankerman, 2018. "Patent rights, innovation, and firm exit," RAND Journal of Economics, RAND Corporation, vol. 49(1), pages 64-86, March.
    41. Mariyana Zapryanova, 2020. "The Effects of Time in Prison and Time on Parole on Recidivism," Journal of Law and Economics, University of Chicago Press, vol. 63(4), pages 699-727.
    42. Logan M. Lee, 2023. "Halfway Home? Residential Housing and Reincarceration," American Economic Journal: Applied Economics, American Economic Association, vol. 15(3), pages 117-149, July.
    43. Anna Aizer & Joseph J. Doyle, 2015. "Juvenile Incarceration, Human Capital, and Future Crime: Evidence from Randomly Assigned Judges," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 130(2), pages 759-803.
    44. Rieber, Alexander & Schechinger, Steffen, 2019. "Herding Behavior between Rating Agencies," VfS Annual Conference 2019 (Leipzig): 30 Years after the Fall of the Berlin Wall - Democracy and Market Economy 203580, Verein für Socialpolitik / German Economic Association.
    45. Arpit Gupta & Christopher Hansman & Ethan Frenchman, 2016. "The Heavy Costs of High Bail: Evidence from Judge Randomization," The Journal of Legal Studies, University of Chicago Press, vol. 45(2), pages 471-505.
    46. Sendhil Mullainathan & Ziad Obermeyer, 2022. "Diagnosing Physician Error: A Machine Learning Approach to Low-Value Health Care [“The Determinants of Productivity in Medical Testing: Intensity and Allocation of Care,”]," The Quarterly Journal of Economics, Oxford University Press, vol. 137(2), pages 679-727.
    47. Angrist, J D & Imbens, G W & Krueger, A B, 1999. "Jackknife Instrumental Variables Estimation," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 14(1), pages 57-67, Jan.-Feb..
    48. Denis Chetverikov & Jinyong Hahn & Zhipeng Liao & Andres Santos, 2023. "Standard errors when a regressor is randomly assigned," Papers 2303.10306, arXiv.org.
    49. Anthony Bald & Eric Chyn & Justine Hastings & Margarita Machelett, 2022. "The Causal Impact of Removing Children from Abusive and Neglectful Homes," Journal of Political Economy, University of Chicago Press, vol. 130(7), pages 1919-1962.
    50. Nicole Maestas & Kathleen J. Mullen & Alexander Strand, 2013. "Does Disability Insurance Receipt Discourage Work? Using Examiner Assignment to Estimate Causal Effects of SSDI Receipt," American Economic Review, American Economic Association, vol. 103(5), pages 1797-1829, August.
    51. William Arbour, 2021. "Can recidivism be prevented from behind bars? Evidence from a behavioral program," Working Papers 2021.07, International Network for Economic Research - INFER.
    52. Emily Leslie & Nolan G. Pope, 2017. "The Unintended Impact of Pretrial Detention on Case Outcomes: Evidence from New York City Arraignments," Journal of Law and Economics, University of Chicago Press, vol. 60(3), pages 529-557.
    53. Ing-Haw Cheng & Felipe Severino & Richard R Townsend & Stijn Van Nieuwerburgh, 2021. "How Do Consumers Fare When Dealing with Debt Collectors? Evidence from Out-of-Court Settlements [Why don’t lenders renegotiate more home mortgages? Redefaults, self-cures and securitization]," The Review of Financial Studies, Society for Financial Studies, vol. 34(4), pages 1617-1660.
    54. Amanda Agan & Jennifer L Doleac & Anna Harvey, 2023. "Misdemeanor Prosecution," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 138(3), pages 1453-1505.
    55. Megan T Stevenson, 2018. "Distortion of Justice: How the Inability to Pay Bail Affects Case Outcomes," The Journal of Law, Economics, and Organization, Oxford University Press, vol. 34(4), pages 511-542.
    56. Randi Hjalmarsson & Matthew J Lindquist, 2019. "The Causal Effect of Military Conscription on Crime," The Economic Journal, Royal Economic Society, vol. 129(622), pages 2522-2562.
    57. Jacelly C. Cespedes & Carlos R. Parra & Clemens Sialm, 2021. "The Effect of Principal Reduction on Household Distress: Evidence from Mortgage Cramdown," NBER Working Papers 28900, National Bureau of Economic Research, Inc.
    58. David C Chan & Matthew Gentzkow & Chuan Yu, 2022. "Selection with Variation in Diagnostic Skill: Evidence from Radiologists [The Determinants of Productivity in Medical Testing: Intensity and Allocation of Care]," The Quarterly Journal of Economics, Oxford University Press, vol. 137(2), pages 729-783.
    59. Johannes W. Ligtenberg, 2023. "Inference in IV models with clustered dependence, many instruments and weak identification," Papers 2306.08559, arXiv.org, revised Mar 2024.
    60. Alberto Galasso & Mark Schankerman, 2015. "Patents and Cumulative Innovation: Causal Evidence from the Courts," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 130(1), pages 317-369.
    61. Joseph J. Doyle Jr., 2008. "Child Protection and Adult Crime: Using Investigator Assignment to Estimate Causal Effects of Foster Care," Journal of Political Economy, University of Chicago Press, vol. 116(4), pages 746-770, August.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Brigham Frandsen & Lars Lefgren & Emily Leslie, 2023. "Judging Judge Fixed Effects," American Economic Review, American Economic Association, vol. 113(1), pages 253-277, January.
    2. Robert Collinson & John Eric Humphries & Nicholas Mader & Davin Reed & Daniel Tannenbaum & Winnie van Dijk, 2024. "Eviction and Poverty in American Cities," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 139(1), pages 57-120.
    3. Sofia Amaral & Gordon B. Dahl & Victoria Endl-Geyer & Timo Hener & Helmut Rainer, 2023. "Deterrence or Backlash? Arrests and the Dynamics of Domestic Violence," NBER Working Papers 30855, National Bureau of Economic Research, Inc.
    4. Manudeep Bhuller & Gordon B. Dahl & Katrine V. Løken & Magne Mogstad, 2020. "Incarceration, Recidivism, and Employment," Journal of Political Economy, University of Chicago Press, vol. 128(4), pages 1269-1324.
    5. William Arbour & Steeve Marchand, 2022. "Parole, Recidivism, and the Role of Supervised Transition," Working Papers tecipa-725, University of Toronto, Department of Economics.
    6. Ivan A. Canay & Magne Mogstad & Jack Mountjoy, 2020. "On the Use of Outcome Tests for Detecting Bias in Decision Making," NBER Working Papers 27802, National Bureau of Economic Research, Inc.
    7. Will Dobbie & Jacob Goldin & Crystal S. Yang, 2018. "The Effects of Pretrial Detention on Conviction, Future Crime, and Employment: Evidence from Randomly Assigned Judges," American Economic Review, American Economic Association, vol. 108(2), pages 201-240, February.
    8. Carolina Arteaga, 2021. "Parental Incarceration and Children's Educational Attainment," Working Papers tecipa-703, University of Toronto, Department of Economics.
    9. Michelle Yin & Garima Siwach & Dajun Lin, 2023. "Vocational Rehabilitation Services and Labor Market Outcomes for Transition‐Age Youth with Disabilities in Maine," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 42(1), pages 166-197, January.
    10. Patrick Gaulé, 2018. "Patents and the Success of Venture‐Capital Backed Startups: Using Examiner Assignment to Estimate Causal Effects," Journal of Industrial Economics, Wiley Blackwell, vol. 66(2), pages 350-376, June.
    11. Johannes W. Ligtenberg, 2023. "Inference in IV models with clustered dependence, many instruments and weak identification," Papers 2306.08559, arXiv.org, revised Mar 2024.
    12. Bhuller, Manudeep & Dahl, Gordon B & Løken, Katrine V. & Mogstad, Magne, 2018. "Incarceration Spillovers in Criminal and Family Networks," Discussion Paper Series in Economics 15/2018, Norwegian School of Economics, Department of Economics.
    13. Araujo, Aloisio & Ferreira, Rafael & Lagaras, Spyridon & Moraes, Flavio & Ponticelli, Jacopo & Tsoutsoura, Margarita, 2023. "The labor effects of judicial bias in bankruptcy," Journal of Financial Economics, Elsevier, vol. 150(2).
    14. Anthony Bald & Eric Chyn & Justine Hastings & Margarita Machelett, 2022. "The Causal Impact of Removing Children from Abusive and Neglectful Homes," Journal of Political Economy, University of Chicago Press, vol. 130(7), pages 1919-1962.
    15. Sølvsten, Mikkel, 2020. "Robust estimation with many instruments," Journal of Econometrics, Elsevier, vol. 214(2), pages 495-512.
    16. Agan, Amanda & Doleac, Jennifer & Harvey, Anna, 2021. "Misdemeanor Prosecution," IZA Discussion Papers 14234, Institute of Labor Economics (IZA).
    17. Elsa Augustine & Johanna Lacoe & Steven Raphael & Alissa Skog, 2022. "The Impact of Felony Diversion in San Francisco," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 41(3), pages 683-709, June.
    18. Anna Mikusheva & Liyang Sun, 2024. "Weak identification with many instruments," The Econometrics Journal, Royal Economic Society, vol. 27(2), pages -28.
    19. Okudaira, Hiroko, 2018. "The economic costs of court decisions concerning dismissals in Japan: Identification by judge transfers," International Review of Law and Economics, Elsevier, vol. 53(C), pages 60-75.
    20. Lim, Dennis & Wang, Wenjie & Zhang, Yichong, 2024. "A conditional linear combination test with many weak instruments," Journal of Econometrics, Elsevier, vol. 238(2).

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2406.09473. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.