IDEAS home Printed from https://ideas.repec.org/p/hal/journl/hal-03597819.html
   My bibliography  Save this paper

Measuring Judicial Sentiment: Methods and Application to US Circuit Courts

Author

Listed:
  • Elliott Ash

    (ETH Zürich - Eidgenössische Technische Hochschule - Swiss Federal Institute of Technology [Zürich])

  • Daniel L. Chen

    (IAST - Institute for Advanced Study in Toulouse, TSE-R - Toulouse School of Economics - UT Capitole - Université Toulouse Capitole - UT - Université de Toulouse - EHESS - École des hautes études en sciences sociales - CNRS - Centre National de la Recherche Scientifique - INRAE - Institut National de Recherche pour l’Agriculture, l’Alimentation et l’Environnement, CNRS - Centre National de la Recherche Scientifique)

  • Sergio Galletta

    (UniBg - Università degli Studi di Bergamo = University of Bergamo)

Abstract

This paper provides a general method for analysing the sentiments expressed in the language of judicial rulings. We apply natural language processing tools to the text of US appellate court opinions to extrapolate judges' sentiments (positive/good vs. negative/bad) towards a number of target social groups. We explore descriptively how these sentiments vary over time and across types of judges. In addition, we provide a method for using random assignment of judges in an instrumental variables framework to estimate causal effects of judges' sentiments. In an empirical application, we show that more positive sentiment influences future judges by increasing the likelihood of reversal but also increasing the number of forward citations.

Suggested Citation

  • Elliott Ash & Daniel L. Chen & Sergio Galletta, 2022. "Measuring Judicial Sentiment: Methods and Application to US Circuit Courts," Post-Print hal-03597819, HAL.
    • Handle: RePEc:hal:journl:hal-03597819
    DOI: 10.1111/ecca.12397
    Note: View the original document on HAL open archive server: https://hal.science/hal-03597819
    as

    Download full text from publisher

    File URL: https://hal.science/hal-03597819/document
    Download Restriction: no
    File URL: https://libkey.io/10.1111/ecca.12397?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Ash, Elliott & MacLeod, W. Bentley, 2021. "Reducing partisanship in judicial elections can improve judge quality: Evidence from U.S. state supreme courts," Journal of Public Economics, Elsevier, vol. 201(C).
    2. Will Dobbie & Jae Song, 2015. "Debt Relief and Debtor Outcomes: Measuring the Effects of Consumer Bankruptcy Protection," American Economic Review, American Economic Association, vol. 105(3), pages 1272-1311, March.
    3. Victor Chernozhukov & Denis Chetverikov & Mert Demirer & Esther Duflo & Christian Hansen & Whitney Newey & James Robins, 2018. "Double/debiased machine learning for treatment and structural parameters," Econometrics Journal, Royal Economic Society, vol. 21(1), pages 1-68, February.
    4. Claire S. H. Lim & James M. Snyder Jr. & David Strömberg Jr., 2015. "The Judge, the Politician, and the Press: Newspaper Coverage and Criminal Sentencing across Electoral Systems," American Economic Journal: Applied Economics, American Economic Association, vol. 7(4), pages 103-135, October.
    5. Rafael Di Tella & Ernesto Schargrodsky, 2013. "Criminal Recidivism after Prison and Electronic Monitoring," Journal of Political Economy, University of Chicago Press, vol. 121(1), pages 28-73.
    6. Chen, Daniel L. & Levonyan, Vardges & Yeh, Susan, 2016. "Policies Affect Preferences: Evidence from Random Variation in Abortion Jurisprudence," IAST Working Papers 16-58, Institute for Advanced Study in Toulouse (IAST).
    7. Angrist, Joshua D & Krueger, Alan B, 1995. "Split-Sample Instrumental Variables Estimates of the Return to Schooling," Journal of Business & Economic Statistics, American Statistical Association, vol. 13(2), pages 225-235, April.
    8. Mirko Draca & Carlo Schwarz, 2024. "How Polarised are Citizens? Measuring Ideology from the Ground up," The Economic Journal, Royal Economic Society, vol. 134(661), pages 1950-1984.
    9. A. Belloni & D. Chen & V. Chernozhukov & C. Hansen, 2012. "Sparse Models and Methods for Optimal Instruments With an Application to Eminent Domain," Econometrica, Econometric Society, vol. 80(6), pages 2369-2429, November.
    10. Oriana Bandiera & Andrea Prat & Stephen Hansen & Raffaella Sadun, 2020. "CEO Behavior and Firm Performance," Journal of Political Economy, University of Chicago Press, vol. 128(4), pages 1325-1369.
    11. Nicole Maestas & Kathleen J. Mullen & Alexander Strand, 2013. "Does Disability Insurance Receipt Discourage Work? Using Examiner Assignment to Estimate Causal Effects of SSDI Receipt," American Economic Review, American Economic Association, vol. 103(5), pages 1797-1829, August.
    12. Alberto Galasso & Mark Schankerman, 2014. "Patents and Cumulative Innovation: Causal Evidence from the Courts," NBER Working Papers 20269, National Bureau of Economic Research, Inc.
    13. Joshua D. Angrist & Alan B. Krueger, 1993. "Split Sample Instrumental Variables," Working Papers 699, Princeton University, Department of Economics, Industrial Relations Section..
    14. Jeffrey R. Kling, 2006. "Incarceration Length, Employment, and Earnings," American Economic Review, American Economic Association, vol. 96(3), pages 863-876, June.
    15. Elliott Ash & W. Bentley MacLeod, 2015. "Intrinsic Motivation in Public Service: Theory and Evidence from State Supreme Courts," Journal of Law and Economics, University of Chicago Press, vol. 58(4).
    16. Victor Chernozhukov & Denis Chetverikov & Mert Demirer & Esther Duflo & Christian Hansen & Whitney Newey, 2017. "Double/Debiased/Neyman Machine Learning of Treatment Effects," American Economic Review, American Economic Association, vol. 107(5), pages 261-265, May.
    17. Bhaven Sampat & Heidi L. Williams, 2019. "How Do Patents Affect Follow-On Innovation? Evidence from the Human Genome," American Economic Review, American Economic Association, vol. 109(1), pages 203-236, January.
    18. Elliott Ash & Daniel L. Chen, 2019. "Case Vectors: Spatial Representations of the Law Using Document Embeddings," Post-Print hal-03161822, HAL.
    19. Matthew Gentzkow & Jesse M. Shapiro, 2010. "What Drives Media Slant? Evidence From U.S. Daily Newspapers," Econometrica, Econometric Society, vol. 78(1), pages 35-71, January.
    20. Victor Chernozhukov & Denis Chetverikov & Mert Demirer & Esther Duflo & Christian Hansen & Whitney Newey & James Robins, 2016. "Double/Debiased Machine Learning for Treatment and Causal Parameters," Papers 1608.00060, arXiv.org, revised Nov 2024.
    21. Hui Zou & Trevor Hastie, 2005. "Addendum: Regularization and variable selection via the elastic net," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 67(5), pages 768-768, November.
    22. Matthew Gentzkow & Jesse M. Shapiro & Matt Taddy, 2019. "Measuring Group Differences in High‐Dimensional Choices: Method and Application to Congressional Speech," Econometrica, Econometric Society, vol. 87(4), pages 1307-1340, July.
    23. Hui Zou & Trevor Hastie, 2005. "Regularization and variable selection via the elastic net," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 67(2), pages 301-320, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. van Loon, Austin, 2022. "Three Families of Automated Text Analysis," SocArXiv htnej, Center for Open Science.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Andres Algaba & David Ardia & Keven Bluteau & Samuel Borms & Kris Boudt, 2020. "Econometrics Meets Sentiment: An Overview Of Methodology And Applications," Journal of Economic Surveys, Wiley Blackwell, vol. 34(3), pages 512-547, July.
    2. Hoang, Daniel & Wiegratz, Kevin, 2022. "Machine learning methods in finance: Recent applications and prospects," Working Paper Series in Economics 158, Karlsruhe Institute of Technology (KIT), Department of Economics and Management.
    3. Mochen Yang & Edward McFowland & Gordon Burtch & Gediminas Adomavicius, 2022. "Achieving Reliable Causal Inference with Data-Mined Variables: A Random Forest Approach to the Measurement Error Problem," INFORMS Joural on Data Science, INFORMS, vol. 1(2), pages 138-155, October.
    4. Ivan A Canay & Magne Mogstad & Jack Mount, 2024. "On the Use of Outcome Tests for Detecting Bias in Decision Making," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 91(4), pages 2135-2167.
    5. Patrick Gaulé, 2018. "Patents and the Success of Venture‐Capital Backed Startups: Using Examiner Assignment to Estimate Causal Effects," Journal of Industrial Economics, Wiley Blackwell, vol. 66(2), pages 350-376, June.
    6. Ricardo P. Masini & Marcelo C. Medeiros & Eduardo F. Mendes, 2023. "Machine learning advances for time series forecasting," Journal of Economic Surveys, Wiley Blackwell, vol. 37(1), pages 76-111, February.
    7. Manudeep Bhuller & Gordon B. Dahl & Katrine V. Løken & Magne Mogstad, 2020. "Incarceration, Recidivism, and Employment," Journal of Political Economy, University of Chicago Press, vol. 128(4), pages 1269-1324.
    8. Jiaming Mao & Jingzhi Xu, 2020. "Ensemble Learning with Statistical and Structural Models," Papers 2006.05308, arXiv.org.
    9. Thomas Wiemann, 2023. "Optimal Categorical Instrumental Variables," Papers 2311.17021, arXiv.org, revised May 2024.
    10. Jiaming Mao & Zhesheng Zheng, 2020. "Structural Regularization," Papers 2004.12601, arXiv.org, revised Jun 2020.
    11. Johannes W. Ligtenberg & Tiemen Woutersen, 2024. "Multidimensional clustering in judge designs," Papers 2406.09473, arXiv.org.
    12. Huber Martin & Wüthrich Kaspar, 2019. "Local Average and Quantile Treatment Effects Under Endogeneity: A Review," Journal of Econometric Methods, De Gruyter, vol. 8(1), pages 1-27, January.
    13. Brigham Frandsen & Lars Lefgren & Emily Leslie, 2023. "Judging Judge Fixed Effects," American Economic Review, American Economic Association, vol. 113(1), pages 253-277, January.
    14. Megan T Stevenson, 2018. "Distortion of Justice: How the Inability to Pay Bail Affects Case Outcomes," The Journal of Law, Economics, and Organization, Oxford University Press, vol. 34(4), pages 511-542.
    15. Sander Gerritsen & Mark Kattenberg & Sonny Kuijpers, 2019. "The impact of age at arrival on education and mental health," CPB Discussion Paper 389.rdf, CPB Netherlands Bureau for Economic Policy Analysis.
    16. Michael Lechner, 2023. "Causal Machine Learning and its use for public policy," Swiss Journal of Economics and Statistics, Springer;Swiss Society of Economics and Statistics, vol. 159(1), pages 1-15, December.
    17. Kirill Borusyak & Peter Hull & Xavier Jaravel, 2022. "Quasi-Experimental Shift-Share Research Designs," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 89(1), pages 181-213.
    18. Sander Gerritsen & Mark Kattenberg & Sonny Kuijpers, 2019. "The impact of age at arrival on education and mental health," CPB Discussion Paper 389, CPB Netherlands Bureau for Economic Policy Analysis.
    19. Sofia Amaral & Gordon B. Dahl & Victoria Endl-Geyer & Timo Hener & Helmut Rainer, 2023. "Deterrence or Backlash? Arrests and the Dynamics of Domestic Violence," NBER Working Papers 30855, National Bureau of Economic Research, Inc.
    20. Qingliang Fan & Yaqian Wu, 2020. "Endogenous Treatment Effect Estimation with some Invalid and Irrelevant Instruments," Papers 2006.14998, arXiv.org.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:hal:journl:hal-03597819. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: CCSD (email available below). General contact details of provider: https://hal.archives-ouvertes.fr/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.