IDEAS home Printed from https://ideas.repec.org/a/wly/empleg/v20y2023i4p1005-1044.html
   My bibliography  Save this article

Measuring law's normative force

Author

Listed:
  • Kevin L. Cope

Abstract

An important question in legal theory and policy is when people are willing to put aside their policy preferences to uphold higher‐order legal values. That is, when does constitutional or international law, for instance, have “normative force”? Around two‐dozen experimental studies have attempted to measure this question empirically, but their designs contain an inherent limitation. While they are interested in gauging the effect of internalizing a norm, they measure only the effect of exposure to that norm. This is significant because subjects in the treatment group whose priors are strongly contrary to the treatment message on the legality of a policy may effectively be “treatment resistant”: it is difficult to successfully treat them because their prior beliefs on the issue are entrenched; as a result, they simply do not believe the treatment message. This treatment failure attenuates any effects, and where a significant portion of the treatment and/or control group is not successfully treated, the results will be biased toward small, null, or even backfire findings. This article first formally models the mechanism underlying experiments on law's normative force. I then demonstrate a methodological solution to the problem of treatment resistance. By using the experimental treatment as an instrumental variable and employing a post‐treatment treatment‐uptake test, the researcher can estimate the causal effect of the real explanatory variable of interest: sincerely holding a belief about a policy's higher‐order lawfulness. Using new data from a 2022 survey experiment conducted on US residents, I illustrate this method for three constitutional or international law issues. The theoretical and empirical results together suggest that backfire effects documented by some studies do not reflect a tepid or negative response against the legal source per se, but rather reflect treatment resistance. These findings suggest that we should re‐evaluate the existing body of experimental studies on law's normative force, and they should prompt researchers to reconsider how we conduct future research in this domain.

Suggested Citation

  • Kevin L. Cope, 2023. "Measuring law's normative force," Journal of Empirical Legal Studies, John Wiley & Sons, vol. 20(4), pages 1005-1044, December.
    • Handle: RePEc:wly:empleg:v:20:y:2023:i:4:p:1005-1044
    DOI: 10.1111/jels.12364
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/jels.12364
    Download Restriction: no
    File URL: https://libkey.io/10.1111/jels.12364?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Adam S. Chilton & Mila Versteeg, 2016. "Do Constitutional Rights Make a Difference?," American Journal of Political Science, John Wiley & Sons, vol. 60(3), pages 575-589, July.
    2. Angrist, Joshua D, 1990. "Lifetime Earnings and the Vietnam Era Draft Lottery: Evidence from Social Security Administrative Records," American Economic Review, American Economic Association, vol. 80(3), pages 313-336, June.
    3. Yonatan Lupu & Geoffrey P. R. Wallace, 2019. "Violence, Nonviolence, and the Effects of International Human Rights Law," American Journal of Political Science, John Wiley & Sons, vol. 63(2), pages 411-426, April.
    4. Katerina Linos, 2011. "Diffusion through Democracy," American Journal of Political Science, John Wiley & Sons, vol. 55(3), pages 678-695, July.
    5. Charles S. Taber & Milton Lodge, 2006. "Motivated Skepticism in the Evaluation of Political Beliefs," American Journal of Political Science, John Wiley & Sons, vol. 50(3), pages 755-769, July.
    6. Aaron Chalfin & Justin McCrary, 2017. "Criminal Deterrence: A Review of the Literature," Journal of Economic Literature, American Economic Association, vol. 55(1), pages 5-48, March.
    7. Rafael Di Tella & Ernesto Schargrodsky, 2013. "Criminal Recidivism after Prison and Electronic Monitoring," Journal of Political Economy, University of Chicago Press, vol. 121(1), pages 28-73.
    8. Markus Frölich & Martin Huber, 2014. "Treatment Evaluation With Multiple Outcome Periods Under Endogeneity and Attrition," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 109(508), pages 1697-1711, December.
    9. Berinsky, Adam J., 2017. "Rumors and Health Care Reform: Experiments in Political Misinformation," British Journal of Political Science, Cambridge University Press, vol. 47(2), pages 241-262, April.
    10. Barabas, Jason & Jerit, Jennifer, 2010. "Are Survey Experiments Externally Valid?," American Political Science Review, Cambridge University Press, vol. 104(2), pages 226-242, May.
    11. James N. Druckman & Thomas J. Leeper, 2012. "Learning More from Political Communication Experiments: Pretreatment and Its Effects," American Journal of Political Science, John Wiley & Sons, vol. 56(4), pages 875-896, October.
    12. Kevin L. Cope & Charles Crabtree, 2020. "A Nationalist Backlash to International Refugee Law: Evidence from a Survey Experiment in Turkey," Journal of Empirical Legal Studies, John Wiley & Sons, vol. 17(4), pages 752-788, December.
    13. Gaines, Brian J. & Kuklinski, James H. & Quirk, Paul J., 2007. "The Logic of the Survey Experiment Reexamined," Political Analysis, Cambridge University Press, vol. 15(1), pages 1-20, January.
    14. Adam S. Chilton, 2015. "The Laws of War and Public Opinion: An Experimental Study," Journal of Institutional and Theoretical Economics (JITE), Mohr Siebeck, Tübingen, vol. 171(1), pages 181-201, March.
    15. Chaudoin, Stephen, 2014. "Promises or Policies? An Experimental Analysis of International Agreements and Audience Reactions," International Organization, Cambridge University Press, vol. 68(1), pages 235-256, January.
    16. Diana C. Mutz & Robin Pemantle & Philip Pham, 2019. "The Perils of Balance Testing in Experimental Design: Messy Analyses of Clean Data," The American Statistician, Taylor & Francis Journals, vol. 73(1), pages 32-42, January.
    17. Angrist, Joshua D, 1990. "Lifetime Earnings and the Vietnam Era Draft Lottery: Evidence from Social Security Administrative Records: Errata," American Economic Review, American Economic Association, vol. 80(5), pages 1284-1286, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Huber Martin & Wüthrich Kaspar, 2019. "Local Average and Quantile Treatment Effects Under Endogeneity: A Review," Journal of Econometric Methods, De Gruyter, vol. 8(1), pages 1-27, January.
    2. Kaitlin Anderson & Gema Zamarro & Jennifer Steele & Trey Miller, 2021. "Comparing Performance of Methods to Deal With Differential Attrition in Randomized Experimental Evaluations," Evaluation Review, , vol. 45(1-2), pages 70-104, February.
    3. Dautović, Ernest & Gambacorta, Leonardo & Reghezza, Alessio, 2023. "Supervisory policy stimulus: evidence from the euro area dividend recommendation," Working Paper Series 2796, European Central Bank.
    4. Jinhyun Lee, 2013. "A Consistent Nonparametric Bootstrap Test of Exogeneity," Discussion Paper Series, School of Economics and Finance 201316, School of Economics and Finance, University of St Andrews.
    5. Torun, Huzeyfe, 2019. "Ex-ante labor market effects of compulsory military service," Journal of Comparative Economics, Elsevier, vol. 47(1), pages 90-110.
    6. Poutvaara, Panu & Wagener, Andreas, 2007. "To draft or not to draft? Inefficiency, generational incidence, and political economy of military conscription," European Journal of Political Economy, Elsevier, vol. 23(4), pages 975-987, December.
    7. Shantanu Gupta & Zachary C. Lipton & David Childers, 2021. "Efficient Online Estimation of Causal Effects by Deciding What to Observe," Papers 2108.09265, arXiv.org, revised Oct 2021.
    8. Bagues, Manuel & Roth, Christopher, 2020. "Interregional Contact and National Identity," CAGE Online Working Paper Series 526, Competitive Advantage in the Global Economy (CAGE).
    9. Orley Ashenfelter & William J. Collins & Albert Yoon, 2006. "Evaluating the Role of Brown v. Board of Education in School Equalization, Desegregation, and the Income of African Americans," American Law and Economics Review, American Law and Economics Association, vol. 8(2), pages 213-248.
    10. Timothy F. Harris & Aaron Yelowitz, 2018. "Life Insurance Holdings And Well‐Being Of Surviving Spouses," Contemporary Economic Policy, Western Economic Association International, vol. 36(3), pages 526-538, July.
    11. Guillermo Cruces & Martín A. Rossi & Ernesto Schargrodsky, 2023. "Dishonesty and Public Employment," American Economic Review: Insights, American Economic Association, vol. 5(4), pages 511-526, December.
    12. Shinichi Kitano, 2021. "Estimation of Determinants of Farmland Abandonment and Its Data Problems," Land, MDPI, vol. 10(6), pages 1-17, June.
    13. Andrea Kutinova, 2009. "Paternity Deferments And The Timing Of Births: U.S. Natality During The Vietnam War," Economic Inquiry, Western Economic Association International, vol. 47(2), pages 351-365, April.
    14. Anna Piil Damm, 2009. "Ethnic Enclaves and Immigrant Labor Market Outcomes: Quasi-Experimental Evidence," Journal of Labor Economics, University of Chicago Press, vol. 27(2), pages 281-314, April.
    15. A. Smith, Jeffrey & E. Todd, Petra, 2005. "Does matching overcome LaLonde's critique of nonexperimental estimators?," Journal of Econometrics, Elsevier, vol. 125(1-2), pages 305-353.
    16. Bäckström, Peter, 2023. "Swedish Veterans After Bosnia: The Relationship Between Military Deployment and Labour Market Marginalisation," Umeå Economic Studies 1011, Umeå University, Department of Economics.
    17. Manuel Denzer, 2019. "Estimating Causal Effects in Binary Response Models with Binary Endogenous Explanatory Variables - A Comparison of Possible Estimators," Working Papers 1916, Gutenberg School of Management and Economics, Johannes Gutenberg-Universität Mainz.
    18. Joshua D. Angrist & Jörn-Steffen Pischke, 2010. "The Credibility Revolution in Empirical Economics: How Better Research Design Is Taking the Con out of Econometrics," Journal of Economic Perspectives, American Economic Association, vol. 24(2), pages 3-30, Spring.
    19. Heather M. Rackin, 2017. "Comparing Veteran and Non-veteran Racial Disparities in Mid-life Health and Well-being," Population Research and Policy Review, Springer;Southern Demographic Association (SDA), vol. 36(3), pages 331-356, June.
    20. Daniel K. Fetter, 2013. "How Do Mortgage Subsidies Affect Home Ownership? Evidence from the Mid-century GI Bills," American Economic Journal: Economic Policy, American Economic Association, vol. 5(2), pages 111-147, May.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wly:empleg:v:20:y:2023:i:4:p:1005-1044. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://doi.org/10.1111/(ISSN)1740-1461 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.