IDEAS home Printed from https://ideas.repec.org/a/wly/empleg/v20y2023i4p1005-1044.html
   My bibliography  Save this article

Measuring law's normative force

Author

Listed:
  • Kevin L. Cope

Abstract

An important question in legal theory and policy is when people are willing to put aside their policy preferences to uphold higher‐order legal values. That is, when does constitutional or international law, for instance, have “normative force”? Around two‐dozen experimental studies have attempted to measure this question empirically, but their designs contain an inherent limitation. While they are interested in gauging the effect of internalizing a norm, they measure only the effect of exposure to that norm. This is significant because subjects in the treatment group whose priors are strongly contrary to the treatment message on the legality of a policy may effectively be “treatment resistant”: it is difficult to successfully treat them because their prior beliefs on the issue are entrenched; as a result, they simply do not believe the treatment message. This treatment failure attenuates any effects, and where a significant portion of the treatment and/or control group is not successfully treated, the results will be biased toward small, null, or even backfire findings. This article first formally models the mechanism underlying experiments on law's normative force. I then demonstrate a methodological solution to the problem of treatment resistance. By using the experimental treatment as an instrumental variable and employing a post‐treatment treatment‐uptake test, the researcher can estimate the causal effect of the real explanatory variable of interest: sincerely holding a belief about a policy's higher‐order lawfulness. Using new data from a 2022 survey experiment conducted on US residents, I illustrate this method for three constitutional or international law issues. The theoretical and empirical results together suggest that backfire effects documented by some studies do not reflect a tepid or negative response against the legal source per se, but rather reflect treatment resistance. These findings suggest that we should re‐evaluate the existing body of experimental studies on law's normative force, and they should prompt researchers to reconsider how we conduct future research in this domain.

Suggested Citation

  • Kevin L. Cope, 2023. "Measuring law's normative force," Journal of Empirical Legal Studies, John Wiley & Sons, vol. 20(4), pages 1005-1044, December.
    • Handle: RePEc:wly:empleg:v:20:y:2023:i:4:p:1005-1044
    DOI: 10.1111/jels.12364
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/jels.12364
    Download Restriction: no
    File URL: https://libkey.io/10.1111/jels.12364?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Rafael Di Tella & Ernesto Schargrodsky, 2013. "Criminal Recidivism after Prison and Electronic Monitoring," Journal of Political Economy, University of Chicago Press, vol. 121(1), pages 28-73.
    2. Adam S. Chilton & Mila Versteeg, 2016. "Do Constitutional Rights Make a Difference?," American Journal of Political Science, John Wiley & Sons, vol. 60(3), pages 575-589, July.
    3. Markus Frölich & Martin Huber, 2014. "Treatment Evaluation With Multiple Outcome Periods Under Endogeneity and Attrition," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 109(508), pages 1697-1711, December.
    4. Angrist, Joshua D, 1990. "Lifetime Earnings and the Vietnam Era Draft Lottery: Evidence from Social Security Administrative Records," American Economic Review, American Economic Association, vol. 80(3), pages 313-336, June.
    5. Yonatan Lupu & Geoffrey P. R. Wallace, 2019. "Violence, Nonviolence, and the Effects of International Human Rights Law," American Journal of Political Science, John Wiley & Sons, vol. 63(2), pages 411-426, April.
    6. Katerina Linos, 2011. "Diffusion through Democracy," American Journal of Political Science, John Wiley & Sons, vol. 55(3), pages 678-695, July.
    7. Kevin L. Cope & Charles Crabtree, 2020. "A Nationalist Backlash to International Refugee Law: Evidence from a Survey Experiment in Turkey," Journal of Empirical Legal Studies, John Wiley & Sons, vol. 17(4), pages 752-788, December.
    8. Gaines, Brian J. & Kuklinski, James H. & Quirk, Paul J., 2007. "The Logic of the Survey Experiment Reexamined," Political Analysis, Cambridge University Press, vol. 15(1), pages 1-20, January.
    9. Charles S. Taber & Milton Lodge, 2006. "Motivated Skepticism in the Evaluation of Political Beliefs," American Journal of Political Science, John Wiley & Sons, vol. 50(3), pages 755-769, July.
    10. Adam S. Chilton, 2015. "The Laws of War and Public Opinion: An Experimental Study," Journal of Institutional and Theoretical Economics (JITE), Mohr Siebeck, Tübingen, vol. 171(1), pages 181-201, March.
    11. Chaudoin, Stephen, 2014. "Promises or Policies? An Experimental Analysis of International Agreements and Audience Reactions," International Organization, Cambridge University Press, vol. 68(1), pages 235-256, January.
    12. Aaron Chalfin & Justin McCrary, 2017. "Criminal Deterrence: A Review of the Literature," Journal of Economic Literature, American Economic Association, vol. 55(1), pages 5-48, March.
    13. Diana C. Mutz & Robin Pemantle & Philip Pham, 2019. "The Perils of Balance Testing in Experimental Design: Messy Analyses of Clean Data," The American Statistician, Taylor & Francis Journals, vol. 73(1), pages 32-42, January.
    14. Angrist, Joshua D, 1990. "Lifetime Earnings and the Vietnam Era Draft Lottery: Evidence from Social Security Administrative Records: Errata," American Economic Review, American Economic Association, vol. 80(5), pages 1284-1286, December.
    15. Berinsky, Adam J., 2017. "Rumors and Health Care Reform: Experiments in Political Misinformation," British Journal of Political Science, Cambridge University Press, vol. 47(2), pages 241-262, April.
    16. Barabas, Jason & Jerit, Jennifer, 2010. "Are Survey Experiments Externally Valid?," American Political Science Review, Cambridge University Press, vol. 104(2), pages 226-242, May.
    17. James N. Druckman & Thomas J. Leeper, 2012. "Learning More from Political Communication Experiments: Pretreatment and Its Effects," American Journal of Political Science, John Wiley & Sons, vol. 56(4), pages 875-896, October.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Huber Martin & Wüthrich Kaspar, 2019. "Local Average and Quantile Treatment Effects Under Endogeneity: A Review," Journal of Econometric Methods, De Gruyter, vol. 8(1), pages 1-27, January.
    2. Kaitlin Anderson & Gema Zamarro & Jennifer Steele & Trey Miller, 2021. "Comparing Performance of Methods to Deal With Differential Attrition in Randomized Experimental Evaluations," Evaluation Review, , vol. 45(1-2), pages 70-104, February.
    3. Dautović, Ernest & Gambacorta, Leonardo & Reghezza, Alessio, 2023. "Supervisory policy stimulus: evidence from the euro area dividend recommendation," Working Paper Series 2796, European Central Bank.
    4. Akresh, Richard & Lucchetti, Leonardo & Thirumurthy, Harsha, 2012. "Wars and child health: Evidence from the Eritrean–Ethiopian conflict," Journal of Development Economics, Elsevier, vol. 99(2), pages 330-340.
    5. Puhani, Patrick A. & Sterrenberg, Margret K., 2021. "Effects of Mandatory Military Service on Wages and Other Socioeconomic Outcomes," Hannover Economic Papers (HEP) dp-684, Leibniz Universität Hannover, Wirtschaftswissenschaftliche Fakultät.
    6. Jinhyun Lee, 2013. "A Consistent Nonparametric Bootstrap Test of Exogeneity," Discussion Paper Series, School of Economics and Finance 201316, School of Economics and Finance, University of St Andrews.
    7. repec:hal:wpspec:info:hdl:2441/5rkqqmvrn4tl22s9mc4b6ga2g is not listed on IDEAS
    8. Torun, Huzeyfe, 2019. "Ex-ante labor market effects of compulsory military service," Journal of Comparative Economics, Elsevier, vol. 47(1), pages 90-110.
    9. Muhammad Asali, 2017. "Military Service and Future Earnings Revisited," Working Papers 005-17 JEL Codes: J24, J3, International School of Economics at TSU, Tbilisi, Republic of Georgia.
    10. Poutvaara, Panu & Wagener, Andreas, 2007. "To draft or not to draft? Inefficiency, generational incidence, and political economy of military conscription," European Journal of Political Economy, Elsevier, vol. 23(4), pages 975-987, December.
    11. Katarina Keller & Panu Poutvaara & Andreas Wagener, 2009. "Does Military Draft Discourage Enrollment in Higher Education? Evidence from OECD Countries," CESifo Working Paper Series 2838, CESifo.
    12. Xintong Wang & Carlos A. Flores & Alfonso Flores-Lagunes, 2020. "The Effects of Vietnam-Era Military Service on the Long-Term Health of Veterans: A Bounds Analysis," Center for Policy Research Working Papers 234, Center for Policy Research, Maxwell School, Syracuse University.
    13. Angrist, Joshua D. & Chen, Stacey H. & Frandsen, Brigham R., 2010. "Did Vietnam veterans get sicker in the 1990s? The complicated effects of military service on self-reported health," Journal of Public Economics, Elsevier, vol. 94(11-12), pages 824-837, December.
    14. Frank Hubers & Dinand Webbink, 2015. "The long-term effects of military conscription on educational attainment and wages," IZA Journal of Labor Economics, Springer;Forschungsinstitut zur Zukunft der Arbeit GmbH (IZA), vol. 4(1), pages 1-16, December.
    15. Imbens, Guido W & Angrist, Joshua D, 1994. "Identification and Estimation of Local Average Treatment Effects," Econometrica, Econometric Society, vol. 62(2), pages 467-475, March.
    16. Kadir Atalay & Garry F. Barrett & Peter Siminski, 2019. "Pension incentives and the joint retirement of couples: evidence from two natural experiments," Journal of Population Economics, Springer;European Society for Population Economics, vol. 32(3), pages 735-767, July.
    17. Bastien Michel & Camille Hémet, 2022. "Custodial versus non-custodial sentences: Long-run evidence from an anticipated reform," PSE Working Papers halshs-03899897, HAL.
    18. Hicken, Allen & Leider, Stephen & Ravanilla, Nico & Yang, Dean, 2018. "Temptation in vote-selling: Evidence from a field experiment in the Philippines," Journal of Development Economics, Elsevier, vol. 131(C), pages 1-14.
    19. DiTraglia, Francis J. & García-Jimeno, Camilo, 2019. "Identifying the effect of a mis-classified, binary, endogenous regressor," Journal of Econometrics, Elsevier, vol. 209(2), pages 376-390.
    20. Tim Johnson & Dalton Conley, 2019. "Military Service and Public Sector Employment: Birthdates Called in the Vietnam Draft Lotteries Appear Excessively in the Population of Civilian U.S. Federal Personnel Records," NBER Working Papers 25859, National Bureau of Economic Research, Inc.
    21. David S. Lee & Thomas Lemieux, 2009. "Regression Discontinuity Designs In Economics," Working Papers 1118, Princeton University, Department of Economics, Industrial Relations Section..

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wly:empleg:v:20:y:2023:i:4:p:1005-1044. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://doi.org/10.1111/(ISSN)1740-1461 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.