IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2403.03299.html
   My bibliography  Save this paper

Demystifying and avoiding the OLS "weighting problem": Unmodeled heterogeneity and straightforward solutions

Author

Listed:
  • Chad Hazlett
  • Tanvi Shinkre

Abstract

Researchers have long run regressions of an outcome variable (Y) on a treatment (D) and covariates (X) to estimate treatment effects. Even absent unobserved confounding, the regression coefficient on D in this setup reports a conditional variance weighted average of strata-wise average effects, not generally equal to the average treatment effect (ATE). Numerous proposals have been offered to cope with this "weighting problem", including interpretational tools to help characterize the weights and diagnostic aids to help researchers assess the potential severity of this problem. We make two contributions that together suggest an alternative direction for researchers and this literature. Our first contribution is conceptual, demystifying these weights. Simply put, under heterogeneous treatment effects (and varying probability of treatment), the linear regression of Y on D and X will be misspecified. The "weights" of regression offer one characterization for the coefficient from regression that helps to clarify how it will depart from the ATE. We also derive a more general expression for the weights than what is usually referenced. Our second contribution is practical: as these weights simply characterize misspecification bias, we suggest simply avoiding them through an approach that tolerate heterogeneous effects. A wide range of longstanding alternatives (regression-imputation/g-computation, interacted regression, and balancing weights) relax specification assumptions to allow heterogeneous effects. We make explicit the assumption of "separate linearity", under which each potential outcome is separately linear in X. This relaxation of conventional linearity offers a common justification for all of these methods and avoids the weighting problem, at an efficiency cost that will be small when there are few covariates relative to sample size.

Suggested Citation

  • Chad Hazlett & Tanvi Shinkre, 2024. "Demystifying and avoiding the OLS "weighting problem": Unmodeled heterogeneity and straightforward solutions," Papers 2403.03299, arXiv.org, revised Nov 2024.
  • Handle: RePEc:arx:papers:2403.03299
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2403.03299
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Hoffmann, Nathan Isaac, 2023. "Double Robust, Flexible Adjustment Methods for Causal Inference: An Overview and an Evaluation," SocArXiv dzayg, Center for Open Science.
    2. repec:cup:apsrev:v:113:y:2019:i:03:p:838-859_00 is not listed on IDEAS
    3. Victor Chernozhukov & Iván Fernández‐Val & Jinyong Hahn & Whitney Newey, 2013. "Average and Quantile Effects in Nonseparable Panel Models," Econometrica, Econometric Society, vol. 81(2), pages 535-580, March.
    4. Joshua D. Angrist & Jörn-Steffen Pischke, 2009. "Mostly Harmless Econometrics: An Empiricist's Companion," Economics Books, Princeton University Press, edition 1, number 8769.
    5. Peter M. Aronow & Cyrus Samii, 2016. "Does Regression Produce Representative Estimates of Causal Effects?," American Journal of Political Science, John Wiley & Sons, vol. 60(1), pages 250-267, January.
    6. Blair, Graeme & Cooper, Jasper & Coppock, Alexander & Humphreys, Macartan, 2019. "Declaring and Diagnosing Research Designs," American Political Science Review, Cambridge University Press, vol. 113(3), pages 838-859, August.
    7. Alberto Abadie & Guido W. Imbens, 2006. "Large Sample Properties of Matching Estimators for Average Treatment Effects," Econometrica, Econometric Society, vol. 74(1), pages 235-267, January.
    8. Ambarish Chattopadhyay & José R Zubizarreta, 2023. "On the implied weights of linear regression for causal inference," Biometrika, Biometrika Trust, vol. 110(3), pages 615-629.
    9. Blair, Graeme & Cooper, Jasper & Coppock, Alexander & Humphreys, Macartan, 2019. "Declaring and Diagnosing Research Designs," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 113(3), pages 838-859.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Sloczynski, Tymon, 2018. "A General Weighted Average Representation of the Ordinary and Two-Stage Least Squares Estimands," IZA Discussion Papers 11866, Institute of Labor Economics (IZA).
    2. Tymon S{l}oczy'nski, 2018. "Interpreting OLS Estimands When Treatment Effects Are Heterogeneous: Smaller Groups Get Larger Weights," Papers 1810.01576, arXiv.org, revised May 2020.
    3. Słoczyński, Tymon, 2012. "New Evidence on Linear Regression and Treatment Effect Heterogeneity," MPRA Paper 39524, University Library of Munich, Germany.
    4. Sant’Anna, Pedro H.C. & Zhao, Jun, 2020. "Doubly robust difference-in-differences estimators," Journal of Econometrics, Elsevier, vol. 219(1), pages 101-122.
    5. Caballero, Julián, 2021. "Corporate dollar debt and depreciations: All’s well that ends well?," Journal of Banking & Finance, Elsevier, vol. 130(C).
    6. Tymon Słoczyński, 2022. "Interpreting OLS Estimands When Treatment Effects Are Heterogeneous: Smaller Groups Get Larger Weights," The Review of Economics and Statistics, MIT Press, vol. 104(3), pages 501-509, May.
    7. Baccini, Leonardo & Impullitti, Giammario & Malesky, Edmund J., 2019. "Globalization and state capitalism: Assessing Vietnam's accession to the WTO," Journal of International Economics, Elsevier, vol. 119(C), pages 75-92.
    8. Heissel, Jennifer, 2016. "The relative benefits of live versus online delivery: Evidence from virtual algebra I in North Carolina," Economics of Education Review, Elsevier, vol. 53(C), pages 99-115.
    9. Giovanni Marin & Marianna Marino & Claudia Pellegrin, 2018. "The Impact of the European Emission Trading Scheme on Multiple Measures of Economic Performance," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 71(2), pages 551-582, October.
    10. Mano, Yukichi & Akoten, John & Yoshino, Yutaka & Sonobe, Tetsushi, 2014. "Teaching KAIZEN to small business owners: An experiment in a metalworking cluster in Nairobi," Journal of the Japanese and International Economies, Elsevier, vol. 33(C), pages 25-42.
    11. Marc F. Bellemare & Lindsey Novak, 2017. "Contract Farming and Food Security," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 99(2), pages 357-378.
    12. Jones, Kelly W. & Muñoz Brenes, Carlos L. & Shinbrot, Xoco A. & López-Báez, Walter & Rivera-Castañeda, Andrómeda, 2018. "The influence of cash and technical assistance on household-level outcomes in payments for hydrological services programs in Chiapas, Mexico," Ecosystem Services, Elsevier, vol. 31(PA), pages 208-218.
    13. Yihui He & Fang Han, 2023. "On propensity score matching with a diverging number of matches," Papers 2310.14142, arXiv.org, revised Nov 2023.
    14. Itzhak Ben-DAVID & Francesco A. FRANZONI & Rabih MOUSSAWI & John SEDUNOV III, 2015. "The Granular Nature of Large Institutional Investors," Swiss Finance Institute Research Paper Series 15-67, Swiss Finance Institute, revised Apr 2016.
    15. Peter Hull & Michal Kolesár & Christopher Walters, 2022. "Labor by design: contributions of David Card, Joshua Angrist, and Guido Imbens," Scandinavian Journal of Economics, Wiley Blackwell, vol. 124(3), pages 603-645, July.
    16. Brodeur, Abel & Esterling, Kevin & Ankel-Peters, Jörg & Bueno, Natália S & Desposato, Scott & Dreber, Anna & Genovese, Federica & Green, Donald P & Hepplewhite, Matthew & de la Guardia, Fernando Hoces, 2024. "Promoting Reproducibility and Replicability in Political Science," Department of Economics, Working Paper Series qt23n3n3dg, Department of Economics, Institute for Business and Economic Research, UC Berkeley.
    17. McKenzie, David & Mohpal, Aakash & Yang, Dean, 2022. "Aspirations and financial decisions: Experimental evidence from the Philippines," Journal of Development Economics, Elsevier, vol. 156(C).
    18. Charles Angelucci & Julia Cagé & Michael Sinkinson, 2024. "Media Competition and News Diets," American Economic Journal: Microeconomics, American Economic Association, vol. 16(2), pages 62-102, May.
    19. Guido W. Imbens, 2015. "Matching Methods in Practice: Three Examples," Journal of Human Resources, University of Wisconsin Press, vol. 50(2), pages 373-419.
    20. Busu, Mihail & Caraiani, Petre & Hadad, Shahrazad & Incze, Cynthia Bianka & Vargas, Madalina Vanesa, 2021. "The performance of publicly funded startups in Romania," Economic Systems, Elsevier, vol. 45(3).

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2403.03299. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.