IDEAS home Printed from https://ideas.repec.org/a/sae/somere/v48y2019i2p431-447.html
   My bibliography  Save this article

The Difference Between Causal Analysis and Predictive Models: Response to “Comment on Young and Holsteen (2017)â€

Author

Listed:
  • Cristobal Young

Abstract

The commenter’s proposal may be a reasonable method for addressing uncertainty in predictive modeling, where the goal is to predict y . In a treatment effects framework, where the goal is causal inference by conditioning-on-observables, the commenter’s proposal is deeply flawed. The proposal (1) ignores the definition of omitted-variable bias, thus systematically omitting critical kinds of controls; (2) assumes for convenience there are no bad controls in the model space, thus waving off the premise of model uncertainty; and (3) deletes virtually all alternative models to select a single model with the highest R 2 . Rather than showing what model assumptions are necessary to support one’s preferred results, this proposal favors biased parameter estimates and deletes alternative results before anyone has a chance to see them. In a treatment effects framework, this is not model robustness analysis but simply biased model selection.

Suggested Citation

  • Cristobal Young, 2019. "The Difference Between Causal Analysis and Predictive Models: Response to “Comment on Young and Holsteen (2017)â€," Sociological Methods & Research, , vol. 48(2), pages 431-447, May.
    • Handle: RePEc:sae:somere:v:48:y:2019:i:2:p:431-447
    DOI: 10.1177/0049124118782542
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/0049124118782542
    Download Restriction: no
    File URL: https://libkey.io/10.1177/0049124118782542?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Caplin, Andrew & Schotter, Andrew, 2008. "The Foundations of Positive and Normative Economics: A Handbook," OUP Catalogue, Oxford University Press, number 9780195328318.
    2. Alexandre Belloni & Victor Chernozhukov & Christian Hansen, 2014. "Inference on Treatment Effects after Selection among High-Dimensional Controlsâ€," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 81(2), pages 608-650.
    3. Acharya, Avidit & Blackwell, Matthew & Sen, Maya, 2016. "Explaining Causal Findings Without Bias: Detecting and Assessing Direct Effects," American Political Science Review, Cambridge University Press, vol. 110(3), pages 512-529, August.
    4. Ley, Eduardo & Steel, Mark F.J., 2007. "Jointness in Bayesian variable selection with applications to growth regression," Journal of Macroeconomics, Elsevier, vol. 29(3), pages 476-493, September.
    5. Jacob M. Montgomery & Brendan Nyhan & Michelle Torres, 2018. "How Conditioning on Posttreatment Variables Can Ruin Your Experiment and What to Do about It," American Journal of Political Science, John Wiley & Sons, vol. 62(3), pages 760-775, July.
    6. Jennifer Kane & S. Morgan & Kathleen Harris & David Guilkey, 2013. "The Educational Consequences of Teen Childbearing," Demography, Springer;Population Association of America (PAA), vol. 50(6), pages 2129-2150, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Cantone, Giulio Giacomo & Tomaselli, Venera, 2024. "On the Coherence of Composite Indexes: Multiversal Model and Specification Analysis for an Index of Well-Being," MetaArXiv d5y26, Center for Open Science.
    2. Cantone, Giulio Giacomo, 2023. "The multiversal methodology as a remedy of the replication crisis," MetaArXiv kuhmz, Center for Open Science.
    3. Humphreys, John M. & Srygley, Robert B. & Lawton, Douglas & Hudson, Amy R. & Branson, David H., 2022. "Grasshoppers exhibit asynchrony and spatial non-stationarity in response to the El Niño/Southern and Pacific Decadal Oscillations," Ecological Modelling, Elsevier, vol. 471(C).
    4. Verhagen, Mark D., 2021. "A Pragmatist's Guide to Prediction in the Social Sciences," SocArXiv tjkcy, Center for Open Science.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Christoph Dworschak, 2024. "Bias mitigation in empirical peace and conflict studies: A short primer on posttreatment variables," Journal of Peace Research, Peace Research Institute Oslo, vol. 61(3), pages 462-476, May.
    2. Schuessler, Julian, 2024. "Causal analysis with observational data," OSF Preprints wam94, Center for Open Science.
    3. Benjamin Krick & Jonathan Petkun & Mara Revkin, 2023. "What Determines Military Legitimacy? Evidence from the Battle of Mosul in Iraq," HiCN Working Papers 402, Households in Conflict Network.
    4. Stefan Dimitriadis, 2024. "Bribery, insecurity, and firm performance: Evidence from the Boko Haram insurgency in Nigeria," Strategic Management Journal, Wiley Blackwell, vol. 45(6), pages 1061-1086, June.
    5. Ertefaie Ashkan & Asgharian Masoud & Stephens David A., 2018. "Variable Selection in Causal Inference using a Simultaneous Penalization Method," Journal of Causal Inference, De Gruyter, vol. 6(1), pages 1-16, March.
    6. Aguilar-Gomez, Sandra & Benshaul-Tolonen, Anja, 2023. "The evolution and persistence of women's roles: Evidence from the Gold Rush," Journal of Economic Behavior & Organization, Elsevier, vol. 215(C), pages 364-381.
    7. Humphreys, John M. & Srygley, Robert B. & Lawton, Douglas & Hudson, Amy R. & Branson, David H., 2022. "Grasshoppers exhibit asynchrony and spatial non-stationarity in response to the El Niño/Southern and Pacific Decadal Oscillations," Ecological Modelling, Elsevier, vol. 471(C).
    8. Guimbeau, Amanda & Ji, Xinde James & Long, Zi & Menon, Nidhiya, 2024. "Ocean salinity, early-life health, and adaptation," Journal of Environmental Economics and Management, Elsevier, vol. 125(C).
    9. Omar García-Ponce & Lauren E Young & Thomas Zeitzoff, 2023. "Anger and support for retribution in Mexico’s drug war," Journal of Peace Research, Peace Research Institute Oslo, vol. 60(2), pages 274-290, March.
    10. Laurel Wheeler & Robert Garlick & Eric Johnson & Patrick Shaw & Marissa Gargano, 2022. "LinkedIn(to) Job Opportunities: Experimental Evidence from Job Readiness Training," American Economic Journal: Applied Economics, American Economic Association, vol. 14(2), pages 101-125, April.
    11. S Anukriti & Catalina Herrera‐Almanza & Praveen K. Pathak & Mahesh Karra, 2020. "Curse of the Mummy‐ji: The Influence of Mothers‐in‐Law on Women in India†," American Journal of Agricultural Economics, John Wiley & Sons, vol. 102(5), pages 1328-1351, October.
    12. Alexandre Belloni & Victor Chernozhukov & Denis Chetverikov & Christian Hansen & Kengo Kato, 2018. "High-dimensional econometrics and regularized GMM," CeMMAP working papers CWP35/18, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    13. Alexandre Belloni & Victor Chernozhukov & Kengo Kato, 2019. "Valid Post-Selection Inference in High-Dimensional Approximately Sparse Quantile Regression Models," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 114(526), pages 749-758, April.
    14. Christopher P. Chambers & Federico Echenique & Eran Shmaya, 2014. "The Axiomatic Structure of Empirical Content," American Economic Review, American Economic Association, vol. 104(8), pages 2303-2319, August.
    15. Kyle Colangelo & Ying-Ying Lee, 2019. "Double debiased machine learning nonparametric inference with continuous treatments," CeMMAP working papers CWP72/19, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    16. Sant’Anna, Pedro H.C. & Zhao, Jun, 2020. "Doubly robust difference-in-differences estimators," Journal of Econometrics, Elsevier, vol. 219(1), pages 101-122.
    17. Schnellenbach, Jan, 2012. "Nudges and norms: On the political economy of soft paternalism," European Journal of Political Economy, Elsevier, vol. 28(2), pages 266-277.
    18. Mbassi, Christophe Martial & Messono, Omang Ombolo, 2023. "Historical technology and current economic development: Reassessing the nature of the relationship," Technological Forecasting and Social Change, Elsevier, vol. 195(C).
    19. Khanh Duong, 2024. "Is meritocracy just? New evidence from Boolean analysis and Machine learning," Journal of Computational Social Science, Springer, vol. 7(2), pages 1795-1821, October.
    20. Susan Athey & Guido W. Imbens & Stefan Wager, 2018. "Approximate residual balancing: debiased inference of average treatment effects in high dimensions," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 80(4), pages 597-623, September.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:somere:v:48:y:2019:i:2:p:431-447. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.