IDEAS home Printed from https://ideas.repec.org/a/sae/somere/v48y2019i2p431-447.html
   My bibliography  Save this article

The Difference Between Causal Analysis and Predictive Models: Response to “Comment on Young and Holsteen (2017)â€

Author

Listed:
  • Cristobal Young

Abstract

The commenter’s proposal may be a reasonable method for addressing uncertainty in predictive modeling, where the goal is to predict y . In a treatment effects framework, where the goal is causal inference by conditioning-on-observables, the commenter’s proposal is deeply flawed. The proposal (1) ignores the definition of omitted-variable bias, thus systematically omitting critical kinds of controls; (2) assumes for convenience there are no bad controls in the model space, thus waving off the premise of model uncertainty; and (3) deletes virtually all alternative models to select a single model with the highest R 2 . Rather than showing what model assumptions are necessary to support one’s preferred results, this proposal favors biased parameter estimates and deletes alternative results before anyone has a chance to see them. In a treatment effects framework, this is not model robustness analysis but simply biased model selection.

Suggested Citation

  • Cristobal Young, 2019. "The Difference Between Causal Analysis and Predictive Models: Response to “Comment on Young and Holsteen (2017)â€," Sociological Methods & Research, , vol. 48(2), pages 431-447, May.
    • Handle: RePEc:sae:somere:v:48:y:2019:i:2:p:431-447
    DOI: 10.1177/0049124118782542
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/0049124118782542
    Download Restriction: no
    File URL: https://libkey.io/10.1177/0049124118782542?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Caplin, Andrew & Schotter, Andrew, 2008. "The Foundations of Positive and Normative Economics: A Handbook," OUP Catalogue, Oxford University Press, number 9780195328318.
    2. Acharya, Avidit & Blackwell, Matthew & Sen, Maya, 2016. "Explaining Causal Findings Without Bias: Detecting and Assessing Direct Effects," American Political Science Review, Cambridge University Press, vol. 110(3), pages 512-529, August.
    3. Jennifer Kane & S. Morgan & Kathleen Harris & David Guilkey, 2013. "The Educational Consequences of Teen Childbearing," Demography, Springer;Population Association of America (PAA), vol. 50(6), pages 2129-2150, December.
    4. Ley, Eduardo & Steel, Mark F.J., 2007. "Jointness in Bayesian variable selection with applications to growth regression," Journal of Macroeconomics, Elsevier, vol. 29(3), pages 476-493, September.
    5. Jacob M. Montgomery & Brendan Nyhan & Michelle Torres, 2018. "How Conditioning on Posttreatment Variables Can Ruin Your Experiment and What to Do about It," American Journal of Political Science, John Wiley & Sons, vol. 62(3), pages 760-775, July.
    6. Alexandre Belloni & Victor Chernozhukov & Christian Hansen, 2014. "Inference on Treatment Effects after Selection among High-Dimensional Controlsâ€," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 81(2), pages 608-650.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Cantone, Giulio Giacomo & Tomaselli, Venera, 2024. "On the Coherence of Composite Indexes: Multiversal Model and Specification Analysis for an Index of Well-Being," MetaArXiv d5y26, Center for Open Science.
    2. Humphreys, John M. & Srygley, Robert B. & Lawton, Douglas & Hudson, Amy R. & Branson, David H., 2022. "Grasshoppers exhibit asynchrony and spatial non-stationarity in response to the El Niño/Southern and Pacific Decadal Oscillations," Ecological Modelling, Elsevier, vol. 471(C).
    3. Cantone, Giulio Giacomo, 2023. "The multiversal methodology as a remedy of the replication crisis," MetaArXiv kuhmz, Center for Open Science.
    4. Verhagen, Mark D., 2021. "A Pragmatist's Guide to Prediction in the Social Sciences," SocArXiv tjkcy, Center for Open Science.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Christoph Dworschak, 2024. "Bias mitigation in empirical peace and conflict studies: A short primer on posttreatment variables," Journal of Peace Research, Peace Research Institute Oslo, vol. 61(3), pages 462-476, May.
    2. Schuessler, Julian, 2024. "Causal analysis with observational data," OSF Preprints wam94, Center for Open Science.
    3. Benjamin Krick & Jonathan Petkun & Mara Revkin, 2023. "What Determines Military Legitimacy? Evidence from the Battle of Mosul in Iraq," HiCN Working Papers 402, Households in Conflict Network.
    4. Ertefaie Ashkan & Asgharian Masoud & Stephens David A., 2018. "Variable Selection in Causal Inference using a Simultaneous Penalization Method," Journal of Causal Inference, De Gruyter, vol. 6(1), pages 1-16, March.
    5. Humphreys, John M. & Srygley, Robert B. & Lawton, Douglas & Hudson, Amy R. & Branson, David H., 2022. "Grasshoppers exhibit asynchrony and spatial non-stationarity in response to the El Niño/Southern and Pacific Decadal Oscillations," Ecological Modelling, Elsevier, vol. 471(C).
    6. Guimbeau, Amanda & Ji, Xinde James & Long, Zi & Menon, Nidhiya, 2024. "Ocean salinity, early-life health, and adaptation," Journal of Environmental Economics and Management, Elsevier, vol. 125(C).
    7. Omar García-Ponce & Lauren E Young & Thomas Zeitzoff, 2023. "Anger and support for retribution in Mexico’s drug war," Journal of Peace Research, Peace Research Institute Oslo, vol. 60(2), pages 274-290, March.
    8. Laurel Wheeler & Robert Garlick & Eric Johnson & Patrick Shaw & Marissa Gargano, 2022. "LinkedIn(to) Job Opportunities: Experimental Evidence from Job Readiness Training," American Economic Journal: Applied Economics, American Economic Association, vol. 14(2), pages 101-125, April.
    9. Aguilar-Gomez, Sandra & Benshaul-Tolonen, Anja, 2023. "The evolution and persistence of women's roles: Evidence from the Gold Rush," Journal of Economic Behavior & Organization, Elsevier, vol. 215(C), pages 364-381.
    10. Alexandre Belloni & Victor Chernozhukov & Kengo Kato, 2019. "Valid Post-Selection Inference in High-Dimensional Approximately Sparse Quantile Regression Models," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 114(526), pages 749-758, April.
    11. Sant’Anna, Pedro H.C. & Zhao, Jun, 2020. "Doubly robust difference-in-differences estimators," Journal of Econometrics, Elsevier, vol. 219(1), pages 101-122.
    12. Schnellenbach, Jan, 2012. "Nudges and norms: On the political economy of soft paternalism," European Journal of Political Economy, Elsevier, vol. 28(2), pages 266-277.
    13. Khanh Duong, 2024. "Is meritocracy just? New evidence from Boolean analysis and Machine learning," Journal of Computational Social Science, Springer, vol. 7(2), pages 1795-1821, October.
    14. Chenchuan (Mark) Li & Ulrich K. Müller, 2021. "Linear regression with many controls of limited explanatory power," Quantitative Economics, Econometric Society, vol. 12(2), pages 405-442, May.
    15. Till Grüne-Yanoff, 2012. "Old wine in new casks: libertarian paternalism still violates liberal principles," Social Choice and Welfare, Springer;The Society for Social Choice and Welfare, vol. 38(4), pages 635-645, April.
    16. Andrzej Cieślik & Oleg Gurshev, 2021. "Factor Endowments, Economic Integration, Round-Tripping, and Inward FDI: Evidence from the Baltic Economies," JRFM, MDPI, vol. 14(8), pages 1-26, July.
    17. Hector Espinoza & Stefan Speckesser, 2019. "A Comparison of Earnings Related to Higher Level Vocational/Technical and Academic Education," National Institute of Economic and Social Research (NIESR) Discussion Papers 502, National Institute of Economic and Social Research.
    18. Faruk Gul & Wolfgang Pesendorfer, 2009. "A Comment on Bernheim's Appraisal of Neuroeconomics," American Economic Journal: Microeconomics, American Economic Association, vol. 1(2), pages 42-47, August.
    19. Emilio Depetris-Chauvin & Ömer Özak, 2020. "The origins of the division of labor in pre-industrial times," Journal of Economic Growth, Springer, vol. 25(3), pages 297-340, September.
    20. Guo, Jiaqi & Wang, Qiang & Li, Rongrong, 2024. "Can official development assistance promote renewable energy in sub-Saharan Africa countries? A matter of institutional transparency of recipient countries," Energy Policy, Elsevier, vol. 186(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:somere:v:48:y:2019:i:2:p:431-447. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.