IDEAS home Printed from https://ideas.repec.org/a/sae/somere/v46y2017i1p3-40.html
   My bibliography  Save this article

Model Uncertainty and Robustness

Author

Listed:
  • Cristobal Young
  • Katherine Holsteen

Abstract

Model uncertainty is pervasive in social science. A key question is how robust empirical results are to sensible changes in model specification. We present a new approach and applied statistical software for computational multimodel analysis. Our approach proceeds in two steps: First, we estimate the modeling distribution of estimates across all combinations of possible controls as well as specified functional form issues, variable definitions, standard error calculations, and estimation commands. This allows analysts to present their core, preferred estimate in the context of a distribution of plausible estimates. Second, we develop a model influence analysis showing how each model ingredient affects the coefficient of interest. This shows which model assumptions, if any, are critical to obtaining an empirical result. We demonstrate the architecture and interpretation of multimodel analysis using data on the union wage premium, gender dynamics in mortgage lending, and tax flight migration among U.S. states. These illustrate how initial results can be strongly robust to alternative model specifications or remarkably dependent on a knife-edge specification.

Suggested Citation

  • Cristobal Young & Katherine Holsteen, 2017. "Model Uncertainty and Robustness," Sociological Methods & Research, , vol. 46(1), pages 3-40, January.
    • Handle: RePEc:sae:somere:v:46:y:2017:i:1:p:3-40
    DOI: 10.1177/0049124115610347
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/0049124115610347
    Download Restriction: no
    File URL: https://libkey.io/10.1177/0049124115610347?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Joshua D. Angrist & Alan B. Krueger, 2001. "Instrumental Variables and the Search for Identification: From Supply and Demand to Natural Experiments," Journal of Economic Perspectives, American Economic Association, vol. 15(4), pages 69-85, Fall.
    2. Antonio Ciccone & Marek Jarociński, 2010. "Determinants of Economic Growth: Will Data Tell?," American Economic Journal: Macroeconomics, American Economic Association, vol. 2(4), pages 222-246, October.
    3. Jennifer Montez & Robert Hummer & Mark Hayward, 2012. "Educational Attainment and Adult Mortality in the United States: A Systematic Analysis of Functional Form," Demography, Springer;Population Association of America (PAA), vol. 49(1), pages 315-336, February.
    4. Andrew Leigh, 2007. "How Closely Do Top Income Shares Track Other Measures of Inequality?," Economic Journal, Royal Economic Society, vol. 117(524), pages 619-633, November.
    5. Xavier Sala-I-Martin & Gernot Doppelhofer & Ronald I. Miller, 2004. "Determinants of Long-Term Growth: A Bayesian Averaging of Classical Estimates (BACE) Approach," American Economic Review, American Economic Association, vol. 94(4), pages 813-835, September.
    6. Conway, Karen Smith & Rork, Jonathan C., 2012. "No Country for Old Men (Or Women) — Do State Tax Policies Drive Away the Elderly?," National Tax Journal, National Tax Association;National Tax Journal, vol. 65(2), pages 313-356, June.
    7. Joshua Angrist & Alan Krueger, 2001. "Instrumental Variables and the Search for Identification: From Supply and Demand to Natural Experiments," Working Papers 834, Princeton University, Department of Economics, Industrial Relations Section..
    8. Henrik Jacobsen Kleven & Camille Landais & Emmanuel Saez, 2013. "Taxation and International Migration of Superstars: Evidence from the European Football Market," American Economic Review, American Economic Association, vol. 103(5), pages 1892-1924, August.
    9. Philippe Aghion & Steven Durlauf (ed.), 2005. "Handbook of Economic Growth," Handbook of Economic Growth, Elsevier, edition 1, volume 1, number 1.
    10. Young, Cristobal & Varner, Charles, 2011. "Millionaire Migration and State Taxation of Top Incomes: Evidence From a Natural Experiment," National Tax Journal, National Tax Association;National Tax Journal, vol. 64(2), pages 255-283, June.
    11. Ho, Daniel E. & Imai, Kosuke & King, Gary & Stuart, Elizabeth A., 2007. "Matching as Nonparametric Preprocessing for Reducing Model Dependence in Parametric Causal Inference," Political Analysis, Cambridge University Press, vol. 15(3), pages 199-236, July.
    12. Steven N. Durlauf & Chao Fu & Salvador Navarro, 2012. "Assumptions Matter: Model Uncertainty and the Deterrent Effect of Capital Punishment," American Economic Review, American Economic Association, vol. 102(3), pages 487-492, May.
    13. Jennifer Kane & S. Morgan & Kathleen Harris & David Guilkey, 2013. "The Educational Consequences of Teen Childbearing," Demography, Springer;Population Association of America (PAA), vol. 50(6), pages 2129-2150, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. James R. Brown & J Anthony Cookson & Rawley Heimer, 2014. "Legal Institutions, Credit Markets, and Economic Activity," Working Papers (Old Series) 1434, Federal Reserve Bank of Cleveland.
    2. Antonio Ciccone & Marek Jarociński, 2010. "Determinants of Economic Growth: Will Data Tell?," American Economic Journal: Macroeconomics, American Economic Association, vol. 2(4), pages 222-246, October.
    3. Mark F. J. Steel, 2020. "Model Averaging and Its Use in Economics," Journal of Economic Literature, American Economic Association, vol. 58(3), pages 644-719, September.
    4. Andros Kourtellos, 2011. "Modeling parameter heterogeneity in cross-country regression models," University of Cyprus Working Papers in Economics 11-2011, University of Cyprus Department of Economics.
    5. Liu, Hao, 2019. "The communication and European Regional economic growth: The interactive fixed effects approach," Economic Modelling, Elsevier, vol. 83(C), pages 299-311.
    6. Capolupo, Rosa, 2009. "The New Growth Theories and Their Empirics after Twenty Years," Economics - The Open-Access, Open-Assessment E-Journal (2007-2020), Kiel Institute for the World Economy (IfW Kiel), vol. 3, pages 1-72.
    7. Rockey, James & Temple, Jonathan, 2016. "Growth econometrics for agnostics and true believers," European Economic Review, Elsevier, vol. 81(C), pages 86-102.
    8. Stimpfle, Alexander & Stadelmann, David, 2015. "The Impact of Fundamental Development Factors on Different Income Groups: International Evidence," VfS Annual Conference 2015 (Muenster): Economic Development - Theory and Policy 113128, Verein für Socialpolitik / German Economic Association.
    9. Georg Duernecker & Moritz Meyer & Fernando Vega‐Redondo, 2022. "Trade openness and growth: A network‐based approach," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 37(6), pages 1182-1203, September.
    10. David R. Agrawal & Dirk Foremny, 2019. "Relocation of the Rich: Migration in Response to Top Tax Rate Changes from Spanish Reforms," The Review of Economics and Statistics, MIT Press, vol. 101(2), pages 214-232, May.
    11. Markus Eberhardt & Francis Teal, 2011. "Econometrics For Grumblers: A New Look At The Literature On Cross‐Country Growth Empirics," Journal of Economic Surveys, Wiley Blackwell, vol. 25(1), pages 109-155, February.
    12. Michael Jetter & Christopher F. Parmeter, 2016. "Uncovering the determinants of corruption," Working Papers 2016-02, University of Miami, Department of Economics.
    13. Amor, Thouraya Hadj & Nouira, Ridha & Rault, Christophe & Sova, Anamaria Diana, 2023. "Real exchange rate misalignments and economic growth in Tunisia: New evidence from a threshold analysis of asymmetric adjustments," The Quarterly Review of Economics and Finance, Elsevier, vol. 88(C), pages 215-227.
    14. M. Shahe Emran & Fenohasina Maret-Rakotondrazaka & Stephen C. Smith, 2014. "Education and Freedom of Choice: Evidence from Arranged Marriages in Vietnam," Journal of Development Studies, Taylor & Francis Journals, vol. 50(4), pages 481-501, April.
    15. Boateng, Elliot & Agbola, Frank W. & Mahmood, Amir, 2021. "Foreign aid volatility and economic growth in Sub-Saharan Africa: Does institutional quality matter?," Economic Modelling, Elsevier, vol. 96(C), pages 111-127.
    16. Giertz, Seth H. & Ramezani, Rasoul, 2018. "Taxes, Wage Capitalization and the Ability of States to Redistribute Income," GLO Discussion Paper Series 291, Global Labor Organization (GLO).
    17. K. Benkovskis & B. Bluhm & E. Bobeica & C. Osbat & S. Zeugner, 2020. "What drives export market shares? It depends! An empirical analysis using Bayesian model averaging," Empirical Economics, Springer, vol. 59(2), pages 817-869, August.
    18. Enrique Moral-Benito, 2012. "Determinants of Economic Growth: A Bayesian Panel Data Approach," The Review of Economics and Statistics, MIT Press, vol. 94(2), pages 566-579, May.
    19. Erik Hembre, 2022. "State income taxes and team performance," International Tax and Public Finance, Springer;International Institute of Public Finance, vol. 29(3), pages 704-725, June.
    20. Ulaşan, Bülent, 2012. "Cross-country growth empirics and model uncertainty: An overview," Economics - The Open-Access, Open-Assessment E-Journal (2007-2020), Kiel Institute for the World Economy (IfW Kiel), vol. 6, pages 1-69.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:somere:v:46:y:2017:i:1:p:3-40. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.