IDEAS home Printed from https://ideas.repec.org/a/bla/jorssa/v180y2017i4p967-1033.html
   My bibliography  Save this article

Beyond subjective and objective in statistics

Author

Listed:
  • Andrew Gelman
  • Christian Hennig

Abstract

Decisions in statistical data analysis are often justified, criticized or avoided by using concepts of objectivity and subjectivity. We argue that the words ‘objective’ and ‘subjective’ in statistics discourse are used in a mostly unhelpful way, and we propose to replace each of them with broader collections of attributes, with objectivity replaced by transparency, consensus, impartiality and correspondence to observable reality, and subjectivity replaced by awareness of multiple perspectives and context dependence. Together with stability, these make up a collection of virtues that we think is helpful in discussions of statistical foundations and practice. The advantage of these reformulations is that the replacement terms do not oppose each other and that they give more specific guidance about what statistical science strives to achieve. Instead of debating over whether a given statistical method is subjective or objective (or normatively debating the relative merits of subjectivity and objectivity in statistical practice), we can recognize desirable attributes such as transparency and acknowledgement of multiple perspectives as complementary goals. We demonstrate the implications of our proposal with recent applied examples from pharmacology, election polling and socio‐economic stratification. The aim of the paper is to push users and developers of statistical methods towards more effective use of diverse sources of information and more open acknowledgement of assumptions and goals.

Suggested Citation

  • Andrew Gelman & Christian Hennig, 2017. "Beyond subjective and objective in statistics," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 180(4), pages 967-1033, October.
  • Handle: RePEc:bla:jorssa:v:180:y:2017:i:4:p:967-1033
    DOI: 10.1111/rssa.12276
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/rssa.12276
    Download Restriction: no

    File URL: https://libkey.io/10.1111/rssa.12276?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Sander Greenland, 2005. "Multiple‐bias modelling for analysis of observational data," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 168(2), pages 267-306, March.
    2. Regina Nuzzo, 2014. "Scientific method: Statistical errors," Nature, Nature, vol. 506(7487), pages 150-152, February.
    3. Jan Hannig & Hari Iyer & Randy C. S. Lai & Thomas C. M. Lee, 2016. "Generalized Fiducial Inference: A Review and New Results," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 111(515), pages 1346-1361, July.
    4. Andrew Gelman, 2011. "Induction and Deduction in Baysian Data Analysis," Rationality, Markets and Morals, Frankfurt School Verlag, Frankfurt School of Finance & Management, vol. 2(43), September.
    5. Simon French, 2012. "Expert Judgment, Meta-analysis, and Participatory Risk Analysis," Decision Analysis, INFORMS, vol. 9(2), pages 119-127, June.
    6. repec:ucp:bkecon:9780226750248 is not listed on IDEAS
    7. Anne-Laure Boulesteix & Robert Hable & Sabine Lauer & Manuel J. A. Eugster, 2015. "A Statistical Framework for Hypothesis Testing in Real Data Comparison Studies," The American Statistician, Taylor & Francis Journals, vol. 69(3), pages 201-212, August.
    8. Jeremy E. Oakley & Anthony O'Hagan, 2004. "Probabilistic sensitivity analysis of complex models: a Bayesian approach," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 66(3), pages 751-769, August.
    9. Andrew Gelman, 2003. "A Bayesian Formulation of Exploratory Data Analysis and Goodness‐of‐fit Testing," International Statistical Review, International Statistical Institute, vol. 71(2), pages 369-382, August.
    10. P. G. Bissiri & C. C. Holmes & S. G. Walker, 2016. "A general framework for updating belief distributions," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 78(5), pages 1103-1130, November.
    11. Manuele Leonelli & James Smith, 2015. "Bayesian decision support for complex systems with many distributed experts," Annals of Operations Research, Springer, vol. 235(1), pages 517-542, December.
    12. Nicholas J. Cox, 2013. "Speaking Stata: Trimming to taste," Stata Journal, StataCorp LP, vol. 13(3), pages 640-666, September.
    13. Christian Hennig & Tim F. Liao, 2013. "How to find an appropriate clustering for mixed-type variables with application to socio-economic stratification," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 62(3), pages 309-369, May.
    14. repec:bla:econom:v:70:y:2003:i:277:p:73-97 is not listed on IDEAS
    15. Wang, Wei & Rothschild, David & Goel, Sharad & Gelman, Andrew, 2015. "Forecasting elections with non-representative polls," International Journal of Forecasting, Elsevier, vol. 31(3), pages 980-991.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Elena Grimaccia & Alessia Naccarato, 2019. "Food Insecurity Individual Experience: A Comparison of Economic and Social Characteristics of the Most Vulnerable Groups in the World," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 143(1), pages 391-410, May.
    2. Delbianco Fernando & Tohmé Fernando, 2023. "What is a relevant control?: An algorithmic proposal," Asociación Argentina de Economía Política: Working Papers 4643, Asociación Argentina de Economía Política.
    3. Mayo, Deborah, 2024. "Error statistics, Bayes-factor Tests and the Fallacy of Non-exhaustive Alternatives," OSF Preprints tmgqd, Center for Open Science.
    4. Wei-Chao Lin & Ching Chen, 2021. "Novel World University Rankings Combining Academic, Environmental and Resource Indicators," Sustainability, MDPI, vol. 13(24), pages 1-15, December.
    5. Jingjing He & Wei Wang & Min Huang & Shaohua Wang & Xuefei Guan, 2021. "Bayesian Inference under Small Sample Sizes Using General Noninformative Priors," Mathematics, MDPI, vol. 9(21), pages 1-20, November.
    6. John Deke & Mariel Finucane & Daniel Thal, "undated". "The BASIE (BAyeSian Interpretation of Estimates) Framework for Interpreting Findings from Impact Evaluations: A Practical Guide for Education Researchers," Mathematica Policy Research Reports 5a0d5dff375d42048799878be, Mathematica Policy Research.
    7. Dimitris Korobilis & Kenichi Shimizu, 2022. "Bayesian Approaches to Shrinkage and Sparse Estimation," Foundations and Trends(R) in Econometrics, now publishers, vol. 11(4), pages 230-354, June.
    8. Francesco De Pretis & Barbara Osimani, 2019. "New Insights in Computational Methods for Pharmacovigilance: E-Synthesis , a Bayesian Framework for Causal Assessment," IJERPH, MDPI, vol. 16(12), pages 1-19, June.
    9. Jan Hannig & Hari Iyer, 2022. "Testing for calibration discrepancy of reported likelihood ratios in forensic science," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 185(1), pages 267-301, January.
    10. Claudia Biancotti & Alfonso Rosolia & Giovanni Veronese & Robert Kirchner & Francois Mouriaux, 2021. "Covid-19 and official statistics: a wakeup call?," Questioni di Economia e Finanza (Occasional Papers) 605, Bank of Italy, Economic Research and International Relations Area.
    11. Brian H. MacGillivray, 2019. "Null Hypothesis Testing ≠ Scientific Inference: A Critique of the Shaky Premise at the Heart of the Science and Values Debate, and a Defense of Value‐Neutral Risk Assessment," Risk Analysis, John Wiley & Sons, vol. 39(7), pages 1520-1532, July.
    12. Taisei Sugiyama & Nicolas Schweighofer & Jun Izawa, 2023. "Reinforcement learning establishes a minimal metacognitive process to monitor and control motor learning performance," Nature Communications, Nature, vol. 14(1), pages 1-14, December.
    13. Helfgott, Ariella & Midgley, Gerald & Chaudhury, Abrar & Vervoort, Joost & Sova, Chase & Ryan, Alex, 2023. "Multi-level participation in integrative, systemic planning: The case of climate adaptation in Ghana," European Journal of Operational Research, Elsevier, vol. 309(3), pages 1201-1217.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jyotirmoy Sarkar, 2018. "Will P†Value Triumph over Abuses and Attacks?," Biostatistics and Biometrics Open Access Journal, Juniper Publishers Inc., vol. 7(4), pages 66-71, July.
    2. Lawrence C. McCandless & Sylvia Richardson & Nicky Best, 2012. "Adjustment for Missing Confounders Using External Validation Data and Propensity Scores," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 107(497), pages 40-51, March.
    3. Julian P. T. Higgins & Simon G. Thompson & David J. Spiegelhalter, 2009. "A re‐evaluation of random‐effects meta‐analysis," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 172(1), pages 137-159, January.
    4. S. Cucurachi & E. Borgonovo & R. Heijungs, 2016. "A Protocol for the Global Sensitivity Analysis of Impact Assessment Models in Life Cycle Assessment," Risk Analysis, John Wiley & Sons, vol. 36(2), pages 357-377, February.
    5. Haoying Wang & Guohui Wu, 2022. "Modeling discrete choices with large fine-scale spatial data: opportunities and challenges," Journal of Geographical Systems, Springer, vol. 24(3), pages 325-351, July.
    6. Jakub Bijak & Jason D. Hilton & Eric Silverman & Viet Dung Cao, 2013. "Reforging the Wedding Ring," Demographic Research, Max Planck Institute for Demographic Research, Rostock, Germany, vol. 29(27), pages 729-766.
    7. Acharki, Naoufal & Bertoncello, Antoine & Garnier, Josselin, 2023. "Robust prediction interval estimation for Gaussian processes by cross-validation method," Computational Statistics & Data Analysis, Elsevier, vol. 178(C).
    8. Fabio Canova & Christian Matthes, 2021. "Dealing with misspecification in structural macroeconometric models," Quantitative Economics, Econometric Society, vol. 12(2), pages 313-350, May.
    9. Xueping Chen & Yujie Gai & Xiaodi Wang, 2023. "A-optimal designs for non-parametric symmetrical global sensitivity analysis," Metrika: International Journal for Theoretical and Applied Statistics, Springer, vol. 86(2), pages 219-237, February.
    10. Matieyendou Lamboni, 2020. "Uncertainty quantification: a minimum variance unbiased (joint) estimator of the non-normalized Sobol’ indices," Statistical Papers, Springer, vol. 61(5), pages 1939-1970, October.
    11. Douglas E. Schaubel & Guanghui Wei, 2011. "Double Inverse-Weighted Estimation of Cumulative Treatment Effects Under Nonproportional Hazards and Dependent Censoring," Biometrics, The International Biometric Society, vol. 67(1), pages 29-38, March.
    12. Mark Richard & Jan Vecer, 2021. "Efficiency Testing of Prediction Markets: Martingale Approach, Likelihood Ratio and Bayes Factor Analysis," Risks, MDPI, vol. 9(2), pages 1-20, February.
    13. Carlos Díaz-Venegas, 2014. "Identifying the Confounders of Marginalization and Mortality in Mexico, 2003–2007," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 118(2), pages 851-875, September.
    14. Gonzalez, Jhonny & Moriarty, John & Palczewski, Jan, 2017. "Bayesian calibration and number of jump components in electricity spot price models," Energy Economics, Elsevier, vol. 65(C), pages 375-388.
    15. Crucinio, Francesca R. & De Bortoli, Valentin & Doucet, Arnaud & Johansen, Adam M., 2024. "Solving a class of Fredholm integral equations of the first kind via Wasserstein gradient flows," Stochastic Processes and their Applications, Elsevier, vol. 173(C).
    16. Isaac Corro Ramos & Maureen P. M. H. Rutten-van Mölken & Maiwenn J. Al, 2013. "The Role of Value-of-Information Analysis in a Health Care Research Priority Setting," Medical Decision Making, , vol. 33(4), pages 472-489, May.
    17. Smith, Simon C. & Timmermann, Allan & Zhu, Yinchu, 2019. "Variable selection in panel models with breaks," Journal of Econometrics, Elsevier, vol. 212(1), pages 323-344.
    18. Elvira Pelle & Roberta Pappadà, 2021. "A clustering procedure for mixed-type data to explore ego network typologies: an application to elderly people living alone in Italy," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 30(5), pages 1507-1533, December.
    19. Veiga, Sébastien Da & Marrel, Amandine, 2020. "Gaussian process regression with linear inequality constraints," Reliability Engineering and System Safety, Elsevier, vol. 195(C).
    20. Gunnarsson, Björn Rafn & vanden Broucke, Seppe & Baesens, Bart & Óskarsdóttir, María & Lemahieu, Wilfried, 2021. "Deep learning for credit scoring: Do or don’t?," European Journal of Operational Research, Elsevier, vol. 295(1), pages 292-305.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:jorssa:v:180:y:2017:i:4:p:967-1033. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://edirc.repec.org/data/rssssea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.