IDEAS home Printed from https://ideas.repec.org/a/nas/journl/v119y2022pe2203150119.html
   My bibliography  Save this article

Observing many researchers using the same data and hypothesis reveals a hidden universe of uncertainty

Author

Listed:
  • Nate Breznau

    (a Research Center on Inequality and Social Policy (SOCIUM), University of Bremen, Bremen, 28359, Germany;)

  • Eike Mark Rinke

    (b School of Politics and International Studies, University of Leeds, Leeds, LS2 9JT, United Kingdom;)

  • Alexander Wuttke

    (l Department of Political Science, Ludwig Maximilian University, 80539 Munich, Germany;)

  • Hung H. V. Nguyen

    (a Research Center on Inequality and Social Policy (SOCIUM), University of Bremen, Bremen, 28359, Germany;; d Bremen International Graduate School of Social Sciences, 28359 Bremen, Germany;)

  • Muna Adem

    (e Department of Sociology, Indiana University, Bloomington, IN 47405;)

  • Jule Adriaans

    (f Socio-Economic Panel Study (SOEP), German Institute for Economic Research (DIW), 10117 Berlin, Germany;)

  • Amalia Alvarez-Benjumea

    (g Mechanisms of Normative Change, Max Planck Institute for Research on Collective Goods, 53113 Bonn, Germany;)

  • Henrik K. Andersen

    (h Institute of Sociology, Chemnitz University of Technology, 09126 Chemnitz, Germany;)

  • Daniel Auer

    (c Mannheim Centre for European Social Research, University of Mannheim, 68131 Mannheim, Germany;)

  • Flavio Azevedo

    (j Department of Psychology, University of Cambridge, Cambridge, CB23RQ, United Kingdom;)

  • Oke Bahnsen

    (i School of Social Sciences, University of Mannheim, 68159 Mannheim, Germany;)

  • Dave Balzer

    (k Institute of Sociology, Johannes Gutenberg University Mainz, 55128 Mainz, Germany;)

  • Gerrit Bauer

    (oooo Department of Sociology, Ludwig Maximilian University, 80801 Munich, Germany;)

  • Paul C. Bauer

    (c Mannheim Centre for European Social Research, University of Mannheim, 68131 Mannheim, Germany;)

  • Markus Baumann

    (m Heidelberg University, 69117 Heidelberg, Germany;; dd Institute for Political Science, Goethe University Frankfurt, 60323 Frankfurt, Germany;)

  • Sharon Baute

    (n Comparative Political Economy, University of Konstanz, 78457 Konstanz, Germany;)

  • Verena Benoit

    (l Department of Political Science, Ludwig Maximilian University, 80539 Munich, Germany;; o Faculty of Social Sciences, Economics, and Business Administration, University of Bamberg, 96052 Bamberg, Germany;)

  • Julian Bernauer

    (c Mannheim Centre for European Social Research, University of Mannheim, 68131 Mannheim, Germany;)

  • Carl Berning

    (pppp Institute for Political Science, Johannes Gutenberg University Mainz, 55099 Mainz, Germany;)

  • Anna Berthold

    (o Faculty of Social Sciences, Economics, and Business Administration, University of Bamberg, 96052 Bamberg, Germany;)

  • Felix S. Bethke

    (p Research Department on Intrastate Conflict, Peace Research Institute Frankfurt, 60329 Frankfurt, Germany;)

  • Thomas Biegert

    (q Department of Social Policy, London School of Economics and Political Science, London, WC2A 2AE, United Kingdom;)

  • Katharina Blinzler

    (r Survey Data Curation, Leibniz Institute for the Social Sciences (GESIS), 50667 Cologne, Germany;)

  • Johannes N. Blumenberg

    (rrrr Knowledge Exchange and Outreach, Leibniz Institute for the Social Sciences (GESIS), 68159 Mannheim, Germany;)

  • Licia Bobzien

    (s Jacques Delors Centre, Hertie School, 10117 Berlin, Germany;)

  • Andrea Bohman

    (t Department of Sociology, Umeå University, 90187 Umeå, Sweden;)

  • Thijs Bol

    (x Social Research Institute, Institute of Education, University College London, London, WC1H 0AL, United Kingdom;; uuuu Department of Sociology, University of Amsterdam, 1001 Amsterdam, The Netherlands;)

  • Amie Bostic

    (u Department of Sociology, The University of Texas Rio Grande Valley, Brownsville, TX 78520;)

  • Zuzanna Brzozowska

    (v Vienna Institute of Demography, Austrian Academy of Sciences, 1030 Vienna, Austria;; w Austrian National Public Health Institute, Gesundheit Österreich (GÖG), 1030 Vienna, Austria;)

  • Katharina Burgdorf

    (i School of Social Sciences, University of Mannheim, 68159 Mannheim, Germany;)

  • Kaspar Burger

    (vvvv Jacobs Center for Productive Youth, University of Zurich, 8050 Zurich, Switzerland;; y Department of Sociology, University of Zurich, 8050 Zurich, Switzerland;)

  • Kathrin B. Busch

    (z Independent researcher;)

  • Juan Carlos-Castillo

    (aa Department of Sociology, University of Chile, Santiago, 7800284, Chile;; mmmm Center for Social Conflict and Cohesion Studies (COES), Pontificia Universidad Católica de Chile, Santiago, 8331150, Chile;)

  • Nathan Chan

    (nnnn Department of Political Science and International Relations, Loyola Marymount University, Los Angeles, CA 90045;)

  • Pablo Christmann

    (ssss Data and Research on Society, Leibniz Institute for the Social Sciences, 68159 Mannheim, Germany;)

  • Roxanne Connelly

    (cc School of Social and Political Science, University of Edinburgh, Edinburgh, EH8 9LD, United Kingdom;)

  • Christian S. Czymara

    (qqqq Institute of Sociology, Goethe University Frankfurt, 60323 Frankfurt, Germany;)

  • Elena Damian

    (yyyy Lifestyle and Chronic Diseases, Epidemiology and Public Health, Sciensano, 1000 Brussels, Belgium;)

  • Alejandro Ecker

    (c Mannheim Centre for European Social Research, University of Mannheim, 68131 Mannheim, Germany;)

  • Achim Edelmann

    (ff Médialab, Sciences Po, 75007 Paris, France;)

  • Maureen A. Eger

    (t Department of Sociology, Umeå University, 90187 Umeå, Sweden;)

  • Simon Ellerbrock

    (c Mannheim Centre for European Social Research, University of Mannheim, 68131 Mannheim, Germany;; i School of Social Sciences, University of Mannheim, 68159 Mannheim, Germany;)

  • Anna Forke

    (z Independent researcher;)

  • Andrea Forster

    (pp Empirical Educational and Higher Education Research, Freie Universität Berlin, 14195 Berlin, Germany;)

  • Chris Gaasendam

    (ee Department of Sociology, Center for Sociological Research, KU Leuven, 3000 Leuven, Belgium;)

  • Konstantin Gavras

    (i School of Social Sciences, University of Mannheim, 68159 Mannheim, Germany;)

  • Vernon Gayle

    (cc School of Social and Political Science, University of Edinburgh, Edinburgh, EH8 9LD, United Kingdom;)

  • Theresa Gessler

    (hhhh Kulturwissenschaftliche Fakultät, European University Viadrina, 15230 Frankfurt (Oder), Germany;)

  • Timo Gnambs

    (gg Educational Measurement, Leibniz Institute for Educational Trajectories, 96047 Bamberg, Germany;)

  • Amélie Godefroidt

    (aaaaa Centre for Research on Peace and Development, KU Leuven, 3000 Leuven, Belgium;)

  • Max Grömping

    (hh School of Government and International Relations, Griffith University, Nathan, QLD, 4111, Australia;)

  • Martin Groß

    (ii Department of Sociology, University of Tübingen, 72074 Tübingen, Germany;)

  • Stefan Gruber

    (jj Max Planck Institute for Social Law and Social Policy, 80799 Munich, Germany;)

  • Tobias Gummer

    (ssss Data and Research on Society, Leibniz Institute for the Social Sciences, 68159 Mannheim, Germany;)

  • Andreas Hadjar

    (rrrrr Department of Social Sciences, University of Luxembourg, 4366 Esch-sur-Alzette, Luxembourg;; mm Wirtschafts- und Sozialwissenschaftliches Institut (WSI), Hans Böckler Foundation, 40474 Düsseldorf, Germany;; llll University of Fribourg, 1700 Fribourg, Switzerland;)

  • Jan Paul Heisig

    (mmmmm Research Group "Health and Social Inequality", Berlin Social Science Center (WZB), 10785 Berlin, Germany;)

  • Sebastian Hellmeier

    (nnnnn Transformations of Democracy Unit, Berlin Social Science Center (WZB), 10785 Berlin, Germany;)

  • Stefanie Heyne

    (c Mannheim Centre for European Social Research, University of Mannheim, 68131 Mannheim, Germany;)

  • Magdalena Hirsch

    (ooooo Research Unit Migration, Integration, Transnationalization, Berlin Social Science Center (WZB), 10785 Berlin, Germany;)

  • Mikael Hjerm

    (t Department of Sociology, Umeå University, 90187 Umeå, Sweden;)

  • Oshrat Hochman

    (ssss Data and Research on Society, Leibniz Institute for the Social Sciences, 68159 Mannheim, Germany;)

  • Andreas Hövermann

    (mm Wirtschafts- und Sozialwissenschaftliches Institut (WSI), Hans Böckler Foundation, 40474 Düsseldorf, Germany;; qq German Socio-Economic Panel Survey, 10117 Berlin, Germany;)

  • Sophia Hunger

    (ppppp Center for Civil Society Research, Berlin Social Science Center, 10785 Berlin, Germany;)

  • Christian Hunkler

    (nn Berlin Institute for Integration and Migration Research (BIM), Humboldt University Berlin, 10099 Berlin, Germany;)

  • Nora Huth

    (oo School of Human and Social Sciences, University of Wuppertal, 42119 Wuppertal, Germany;)

  • Zsófia S. Ignácz

    (qqqq Institute of Sociology, Goethe University Frankfurt, 60323 Frankfurt, Germany;)

  • Laura Jacobs

    (ll Department of Political Science, Université Libre de Bruxelles, 1050 Bruxelles, Belgium;)

  • Jannes Jacobsen

    (tt Zeppelin University, 88045 Friedrichshafen, Germany;; jjjj Cluster "Data-Methods-Monitoring", German Center for Integration and Migration Research (DeZIM),10117 Berlin, Germany;)

  • Bastian Jaeger

    (rr Department of Social Psychology, Tilburg University, 5037AB Tilburg, The Netherlands;)

  • Sebastian Jungkunz

    (iiiii Chair of Political Sociology, University of Bamberg, 96052 Bamberg, Germany;; lll Institute of Political Science, University of Münster, 48149 Münster, Germany;)

  • Nils Jungmann

    (r Survey Data Curation, Leibniz Institute for the Social Sciences (GESIS), 50667 Cologne, Germany;)

  • Mathias Kauff

    (uu Department of Psychology, Medical School Hamburg, 20457 Hamburg, Germany;)

  • Manuel Kleinert

    (uuu Institute of Sociology, Justus Liebig University of Giessen, 35394 Giessen, Germany;)

  • Julia Klinger

    (jjjjj Institute of Sociology and Social Psychology, University of Cologne, 50931 Cologne, Germany;)

  • Jan-Philipp Kolb

    (vv Federal Statistics Office Germany, Destatis, 65189 Wiesbaden, Germany;)

  • Marta Kołczyńska

    (ww Department of Research on Social and Institutional Transformations, Institute of Political Studies of the Polish Academy of Sciences, 00-625 Warsaw, Poland;)

  • John Kuk

    (xx Department of Political Science, University of Oklahoma, Norman, OK 73019;)

  • Katharina Kunißen

    (k Institute of Sociology, Johannes Gutenberg University Mainz, 55128 Mainz, Germany;)

  • Dafina Kurti Sinatra

    (z Independent researcher;)

  • Alexander Langenkamp

    (qqqq Institute of Sociology, Goethe University Frankfurt, 60323 Frankfurt, Germany;)

  • Philipp M. Lersch

    (ddddd Department of Social Sciences, Humboldt University Berlin, 10099 Berlin, Germany;)

  • Lea-Maria Löbel

    (f Socio-Economic Panel Study (SOEP), German Institute for Economic Research (DIW), 10117 Berlin, Germany;)

  • Philipp Lutscher

    (yy Department of Political Science, University of Oslo, 0851 Oslo, Norway;)

  • Matthias Mader

    (zz Department of Politics and Public Administration, University of Konstanz, 78457 Konstanz, Germany;)

  • Joan E. Madia

    (lllll Institute for the Evaluation of Public Policies, Fondazione Bruno Kessler, 38122 Trento, Italy;)

  • Natalia Malancu

    (bbb The Institute of Citizenship Studies (InCite), University of Geneva, 1205 Geneva, Switzerland;)

  • Luis Maldonado

    (ccc Instituto de Sociologia, Pontifical Catholic University of Chile, Santiago, 7820436, Chile;)

  • Helge Marahrens

    (e Department of Sociology, Indiana University, Bloomington, IN 47405;)

  • Nicole Martin

    (ddd Department of Politics, University of Manchester, Manchester, M19 2JS, United Kingdom;)

  • Paul Martinez

    (eee Department of Institutional Research, Western Governors University, Salt Lake City, UT 84107;)

  • Jochen Mayerl

    (h Institute of Sociology, Chemnitz University of Technology, 09126 Chemnitz, Germany;)

  • Oscar J. Mayorga

    (fff Department of Sociology, University of California, Los Angeles, CA 90095;)

  • Patricia McManus

    (e Department of Sociology, Indiana University, Bloomington, IN 47405;)

  • Kyle McWagner

    (bb Department of Political Science, The University of California, Irvine, CA 92617;)

  • Cecil Meeusen

    (ee Department of Sociology, Center for Sociological Research, KU Leuven, 3000 Leuven, Belgium;)

  • Daniel Meierrieks

    (ooooo Research Unit Migration, Integration, Transnationalization, Berlin Social Science Center (WZB), 10785 Berlin, Germany;)

  • Jonathan Mellon

    (ddd Department of Politics, University of Manchester, Manchester, M19 2JS, United Kingdom;)

  • Friedolin Merhout

    (ggg Department of Sociology and Centre for Social Data Science, University of Copenhagen, 1353 Copenhagen, Denmark;)

  • Samuel Merk

    (hhh Department of School Development, University of Education Karlsruhe, 76133 Karlsruhe, Germany;)

  • Daniel Meyer

    (kkkkk Department of Education and Social Sciences, University of Cologne, 50931 Cologne, Germany;)

  • Leticia Micheli

    (iii Department of Psychology III, Julius-Maximilians University Würzburg, 97070 Würzburg, Germany;)

  • Jonathan Mijs

    (jjj Department of Sociology, Boston University, Boston, MA 02215;)

  • Cristóbal Moya

    (kkk Faculty of Sociology, Bielefeld University, 33615 Bielefeld, Germany;)

  • Marcel Neunhoeffer

    (i School of Social Sciences, University of Mannheim, 68159 Mannheim, Germany;)

  • Daniel Nüst

    (hhhhh Department of Geosciences, University of Münster, 49149 Münster, Germany;)

  • Olav Nygård

    (mmm Division of Migration, Ethnicity and Society (REMESO), Linköping University, 60174 Linköping, Sweden;)

  • Fabian Ochsenfeld

    (nnn Administrative Headquarters, Max Planck Society, 80539 Berlin, Germany;)

  • Gunnar Otte

    (k Institute of Sociology, Johannes Gutenberg University Mainz, 55128 Mainz, Germany;)

  • Anna O. Pechenkina

    (ooo Department of Political Science, Utah State University, Logan, UT 84321;)

  • Christopher Prosser

    (ppp Department of Politics, International Relations and Philosophy, Royal Holloway University of London, London, TW20 0EX, United Kingdom;)

  • Louis Raes

    (fffff Department of Economics, Tilburg University, 5037AB Tilburg, The Netherlands;)

  • Kevin Ralston

    (cc School of Social and Political Science, University of Edinburgh, Edinburgh, EH8 9LD, United Kingdom;)

  • Miguel R. Ramos

    (qqq Department of Social Policy, Sociology and Criminology, University of Birmingham, Birmingham, B15 2TT, United Kingdom;)

  • Arne Roets

    (rrr Department of Developmental, Personality and Social Psychology, Ghent University, 9000 Ghent, Belgium;)

  • Jonathan Rogers

    (sss Division of Social Science, New York University Abu Dhabi, Abu Dhabi, 10276, United Arab Emirates;)

  • Guido Ropers

    (i School of Social Sciences, University of Mannheim, 68159 Mannheim, Germany;)

  • Robin Samuel

    (rrrrr Department of Social Sciences, University of Luxembourg, 4366 Esch-sur-Alzette, Luxembourg;)

  • Gregor Sand

    (jj Max Planck Institute for Social Law and Social Policy, 80799 Munich, Germany;)

  • Ariela Schachter

    (ttt Department of Sociology, Washington University in St. Louis, St. Louis, MO 63130;)

  • Merlin Schaeffer

    (qqqqq Department of Sociology, University of Copenhagen, 1353 Copenhagen, Denmark;)

  • David Schieferdecker

    (xxxx Institute for Media and Communication Studies, Freie Universität Berlin, 14195 Berlin, Germany;)

  • Elmar Schlueter

    (uuu Institute of Sociology, Justus Liebig University of Giessen, 35394 Giessen, Germany;)

  • Regine Schmidt

    (o Faculty of Social Sciences, Economics, and Business Administration, University of Bamberg, 96052 Bamberg, Germany;)

  • Katja M. Schmidt

    (f Socio-Economic Panel Study (SOEP), German Institute for Economic Research (DIW), 10117 Berlin, Germany;)

  • Alexander Schmidt-Catran

    (qqqq Institute of Sociology, Goethe University Frankfurt, 60323 Frankfurt, Germany;)

  • Claudia Schmiedeberg

    (oooo Department of Sociology, Ludwig Maximilian University, 80801 Munich, Germany;)

  • Jürgen Schneider

    (ccccc Tübingen School of Education, University of Tübingen, 72074 Tübingen, Germany;)

  • Martijn Schoonvelde

    (sssss Department of European Languages and Cultures, University of Groningen, 9712 EK Groningen, The Netherlands;)

  • Julia Schulte-Cloos

    (kkkk Robert Schuman Center for Advanced Studies, European University Institute, 50133 Florence, Italy;)

  • Sandy Schumann

    (wwww Department of Security and Crime Science, University College London, London,WC1E 6BT, United Kingdom;)

  • Reinhard Schunck

    (oo School of Human and Social Sciences, University of Wuppertal, 42119 Wuppertal, Germany;)

  • Jürgen Schupp

    (f Socio-Economic Panel Study (SOEP), German Institute for Economic Research (DIW), 10117 Berlin, Germany;)

  • Julian Seuring

    (bbbbb Department of Migration, Leibniz Institute for Educational Trajectories, 96047 Bamberg, Germany;)

  • Henning Silber

    (tttt Department of Survey Design and Methodology, Leibniz Institute for the Social Sciences (GESIS), 68159 Mannheim, Germany;)

  • Willem Sleegers

    (rr Department of Social Psychology, Tilburg University, 5037AB Tilburg, The Netherlands;)

  • Nico Sonntag

    (k Institute of Sociology, Johannes Gutenberg University Mainz, 55128 Mainz, Germany;)

  • Alexander Staudt

    (z Independent researcher;)

  • Nadia Steiber

    (www Department of Sociology, University of Vienna, 1090 Vienna, Austria;)

  • Nils Steiner

    (pppp Institute for Political Science, Johannes Gutenberg University Mainz, 55099 Mainz, Germany;)

  • Sebastian Sternberg

    (z Independent researcher;)

  • Dieter Stiers

    (zzzz Center for Political Science Research, KU Leuven, 3000 Leuven, Belgium;)

  • Dragana Stojmenovska

    (uuuu Department of Sociology, University of Amsterdam, 1001 Amsterdam, The Netherlands;)

  • Nora Storz

    (xxx Interdisciplinary Social Science, Utrecht University, 3584 Utrecht, The Netherlands;)

  • Erich Striessnig

    (ttttt Department of Demography, University of Vienna, 1010 Vienna, Austria;)

  • Anne-Kathrin Stroppe

    (r Survey Data Curation, Leibniz Institute for the Social Sciences (GESIS), 50667 Cologne, Germany;)

  • Janna Teltemann

    (yyy Institute for Social Sciences, University of Hildesheim, 31141 Hildesheim, Germany;)

  • Andrey Tibajev

    (mmm Division of Migration, Ethnicity and Society (REMESO), Linköping University, 60174 Linköping, Sweden;)

  • Brian Tung

    (ttt Department of Sociology, Washington University in St. Louis, St. Louis, MO 63130;)

  • Giacomo Vagni

    (x Social Research Institute, Institute of Education, University College London, London, WC1H 0AL, United Kingdom;)

  • Jasper Van Assche

    (eeeee Center for Social and Cultural Psychology, Université Libre de Bruxelles, 1050 Brussels, Belgium;)

  • Meta van der Linden

    (xxx Interdisciplinary Social Science, Utrecht University, 3584 Utrecht, The Netherlands;)

  • Jolanda van der Noll

    (zzz Department of Psychology, University of Hagen, 58097 Hagen, Germany;)

  • Arno Van Hootegem

    (ee Department of Sociology, Center for Sociological Research, KU Leuven, 3000 Leuven, Belgium;)

  • Stefan Vogtenhuber

    (uuuuu Education and Employment, Institute for Advanced Studies, University of Vienna, Vienna, 1080 Austria;)

  • Bogdan Voicu

    (aaaa Research Institute for Quality of Life, Romanian Academy, 010071 Bucharest, Romania;; bbbb Department of Sociology, Lucian Blaga University of Sibiu, 550024 Sibiu, Romania;)

  • Fieke Wagemans

    (vvvvv Policy Perspectives, Citizen Perspectives, and Behaviors, Netherlands Institute for Social Research, 2594 The Hague, The Netherlands;)

  • Nadja Wehl

    (dddd Research Cluster "The Politics of Inequality", University of Konstanz, 78464 Konstanz, Germany;)

  • Hannah Werner

    (zzzz Center for Political Science Research, KU Leuven, 3000 Leuven, Belgium;)

  • Brenton M. Wiernik

    (eeee Department of Psychology, University of South Florida, Tampa, FL 33620;)

  • Fabian Winter

    (g Mechanisms of Normative Change, Max Planck Institute for Research on Collective Goods, 53113 Bonn, Germany;)

  • Christof Wolf

    (wwwww President, Leibniz Institute for the Social Sciences (GESIS), 68159 Mannheim, Germany; i School of Social Sciences, University of Mannheim, 68159 Mannheim, Germany;)

  • Yuki Yamada

    (ffff Faculty of Arts and Science, Kyushu University, Fukuoka, 819-0395, Japan;)

  • Nan Zhang

    (c Mannheim Centre for European Social Research, University of Mannheim, 68131 Mannheim, Germany;)

  • Conrad Ziller

    (ggggg Department of Political Science, University of Duisburg-Essen, 47057 Duisburg, Germany;)

  • Stefan Zins

    (gggg Institute for Employment Research, Federal Employment Agency, 90478 Nuremberg, Germany;)

  • Tomasz Żółtak

    (ww Department of Research on Social and Institutional Transformations, Institute of Political Studies of the Polish Academy of Sciences, 00-625 Warsaw, Poland;)

Abstract

Will different researchers converge on similar findings when analyzing the same data? Seventy-three independent research teams used identical cross-country survey data to test a prominent social science hypothesis: that more immigration will reduce public support for government provision of social policies. Instead of convergence, teams’ results varied greatly, ranging from large negative to large positive effects of immigration on social policy support. The choices made by the research teams in designing their statistical tests explain very little of this variation; a hidden universe of uncertainty remains. Considering this variation, scientists, especially those working with the complexities of human societies and behavior, should exercise humility and strive to better account for the uncertainty in their work.

Suggested Citation

  • Nate Breznau & Eike Mark Rinke & Alexander Wuttke & Hung H. V. Nguyen & Muna Adem & Jule Adriaans & Amalia Alvarez-Benjumea & Henrik K. Andersen & Daniel Auer & Flavio Azevedo & Oke Bahnsen & Dave Bal, 2022. "Observing many researchers using the same data and hypothesis reveals a hidden universe of uncertainty," Proceedings of the National Academy of Sciences, Proceedings of the National Academy of Sciences, vol. 119(44), pages 2203150119-, November.
    • Handle: RePEc:nas:journl:v:119:y:2022:p:e2203150119
    as

    Download full text from publisher

    File URL: http://www.pnas.org/content/119/44/e2203150119.full
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Albert J. Menkveld & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neusüß & Michael Razen & Utz Weitzel & David Abad‐Díaz & Menachem (Meni) Abudy , 2024. "Nonstandard Errors," Journal of Finance, American Finance Association, vol. 79(3), pages 2339-2390, June.
      • Albert J. Menkveld & Anna Dreber & Félix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neusüss & Michael Razen & Utz Weitzel & Gunther Capelle-Blancard, 2021. "Non-Standard Errors," Documents de travail du Centre d'Economie de la Sorbonne 21033, Université Panthéon-Sorbonne (Paris 1), Centre d'Economie de la Sorbonne.
      • Albert J. Menkveld & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neussüs & Michael Razen & Utz Weitzel & Christian Brownlees & Javier Gil-Bazo, 2021. "Non-Standard Errors," Working Papers 1303, Barcelona School of Economics.
      • Menkveld, Albert J. & Dreber, Anna & Holzmeister, Felix & Huber, Jürgen & Johannesson, Magnus & Kirchler, Michael & Neusüss, Sebastian & Razen, Michael & Weitzel, Utz, 2021. "Non-standard errors," IWH Discussion Papers 11/2021, Halle Institute for Economic Research (IWH).
      • Albert J. Menkveld & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neussüs & Michael Razen & Utz Weitzel & Christian T. Brownlees & Javier Gil-Baz, 2021. "Non-standard errors," Economics Working Papers 1807, Department of Economics and Business, Universitat Pompeu Fabra.
      • Menkveld, Albert J. & Dreber, Anna & Holzmeister, Felix & Huber, Juergen & Johannesson, Magnus & Kirchler, Michael & Neusüss, Sebastian & Razen, Michael & Weitzel, Utz & Abad-Díaz, David & Abudy, Mena, 2021. "Non-Standard Errors," Working Papers 2021:17, Lund University, Department of Economics.
      • Albert J. et al. Menkveld, 2021. "Non-Standard Errors," CESifo Working Paper Series 9453, CESifo.
      • Albert J Menkveld & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neusüss & Michael Razen & Utz Weitzel & Gunther Capelle-Blancard & David Abad-Dí, 2021. "Non-Standard Errors," Post-Print halshs-03500882, HAL.
      • Menkveld, Albert J. & Dreber, Anna & Holzmeister, Felix & Huber, Juergen & Johannesson, Magnus & Hasse, Jean-Baptiste & e.a.,, 2023. "Non-Standard Errors," LIDAM Reprints LFIN 2023002, Université catholique de Louvain, Louvain Finance (LFIN).
      • Albert J. Menkveld & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neusüss & Michael Razen & Utz Weitzel & Edwin Baidoo & Michael Frömmel & et al, 2021. "Non-Standard Errors," Working Papers of Faculty of Economics and Business Administration, Ghent University, Belgium 21/1032, Ghent University, Faculty of Economics and Business Administration.
      • Francesco Franzoni & Roxana Mihet & Markus Leippold & Per Ostberg & Olivier Scaillet & Norman Schürhoff & Oksana Bashchenko & Nicola Mano & Michele Pelli, 2022. "Non-Standard Errors," Swiss Finance Institute Research Paper Series 22-09, Swiss Finance Institute.
      • Moinas, Sophie & Declerck, Fany & Menkveld, Albert J. & Dreber, Anna, 2023. "Non-Standard Errors," TSE Working Papers 23-1451, Toulouse School of Economics (TSE).
      • Menkveld, Albert J. & Dreber, Anna & Holzmeister, Felix & Huber, Juergen & Johannesson, Magnus & Kirchler, Michael & Neusüß, Sebastian & Razen, Michael & Weitzel, Utz & Abad-Díaz, David & Abudy, Menac, 2024. "Nonstandard errors," LSE Research Online Documents on Economics 123002, London School of Economics and Political Science, LSE Library.
      • Menkveld, A. & Dreber, A. & Holzmeister, F. & Huber, J. & Johannesson, M. & Kirchler, M. & Neusüss, S. & Razen, M. & Neusüss, S. & Neusüss, S., 2021. "Non-Standard Errors," Cambridge Working Papers in Economics 2182, Faculty of Economics, University of Cambridge.
      • Albert Menkveld & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neusüß & Michael Razen & Utz Weitzel & David Abad-Díaz & Tobias Adrian & Yacine Ai, 2024. "Nonstandard Errors," Post-Print hal-04676112, HAL.
      • Menkveld, Albert J. & Dreber, Anna & Holzmeister, Felix & Huber, Jürgen & Johannesson, Magnus & Kirchler, Michael & Neusüss, Sebastian & Razen, Michael & Weitzel, Utz, 2021. "Non-standard errors," SAFE Working Paper Series 327, Leibniz Institute for Financial Research SAFE.
      • Ferrara, Gerardo & Jurkatis, Simon, 2021. "Non-standard errors," Bank of England working papers 955, Bank of England.
      • Albert J. Menkveld & Anna Dreber & Felix Holzmeister & Jürgen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neusüss & Michael Razen & Utz Weitzel & David Abad-Dí­az & Menachem Abudy & Tobi, 2021. "Non-Standard Errors," Working Papers 2021-31, Faculty of Economics and Statistics, Universität Innsbruck.
      • Albert J Menkveld & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neusüss & Michael Razen & Utz Weitzel & Gunther Capelle-Blancard & David Abad-Dí, 2021. "Non-Standard Errors," Université Paris1 Panthéon-Sorbonne (Post-Print and Working Papers) halshs-03500882, HAL.
      • Menkveld, A. & Dreber, A. & Holzmeister, F. & Huber, J. & Johannesson, M. & Kirchler, M. & Neusüss, S. & Razen, M. & Neusüss, S. & Neusüss, S., 2021. "Non-Standard Errors," Janeway Institute Working Papers 2112, Faculty of Economics, University of Cambridge.
      • Wolff, Christian & Menkveld, Albert J. & Dreber, Anna & Holzmeister, Felix & Huber, Juergen & Johannesson, Magnus & Kirchler, Michael & Neusüess, Sebastian & Razen, Michael & Weitzel, Utz, 2021. "Non-Standard Errors," CEPR Discussion Papers 16751, C.E.P.R. Discussion Papers.
    2. James C. Garand & Ping Xu & Belinda C. Davis, 2017. "Immigration Attitudes and Support for the Welfare State in the American Mass Public," American Journal of Political Science, John Wiley & Sons, vol. 61(1), pages 146-162, January.
    3. James Alt & Torben Iversen, 2017. "Inequality, Labor Market Segmentation, and Preferences for Redistribution," American Journal of Political Science, John Wiley & Sons, vol. 61(1), pages 21-36, January.
    4. Muñoz, Jordi & Pardos-Prado, Sergi, 2019. "Immigration and Support for Social Policy: An Experimental Comparison of Universal and Means-Tested Programs," Political Science Research and Methods, Cambridge University Press, vol. 7(4), pages 717-735, October.
    5. Albert J. Menkveld & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neusüß & Michael Razen & Utz Weitzel & David Abad‐Díaz & Menachem (Meni) Abudy , 2024. "Nonstandard Errors," Journal of Finance, American Finance Association, vol. 79(3), pages 2339-2390, June.
      • Albert J. Menkveld & Anna Dreber & Felix Holzmeister & Jürgen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neusüss & Michael Razen & Utz Weitzel & David Abad-Dí­az & Menachem Abudy & Tobi, 2021. "Non-Standard Errors," Working Papers 2021-31, Faculty of Economics and Statistics, Universität Innsbruck.
      • Ciril Bosch-Rosa & Bernhard Kassner, 2023. "Non-Standard Errors," Rationality and Competition Discussion Paper Series 385, CRC TRR 190 Rationality and Competition.
      • Menkveld, Albert J. & Dreber, Anna & Holzmeister, Felix & Huber, Jürgen & Johannesson, Magnus & Kirchler, Michael & Neusüss, Sebastian & Razen, Michael & Weitzel, Utz, 2021. "Non-standard errors," IWH Discussion Papers 11/2021, Halle Institute for Economic Research (IWH).
      • Albert J. Menkveld & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neussüs & Michael Razen & Utz Weitzel & Christian T. Brownlees & Javier Gil-Baz, 2021. "Non-standard errors," Economics Working Papers 1807, Department of Economics and Business, Universitat Pompeu Fabra.
      • Menkveld, Albert J. & Dreber, Anna & Holzmeister, Felix & Huber, Juergen & Johannesson, Magnus & Kirchler, Michael & Neusüss, Sebastian & Razen, Michael & Weitzel, Utz & Abad-Díaz, David & Abudy, Mena, 2021. "Non-Standard Errors," Working Papers 2021:17, Lund University, Department of Economics.
      • Albert J. et al. Menkveld, 2021. "Non-Standard Errors," CESifo Working Paper Series 9453, CESifo.
      • Albert J. Menkveld & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neusüss & Michael Razen & Utz Weitzel & Edwin Baidoo & Michael Frömmel & et al, 2021. "Non-Standard Errors," Working Papers of Faculty of Economics and Business Administration, Ghent University, Belgium 21/1032, Ghent University, Faculty of Economics and Business Administration.
      • Menkveld, Albert J. & Dreber, Anna & Holzmeister, Felix & Huber, Juergen & Johannesson, Magnus & Kirchler, Michael & Neusüß, Sebastian & Razen, Michael & Weitzel, Utz & Abad-Díaz, David & Abudy, Menac, 2024. "Nonstandard errors," LSE Research Online Documents on Economics 123002, London School of Economics and Political Science, LSE Library.
      • Albert Menkveld & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neusüß & Michael Razen & Utz Weitzel & David Abad-Díaz & Tobias Adrian & Yacine Ai, 2024. "Nonstandard Errors," Post-Print hal-04676112, HAL.
      • Menkveld, A. & Dreber, A. & Holzmeister, F. & Huber, J. & Johannesson, M. & Kirchler, M. & Neusüss, S. & Razen, M. & Neusüss, S. & Neusüss, S., 2021. "Non-Standard Errors," Cambridge Working Papers in Economics 2182, Faculty of Economics, University of Cambridge.
    6. Brady, David & Finnigan, Ryan, 2014. "Does Immigration Undermine Public Support for Social Policy?," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 79(1), pages 17-42.
    7. Schweinsberg, Martin & Feldman, Michael & Staub, Nicola & van den Akker, Olmo R. & van Aert, Robbie C.M. & van Assen, Marcel A.L.M. & Liu, Yang & Althoff, Tim & Heer, Jeffrey & Kale, Alex & Mohamed, Z, 2021. "Same data, different conclusions: Radical dispersion in empirical results when independent analysts operationalize and test the same hypothesis," Organizational Behavior and Human Decision Processes, Elsevier, vol. 165(C), pages 228-249.
    8. Alberto Alesina & Elie Murard & Hillel Rapoport, 2021. "Immigration and preferences for redistribution in Europe1 [Goodbye Lenin (or not): the effect of communism on people’s preferences]," Journal of Economic Geography, Oxford University Press, vol. 21(6), pages 925-954.
    9. Amy Orben & Andrew K. Przybylski, 2019. "The association between adolescent well-being and digital technology use," Nature Human Behaviour, Nature, vol. 3(2), pages 173-182, February.
    10. Rotem Botvinik-Nezer & Felix Holzmeister & Colin F. Camerer & Anna Dreber & Juergen Huber & Magnus Johannesson & Michael Kirchler & Roni Iwanir & Jeanette A. Mumford & R. Alison Adcock & Paolo Avesani, 2020. "Variability in the analysis of a single neuroimaging dataset by many teams," Nature, Nature, vol. 582(7810), pages 84-88, June.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Yu, Bin, 2023. "What is uncertainty in today’s practice of data science?," Journal of Econometrics, Elsevier, vol. 237(1).
    2. Cantone, Giulio Giacomo & Tomaselli, Venera, 2023. "Multiversal Methods and Applications," MetaArXiv ukvw7, Center for Open Science.
    3. Christoph Huber & Anna Dreber & Jürgen Huber & Magnus Johannesson & Michael Kirchler & Utz Weitzel & Miguel Abellán & Xeniya Adayeva & Fehime Ceren Ay & Kai Barron & Zachariah Berry & Werner Bönte , 2023. "Competition and moral behavior: A meta-analysis of forty-five crowd-sourced experimental designs," Proceedings of the National Academy of Sciences, Proceedings of the National Academy of Sciences, vol. 120(23), pages 2215572120-, June.
    4. Felix Holzmeister & Magnus Johannesson & Robert Böhm & Anna Dreber & Jürgen Huber & Michael Kirchler, 2023. "Heterogeneity in effect size estimates: Empirical evidence and practical implications," Working Papers 2023-17, Faculty of Economics and Statistics, Universität Innsbruck.
    5. Cantone, Giulio Giacomo & Tomaselli, Venera, 2023. "A Multiversal Model of Vibration of Effects of the Equitable and Sustainable Well-Being (BES) on Fertility," MetaArXiv z5msx, Center for Open Science.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Elsner, Benjamin & Concannon, Jeff, 2020. "Immigration and Redistribution," IZA Discussion Papers 13676, Institute of Labor Economics (IZA).
    2. Christoph Huber & Christian König-Kersting & Matteo M. Marini, 2022. "Experimenting with Financial Professionals," Working Papers 2022-07, Faculty of Economics and Statistics, Universität Innsbruck, revised Jun 2024.
    3. Dreber, Anna & Johannesson, Magnus, 2023. "A framework for evaluating reproducibility and replicability in economics," Ruhr Economic Papers 1055, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
    4. Christophe Pérignon & Olivier Akmansoy & Christophe Hurlin & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johanneson & Michael Kirchler & Albert Menkveld & Michael Razen & Utz Weitzel, 2022. "Reproducibility of Empirical Results: Evidence from 1,000 Tests in Finance," Working Papers hal-03810013, HAL.
    5. Pascal Achard & Sigrid Suetens, 2023. "The causal effect of ethnic diversity on support for redistribution and the role of discrimination," Journal of Economic Surveys, Wiley Blackwell, vol. 37(5), pages 1678-1696, December.
    6. Dario Krpan & Jonathan E. Booth & Andreea Damien, 2023. "The positive–negative–competence (PNC) model of psychological responses to representations of robots," Nature Human Behaviour, Nature, vol. 7(11), pages 1933-1954, November.
    7. Fišar, Miloš & Greiner, Ben & Huber, Christoph & Katok, Elena & Ozkes, Ali & Management Science Reproducibility Collaboration, 2023. "Reproducibility in Management Science," Department for Strategy and Innovation Working Paper Series 03/2023, WU Vienna University of Economics and Business.
    8. Breznau, Nate & Rinke, Eike Mark & Wuttke, Alexander & Adem, Muna & Adriaans, Jule & Alvarez-Benjumea, Amalia & Andersen, Henrik Kenneth & Auer, Daniel & Azevedo, Flavio & Bahnsen, Oke, 2019. "The Crowdsourced Replication Initiative: Investigating Immigration and Social Policy Preferences. Executive Report," SocArXiv 6j9qb, Center for Open Science.
    9. Schweinsberg, Martin & Feldman, Michael & Staub, Nicola & van den Akker, Olmo R. & van Aert, Robbie C.M. & van Assen, Marcel A.L.M. & Liu, Yang & Althoff, Tim & Heer, Jeffrey & Kale, Alex & Mohamed, Z, 2021. "Same data, different conclusions: Radical dispersion in empirical results when independent analysts operationalize and test the same hypothesis," Organizational Behavior and Human Decision Processes, Elsevier, vol. 165(C), pages 228-249.
    10. Guillaume Coqueret, 2023. "Forking paths in financial economics," Papers 2401.08606, arXiv.org.
    11. Christoph Huber & Anna Dreber & Jürgen Huber & Magnus Johannesson & Michael Kirchler & Utz Weitzel & Miguel Abellán & Xeniya Adayeva & Fehime Ceren Ay & Kai Barron & Zachariah Berry & Werner Bönte , 2023. "Competition and moral behavior: A meta-analysis of forty-five crowd-sourced experimental designs," Proceedings of the National Academy of Sciences, Proceedings of the National Academy of Sciences, vol. 120(23), pages 2215572120-, June.
    12. Felix Holzmeister & Magnus Johannesson & Robert Böhm & Anna Dreber & Jürgen Huber & Michael Kirchler, 2023. "Heterogeneity in effect size estimates: Empirical evidence and practical implications," Working Papers 2023-17, Faculty of Economics and Statistics, Universität Innsbruck.
    13. JaeYoul Shin, 2019. "How Can we Achieve a Sustainable Redistributive Policy? Rethinking the Relationship Between Civic Engagement, Neighborhood Relationship and Labor Market Status," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 142(1), pages 343-362, February.
    14. Riccardo Bruni & Alessandro Gioffré & Maria Marino, 2022. ""In-group bias in preferences for redistribution: a survey experiment in Italy"," IREA Working Papers 202223, University of Barcelona, Research Institute of Applied Economics, revised Nov 2023.
    15. Stephen A. Gorman & Frank J. Fabozzi, 2023. "Alternative risk premium: specification noise," Journal of Asset Management, Palgrave Macmillan, vol. 24(6), pages 459-473, October.
    16. Breznau, Nate & Rinke, Eike Mark & Wuttke, Alexander & Adem, Muna & Adriaans, Jule & Alvarez-Benjumea, Amalia & Andersen, Henrik Kenneth & Auer, Daniel & Azevedo, Flavio & Bahnsen, Oke, 2021. "Observing Many Researchers using the Same Data and Hypothesis Reveals a Hidden Universe of Data Analysis," MetaArXiv cd5j9, Center for Open Science.
    17. Olivier Bargain & Victor Stephane & Jérôme Valette, 2022. "Another brick in the wall. Immigration and electoral preferences: Direct evidence from state ballots," Review of International Economics, Wiley Blackwell, vol. 30(5), pages 1452-1477, November.
    18. Omar Hegazi & Samer Alalalmeh & Ahmad Alfaresi & Soheil Dashtinezhad & Ahmed Bahada & Moyad Shahwan & Ammar Abdulrahman Jairoun & Tesleem K. Babalola & Haya Yasin, 2022. "Development, Validation, and Utilization of a Social Media Use and Mental Health Questionnaire among Middle Eastern and Western Adults: A Pilot Study from the UAE," IJERPH, MDPI, vol. 19(23), pages 1-18, November.
    19. Malte Luebker, 2019. "Can the Structure of Inequality Explain Fiscal Redistribution? Revisiting the Social Affinity Hypothesis," LIS Working papers 762, LIS Cross-National Data Center in Luxembourg.
    20. Nick Huntington‐Klein & Andreu Arenas & Emily Beam & Marco Bertoni & Jeffrey R. Bloem & Pralhad Burli & Naibin Chen & Paul Grieco & Godwin Ekpe & Todd Pugatch & Martin Saavedra & Yaniv Stopnitzky, 2021. "The influence of hidden researcher decisions in applied microeconomics," Economic Inquiry, Western Economic Association International, vol. 59(3), pages 944-960, July.

    More about this item

    Keywords

    metascience; many analysts; researcher degrees of freedom; analytical flexibility; immigration and policy preferences;
    All these keywords.

    JEL classification:

    • C1 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nas:journl:v:119:y:2022:p:e2203150119. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Eric Cain (email available below). General contact details of provider: http://www.pnas.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.