IDEAS home Printed from https://ideas.repec.org/p/ags/gewi17/261998.html
   My bibliography  Save this paper

Is there a cult of statistical significance in Agricultural Economics?

Author

Listed:
  • Rommel, Jens
  • Weltin, Meike

Abstract

In an analysis of articles published in ten years of the American Economic Review, Deirdre McCloskey and Stephen Ziliak have shown that economists often fail to adequately distinguish economic and statistical significance. In this paper, we briefly review their arguments and develop a ten-item questionnaire on the statistical practice in the Agricultural Economics community. We apply our questionnaire to the 2015 volumes of the American Journal of Agricultural Economics, the European Review of Agricultural Economics, the Journal of Agricultural Economics, and the American Economic Review. We specifically focus on the “sizeless stare” and the negligence of economic significance. Our initial results indicate that there is room of improvement in statistical practice. Empirical papers rarely consider the power of statistical tests or run simulations. The economic consequences of estimation results are often not adequately addressed. We discuss the implications of our findings for the publication process and teaching in Agricultural Economics.

Suggested Citation

  • Rommel, Jens & Weltin, Meike, 2017. "Is there a cult of statistical significance in Agricultural Economics?," 57th Annual Conference, Weihenstephan, Germany, September 13-15, 2017 261998, German Association of Agricultural Economists (GEWISOLA).
    • Handle: RePEc:ags:gewi17:261998
    DOI: 10.22004/ag.econ.261998
    as

    Download full text from publisher

    File URL: https://ageconsearch.umn.edu/record/261998/files/Rommel_150.pdf
    Download Restriction: no
    File URL: https://ageconsearch.umn.edu/record/261998/files/Rommel_150.pdf?subformat=pdfa
    Download Restriction: no
    File URL: https://libkey.io/10.22004/ag.econ.261998?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Deirdre N. McCloskey & Stephen T. Ziliak, 1996. "The Standard Error of Regressions," Journal of Economic Literature, American Economic Association, vol. 34(1), pages 97-114, March.
    2. Stephen T. Ziliak & Deirdre N. McCloskey, 2004. "Size Matters: The Standard Error of Regressions in the American Economic Review," Econ Journal Watch, Econ Journal Watch, vol. 1(2), pages 331-358, August.
    3. John A. List & Azeem M. Shaikh & Yang Xu, 2019. "Multiple hypothesis testing in experimental economics," Experimental Economics, Springer;Economic Science Association, vol. 22(4), pages 773-793, December.
    4. Anil Rupasingha & Yongzheng Liu & Mark Partridge, 2015. "Rural Bound: Determinants of Metro to Non-Metro Migration in the United States," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 97(3), pages 680-700.
    5. Marco Costanigro & Oana Deselnicu & Stephan Kroll, 2015. "Food Beliefs: Elicitation, Estimation and Implications for Labeling Policy," Journal of Agricultural Economics, Wiley Blackwell, vol. 66(1), pages 108-128, February.
    6. Abel Brodeur & Mathias Lé & Marc Sangnier & Yanos Zylberberg, 2016. "Star Wars: The Empirics Strike Back," American Economic Journal: Applied Economics, American Economic Association, vol. 8(1), pages 1-32, January.
    7. Lucija Muehlenbachs & Elisheba Spiller & Christopher Timmins, 2015. "The Housing Market Impacts of Shale Gas Development," American Economic Review, American Economic Association, vol. 105(12), pages 3633-3659, December.
    8. Marcus R. Munafò & Brian A. Nosek & Dorothy V. M. Bishop & Katherine S. Button & Christopher D. Chambers & Nathalie Percie du Sert & Uri Simonsohn & Eric-Jan Wagenmakers & Jennifer J. Ware & John P. A, 2017. "A manifesto for reproducible science," Nature Human Behaviour, Nature, vol. 1(1), pages 1-9, January.
    9. Adam Altmejd & Anna Dreber & Eskil Forsell & Juergen Huber & Taisuke Imai & Magnus Johannesson & Michael Kirchler & Gideon Nave & Colin Camerer, 2019. "Predicting the replicability of social science lab experiments," PLOS ONE, Public Library of Science, vol. 14(12), pages 1-18, December.
    10. Dina Pomeranz, 2015. "No Taxation without Information: Deterrence and Self-Enforcement in the Value Added Tax," American Economic Review, American Economic Association, vol. 105(8), pages 2539-2569, August.
    11. Martin B. Hackmann & Jonathan T. Kolstad & Amanda E. Kowalski, 2015. "Adverse Selection and an Individual Mandate: When Theory Meets Practice," American Economic Review, American Economic Association, vol. 105(3), pages 1030-1066, March.
    12. Josef M. Broder & Rodney P. Deprey, 1985. "Monetary Returns to Bachelors and Masters Degrees in Agricultural Economics," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 67(3), pages 666-673.
    13. Marcella Alsan, 2015. "The Effect of the TseTse Fly on African Development," American Economic Review, American Economic Association, vol. 105(1), pages 382-410, January.
    14. Altman, Morris, 2004. "Statistical significance, path dependency, and the culture of journal publication," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 33(5), pages 651-663, November.
    15. Walter Krämer, 2011. "The Cult of Statistical Significance – What Economists Should and Should Not Do to Make their Data Talk," Schmollers Jahrbuch : Journal of Applied Social Science Studies / Zeitschrift für Wirtschafts- und Sozialwissenschaften, Duncker & Humblot, Berlin, vol. 131(3), pages 455-468.
    16. Timothy L. Cross & Gregory M. Perry, 1995. "Depreciation Patterns for Agricultural Machinery," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 77(1), pages 194-204.
    17. Ronald L. Wasserstein & Nicole A. Lazar, 2016. "The ASA's Statement on p -Values: Context, Process, and Purpose," The American Statistician, Taylor & Francis Journals, vol. 70(2), pages 129-133, May.
    18. Esther Duflo, 2018. "Report of the Editor: American Economic Review," AEA Papers and Proceedings, American Economic Association, vol. 108, pages 636-651, May.
    19. Steven Tadelis & Florian Zettelmeyer, 2015. "Information Disclosure as a Matching Mechanism: Theory and Evidence from a Field Experiment," American Economic Review, American Economic Association, vol. 105(2), pages 886-905, February.
    20. Christopher Allen & David M A Mehler, 2019. "Open science challenges, benefits and tips in early career and beyond," PLOS Biology, Public Library of Science, vol. 17(5), pages 1-14, May.
    21. Alessandro Bonanno & Rui Huang & Yizao Liu, 2015. "Editor's choice Simulating welfare effects of the European nutrition and health claims’ regulation: the Italian yogurt market," European Review of Agricultural Economics, Oxford University Press and the European Agricultural and Applied Economics Publications Foundation, vol. 42(3), pages 499-533.
    22. Johannes Sauer & Uwe Latacz-Lohmann, 2015. "Investment, technical change and efficiency: empirical evidence from German dairy production," European Review of Agricultural Economics, Oxford University Press and the European Agricultural and Applied Economics Publications Foundation, vol. 42(1), pages 151-175.
    23. Stefano Castriota & Marco Delmastro, 2015. "The Economics of Collective Reputation: Evidence from the Wine Industry," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 97(2), pages 469-489.
    24. Stephen T. Ziliak, 2016. "Statistical significance and scientific misconduct: improving the style of the published research paper," Review of Social Economy, Taylor & Francis Journals, vol. 74(1), pages 83-97, March.
    25. Alfonso Flores-Lagunes & Troy Timko, 2015. "Does Participation in 4-H Improve Schooling Outcomes? Evidence from Florida," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 97(2), pages 414-434.
    26. Hirschauer Norbert & Mußhoff Oliver & Grüner Sven & Frey Ulrich & Theesfeld Insa & Wagner Peter, 2016. "Die Interpretation des p-Wertes – Grundsätzliche Missverständnisse," Journal of Economics and Statistics (Jahrbuecher fuer Nationaloekonomie und Statistik), De Gruyter, vol. 236(5), pages 557-575, October.
    27. Nicky J. Welton & Howard H. Z. Thom, 2015. "Value of Information," Medical Decision Making, , vol. 35(5), pages 564-566, July.
    28. Roland Herrmann & Ernst Berg & Stephan Dabbert & Siegfried Pöchtrager & Klaus Salhofer, 2011. "Going Beyond Impact Factors: A Survey‐based Journal Ranking by Agricultural Economists," Journal of Agricultural Economics, Wiley Blackwell, vol. 62(3), pages 710-732, September.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Roland Azibo Balgah & Chama Theodore Ketuama & Martin Ngwa Ngwabie & Hynek Roubík, 2024. "Africa’s Energy Availability-Deficiency Paradox: Lessons from Small-scale Biogas Technology and Policy Implications," Environment, Development and Sustainability: A Multidisciplinary Approach to the Theory and Practice of Sustainable Development, Springer, vol. 26(11), pages 29695-29713, November.
    2. Riccardo Vecchio, 2022. "Fostering the debate among scholars to support the advancement of knowledge in the food-related consumer research: A commentary," Economia agro-alimentare, FrancoAngeli Editore, vol. 24(2), pages 1-15.
    3. Sarah A. Janzen & Jeffrey D. Michler, 2021. "Ulysses' pact or Ulysses' raft: Using pre‐analysis plans in experimental and nonexperimental research," Applied Economic Perspectives and Policy, John Wiley & Sons, vol. 43(4), pages 1286-1304, December.
    4. Heckelei, Thomas & Hüttel, Silke & Odening, Martin & Rommel, Jens, 2023. "The p-Value Debate and Statistical (Mal)practice – Implications for the Agricultural and Food Economics Community," German Journal of Agricultural Economics, Humboldt-Universitaet zu Berlin, Department for Agricultural Economics, vol. 72(01), January.
    5. Vecchio, Riccardo, 2022. "Fostering the debate among scholars to support the advancement of knowledge in the food-related consumer research: A commentary," Economia agro-alimentare / Food Economy, Italian Society of Agri-food Economics/Società Italiana di Economia Agro-Alimentare (SIEA), vol. 24(2), September.
    6. Heckelei, Thomas & Huettel, Silke & Odening, Martin & Rommel, Jens, 2021. "The replicability crisis and the p-value debate – what are the consequences for the agricultural and food economics community?," Discussion Papers 316369, University of Bonn, Institute for Food and Resource Economics.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Eszter Czibor & David Jimenez‐Gomez & John A. List, 2019. "The Dozen Things Experimental Economists Should Do (More of)," Southern Economic Journal, John Wiley & Sons, vol. 86(2), pages 371-432, October.
    2. Hirschauer Norbert & Grüner Sven & Mußhoff Oliver & Becker Claudia, 2019. "Twenty Steps Towards an Adequate Inferential Interpretation of p-Values in Econometrics," Journal of Economics and Statistics (Jahrbuecher fuer Nationaloekonomie und Statistik), De Gruyter, vol. 239(4), pages 703-721, August.
    3. Heckelei, Thomas & Huettel, Silke & Odening, Martin & Rommel, Jens, 2021. "The replicability crisis and the p-value debate – what are the consequences for the agricultural and food economics community?," Discussion Papers 316369, University of Bonn, Institute for Food and Resource Economics.
    4. Philipp Doerrenberg & Jan Schmitz, 2017. "Tax compliance and information provision. A field experiment with small firms," Journal of Behavioral Economics for Policy, Society for the Advancement of Behavioral Economics (SABE), vol. 1(1), pages 47-54, February.
    5. Thibaut Arpinon & Romain Espinosa, 2023. "A practical guide to Registered Reports for economists," Journal of the Economic Science Association, Springer;Economic Science Association, vol. 9(1), pages 90-122, June.
    6. Bruns, Stephan B. & Asanov, Igor & Bode, Rasmus & Dunger, Melanie & Funk, Christoph & Hassan, Sherif M. & Hauschildt, Julia & Heinisch, Dominik & Kempa, Karol & König, Johannes & Lips, Johannes & Verb, 2019. "Reporting errors and biases in published empirical findings: Evidence from innovation research," Research Policy, Elsevier, vol. 48(9), pages 1-1.
    7. Thibaut Arpinon & Romain Espinosa, 2023. "A Practical Guide to Registered Reports for Economists," Post-Print halshs-03897719, HAL.
    8. Alexander Libman & Joachim Zweynert, 2014. "Ceremonial Science: The State of Russian Economics Seen Through the Lens of the Work of ‘Doctor of Science’ Candidates," Working Papers 337, Leibniz Institut für Ost- und Südosteuropaforschung (Institute for East and Southeast European Studies).
    9. Gunter, Ulrich & Önder, Irem & Smeral, Egon, 2019. "Scientific value of econometric tourism demand studies," Annals of Tourism Research, Elsevier, vol. 78(C), pages 1-1.
    10. Gillitzer, Christian & Sinning, Mathias, 2020. "Nudging businesses to pay their taxes: Does timing matter?," Journal of Economic Behavior & Organization, Elsevier, vol. 169(C), pages 284-300.
    11. Hallsworth, Michael & List, John A. & Metcalfe, Robert D. & Vlaev, Ivo, 2017. "The behavioralist as tax collector: Using natural field experiments to enhance tax compliance," Journal of Public Economics, Elsevier, vol. 148(C), pages 14-31.
    12. Thomas Mayer, 2012. "Ziliak and McCloskey's Criticisms of Significance Tests: An Assessment," Econ Journal Watch, Econ Journal Watch, vol. 9(3), pages 256-297, September.
    13. Hirschauer, Norbert & Grüner, Sven & Mußhoff, Oliver & Becker, Claudia & Jantsch, Antje, 2020. "Can p-values be meaningfully interpreted without random sampling?," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 14, pages 71-91.
    14. Jan-Emmanuel De Neve & Clément Imbert & Johannes Spinnewijn & Teodora Tsankova & Maarten Luts, 2021. "How to Improve Tax Compliance? Evidence from Population-Wide Experiments in Belgium," Journal of Political Economy, University of Chicago Press, vol. 129(5), pages 1425-1463.
    15. Antinyan, Armenak & Asatryan, Zareh, 2019. "Nudging for tax compliance: A meta-analysis," ZEW Discussion Papers 19-055, ZEW - Leibniz Centre for European Economic Research.
    16. Stephen T. Ziliak & Deirdre N. McCloskey, 2013. "We Agree That Statistical Significance Proves Essentially Nothing: A Rejoinder to Thomas Mayer," Econ Journal Watch, Econ Journal Watch, vol. 10(1), pages 97-107, January.
    17. Libman, Alexander & Zweynert, Joachim, 2014. "Ceremonial science: The state of Russian economics seen through the lens of the work of ‘Doctor of Science’ candidates," Economic Systems, Elsevier, vol. 38(3), pages 360-378.
    18. Igor Asanov & Christoph Buehren & Panagiota Zacharodimou, 2020. "The power of experiments: How big is your n?," MAGKS Papers on Economics 202032, Philipps-Universität Marburg, Faculty of Business Administration and Economics, Department of Economics (Volkswirtschaftliche Abteilung).
    19. Stephan B. Bruns & David I. Stern, 2019. "Lag length selection and p-hacking in Granger causality testing: prevalence and performance of meta-regression models," Empirical Economics, Springer, vol. 56(3), pages 797-830, March.
    20. Antinyan, Armenak & Corazzini, Luca, 2021. "Money does it better! Economic incentives, nudging interventions and reusable shopping bags: Evidence from a natural field experiment," Cardiff Economics Working Papers E2021/29, Cardiff University, Cardiff Business School, Economics Section.

    More about this item

    Keywords

    Research Methods/ Statistical Methods;

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ags:gewi17:261998. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: AgEcon Search (email available below). General contact details of provider: https://edirc.repec.org/data/gewisea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.