IDEAS home Printed from https://ideas.repec.org/a/wly/apecpp/v43y2021i3p1176-1191.html
   My bibliography  Save this article

Is There a Cult of Statistical Significance in Agricultural Economics?

Author

Listed:
  • Jens Rommel
  • Meike Weltin

Abstract

McCloskey and Ziliak analyzed two decades of econometric practice in the American Economic Review (AER). We review the arguments and develop a questionnaire, applying it to three Agricultural Economics journals and the AER. Statistical practice improved over time, but a greater focus on economic significance is still needed. Considering the power of tests and discussing the economic consequences of type I and type II error were rarely practiced. The AER and the American Journal of Agricultural Economics did not substantially differ in their performance. We discuss examples of statistical practice and conclude with implications for the publication process and teaching.

Suggested Citation

  • Jens Rommel & Meike Weltin, 2021. "Is There a Cult of Statistical Significance in Agricultural Economics?," Applied Economic Perspectives and Policy, John Wiley & Sons, vol. 43(3), pages 1176-1191, September.
    • Handle: RePEc:wly:apecpp:v:43:y:2021:i:3:p:1176-1191
    DOI: 10.1002/aepp.13050
    as

    Download full text from publisher

    File URL: https://doi.org/10.1002/aepp.13050
    Download Restriction: no
    File URL: https://libkey.io/10.1002/aepp.13050?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Lucija Muehlenbachs & Elisheba Spiller & Christopher Timmins, 2015. "The Housing Market Impacts of Shale Gas Development," American Economic Review, American Economic Association, vol. 105(12), pages 3633-3659, December.
    2. Abel Brodeur & Mathias Lé & Marc Sangnier & Yanos Zylberberg, 2016. "Star Wars: The Empirics Strike Back," American Economic Journal: Applied Economics, American Economic Association, vol. 8(1), pages 1-32, January.
    3. Martin B. Hackmann & Jonathan T. Kolstad & Amanda E. Kowalski, 2015. "Adverse Selection and an Individual Mandate: When Theory Meets Practice," American Economic Review, American Economic Association, vol. 105(3), pages 1030-1066, March.
    4. Marcella Alsan, 2015. "The Effect of the TseTse Fly on African Development," American Economic Review, American Economic Association, vol. 105(1), pages 382-410, January.
    5. Walter Krämer, 2011. "The Cult of Statistical Significance – What Economists Should and Should Not Do to Make their Data Talk," Schmollers Jahrbuch : Journal of Applied Social Science Studies / Zeitschrift für Wirtschafts- und Sozialwissenschaften, Duncker & Humblot, Berlin, vol. 131(3), pages 455-468.
    6. Timothy L. Cross & Gregory M. Perry, 1995. "Depreciation Patterns for Agricultural Machinery," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 77(1), pages 194-204.
    7. Esther Duflo, 2018. "Report of the Editor: American Economic Review," AEA Papers and Proceedings, American Economic Association, vol. 108, pages 636-651, May.
    8. Alfonso Flores-Lagunes & Troy Timko, 2015. "Does Participation in 4-H Improve Schooling Outcomes? Evidence from Florida," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 97(2), pages 414-434.
    9. Roland Herrmann & Ernst Berg & Stephan Dabbert & Siegfried Pöchtrager & Klaus Salhofer, 2011. "Going Beyond Impact Factors: A Survey‐based Journal Ranking by Agricultural Economists," Journal of Agricultural Economics, Wiley Blackwell, vol. 62(3), pages 710-732, September.
    10. Anil Rupasingha & Yongzheng Liu & Mark Partridge, 2015. "Rural Bound: Determinants of Metro to Non-Metro Migration in the United States," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 97(3), pages 680-700.
    11. Ziliak, Stephen T. & McCloskey, Deirdre N., 2004. "Size matters: the standard error of regressions in the American Economic Review," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 33(5), pages 527-546, November.
    12. Josef M. Broder & Rodney P. Deprey, 1985. "Monetary Returns to Bachelors and Masters Degrees in Agricultural Economics," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 67(3), pages 666-673.
    13. Ronald L. Wasserstein & Nicole A. Lazar, 2016. "The ASA's Statement on p -Values: Context, Process, and Purpose," The American Statistician, Taylor & Francis Journals, vol. 70(2), pages 129-133, May.
    14. John A. List & Azeem M. Shaikh & Yang Xu, 2019. "Multiple hypothesis testing in experimental economics," Experimental Economics, Springer;Economic Science Association, vol. 22(4), pages 773-793, December.
    15. Steven Tadelis & Florian Zettelmeyer, 2015. "Information Disclosure as a Matching Mechanism: Theory and Evidence from a Field Experiment," American Economic Review, American Economic Association, vol. 105(2), pages 886-905, February.
    16. Johannes Sauer & Uwe Latacz-Lohmann, 2015. "Investment, technical change and efficiency: empirical evidence from German dairy production," European Review of Agricultural Economics, Oxford University Press and the European Agricultural and Applied Economics Publications Foundation, vol. 42(1), pages 151-175.
    17. Stefano Castriota & Marco Delmastro, 2015. "The Economics of Collective Reputation: Evidence from the Wine Industry," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 97(2), pages 469-489.
    18. Stephen T. Ziliak, 2016. "Statistical significance and scientific misconduct: improving the style of the published research paper," Review of Social Economy, Taylor & Francis Journals, vol. 74(1), pages 83-97, March.
    19. Hirschauer Norbert & Mußhoff Oliver & Grüner Sven & Frey Ulrich & Theesfeld Insa & Wagner Peter, 2016. "Die Interpretation des p-Wertes – Grundsätzliche Missverständnisse," Journal of Economics and Statistics (Jahrbuecher fuer Nationaloekonomie und Statistik), De Gruyter, vol. 236(5), pages 557-575, October.
    20. Deirdre N. McCloskey & Stephen T. Ziliak, 1996. "The Standard Error of Regressions," Journal of Economic Literature, American Economic Association, vol. 34(1), pages 97-114, March.
    21. Dina Pomeranz, 2015. "No Taxation without Information: Deterrence and Self-Enforcement in the Value Added Tax," American Economic Review, American Economic Association, vol. 105(8), pages 2539-2569, August.
    22. Christopher Allen & David M A Mehler, 2019. "Open science challenges, benefits and tips in early career and beyond," PLOS Biology, Public Library of Science, vol. 17(5), pages 1-14, May.
    23. Adam Altmejd & Anna Dreber & Eskil Forsell & Juergen Huber & Taisuke Imai & Magnus Johannesson & Michael Kirchler & Gideon Nave & Colin Camerer, 2019. "Predicting the replicability of social science lab experiments," PLOS ONE, Public Library of Science, vol. 14(12), pages 1-18, December.
    24. Nicky J. Welton & Howard H. Z. Thom, 2015. "Value of Information," Medical Decision Making, , vol. 35(5), pages 564-566, July.
    25. Marco Costanigro & Oana Deselnicu & Stephan Kroll, 2015. "Food Beliefs: Elicitation, Estimation and Implications for Labeling Policy," Journal of Agricultural Economics, Wiley Blackwell, vol. 66(1), pages 108-128, February.
    26. Marcus R. Munafò & Brian A. Nosek & Dorothy V. M. Bishop & Katherine S. Button & Christopher D. Chambers & Nathalie Percie du Sert & Uri Simonsohn & Eric-Jan Wagenmakers & Jennifer J. Ware & John P. A, 2017. "A manifesto for reproducible science," Nature Human Behaviour, Nature, vol. 1(1), pages 1-9, January.
    27. Altman, Morris, 2004. "Statistical significance, path dependency, and the culture of journal publication," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 33(5), pages 651-663, November.
    28. Alessandro Bonanno & Rui Huang & Yizao Liu, 2015. "Editor's choice Simulating welfare effects of the European nutrition and health claims’ regulation: the Italian yogurt market," European Review of Agricultural Economics, Oxford University Press and the European Agricultural and Applied Economics Publications Foundation, vol. 42(3), pages 499-533.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Heckelei, Thomas & Huettel, Silke & Odening, Martin & Rommel, Jens, 2021. "The replicability crisis and the p-value debate – what are the consequences for the agricultural and food economics community?," Discussion Papers 316369, University of Bonn, Institute for Food and Resource Economics.
    2. Sarah A. Janzen & Jeffrey D. Michler, 2021. "Ulysses' pact or Ulysses' raft: Using pre‐analysis plans in experimental and nonexperimental research," Applied Economic Perspectives and Policy, John Wiley & Sons, vol. 43(4), pages 1286-1304, December.
    3. Vecchio, Riccardo, 2022. "Fostering the debate among scholars to support the advancement of knowledge in the food-related consumer research: A commentary," Economia agro-alimentare / Food Economy, Italian Society of Agri-food Economics/Società Italiana di Economia Agro-Alimentare (SIEA), vol. 24(2), September.
    4. Roland Azibo Balgah & Chama Theodore Ketuama & Martin Ngwa Ngwabie & Hynek Roubík, 2024. "Africa’s Energy Availability-Deficiency Paradox: Lessons from Small-scale Biogas Technology and Policy Implications," Environment, Development and Sustainability: A Multidisciplinary Approach to the Theory and Practice of Sustainable Development, Springer, vol. 26(11), pages 29695-29713, November.
    5. Heckelei, Thomas & Hüttel, Silke & Odening, Martin & Rommel, Jens, 2023. "The p-Value Debate and Statistical (Mal)practice – Implications for the Agricultural and Food Economics Community," German Journal of Agricultural Economics, Humboldt-Universitaet zu Berlin, Department for Agricultural Economics, vol. 72(01), January.
    6. Riccardo Vecchio, 2022. "Fostering the debate among scholars to support the advancement of knowledge in the food-related consumer research: A commentary," Economia agro-alimentare, FrancoAngeli Editore, vol. 24(2), pages 1-15.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Eszter Czibor & David Jimenez‐Gomez & John A. List, 2019. "The Dozen Things Experimental Economists Should Do (More of)," Southern Economic Journal, John Wiley & Sons, vol. 86(2), pages 371-432, October.
    2. Heckelei, Thomas & Huettel, Silke & Odening, Martin & Rommel, Jens, 2021. "The replicability crisis and the p-value debate – what are the consequences for the agricultural and food economics community?," Discussion Papers 316369, University of Bonn, Institute for Food and Resource Economics.
    3. Hirschauer Norbert & Grüner Sven & Mußhoff Oliver & Becker Claudia, 2019. "Twenty Steps Towards an Adequate Inferential Interpretation of p-Values in Econometrics," Journal of Economics and Statistics (Jahrbuecher fuer Nationaloekonomie und Statistik), De Gruyter, vol. 239(4), pages 703-721, August.
    4. Thibaut Arpinon & Romain Espinosa, 2023. "A practical guide to Registered Reports for economists," Journal of the Economic Science Association, Springer;Economic Science Association, vol. 9(1), pages 90-122, June.
    5. Thibaut Arpinon & Romain Espinosa, 2023. "A Practical Guide to Registered Reports for Economists," Post-Print halshs-03897719, HAL.
    6. Philipp Doerrenberg & Jan Schmitz, 2017. "Tax compliance and information provision. A field experiment with small firms," Journal of Behavioral Economics for Policy, Society for the Advancement of Behavioral Economics (SABE), vol. 1(1), pages 47-54, February.
    7. Bruns, Stephan B. & Asanov, Igor & Bode, Rasmus & Dunger, Melanie & Funk, Christoph & Hassan, Sherif M. & Hauschildt, Julia & Heinisch, Dominik & Kempa, Karol & König, Johannes & Lips, Johannes & Verb, 2019. "Reporting errors and biases in published empirical findings: Evidence from innovation research," Research Policy, Elsevier, vol. 48(9), pages 1-1.
    8. Gillitzer, Christian & Sinning, Mathias, 2020. "Nudging businesses to pay their taxes: Does timing matter?," Journal of Economic Behavior & Organization, Elsevier, vol. 169(C), pages 284-300.
    9. Thomas Mayer, 2012. "Ziliak and McCloskey's Criticisms of Significance Tests: An Assessment," Econ Journal Watch, Econ Journal Watch, vol. 9(3), pages 256-297, September.
    10. Hirschauer, Norbert & Grüner, Sven & Mußhoff, Oliver & Becker, Claudia & Jantsch, Antje, 2020. "Can p-values be meaningfully interpreted without random sampling?," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 14, pages 71-91.
    11. Jan-Emmanuel De Neve & Clément Imbert & Johannes Spinnewijn & Teodora Tsankova & Maarten Luts, 2021. "How to Improve Tax Compliance? Evidence from Population-Wide Experiments in Belgium," Journal of Political Economy, University of Chicago Press, vol. 129(5), pages 1425-1463.
    12. Stephen T. Ziliak & Deirdre N. McCloskey, 2013. "We Agree That Statistical Significance Proves Essentially Nothing: A Rejoinder to Thomas Mayer," Econ Journal Watch, Econ Journal Watch, vol. 10(1), pages 97-107, January.
    13. Hallsworth, Michael & List, John A. & Metcalfe, Robert D. & Vlaev, Ivo, 2017. "The behavioralist as tax collector: Using natural field experiments to enhance tax compliance," Journal of Public Economics, Elsevier, vol. 148(C), pages 14-31.
    14. Antinyan, Armenak & Asatryan, Zareh, 2019. "Nudging for tax compliance: A meta-analysis," ZEW Discussion Papers 19-055, ZEW - Leibniz Centre for European Economic Research.
    15. Stephan B. Bruns & David I. Stern, 2019. "Lag length selection and p-hacking in Granger causality testing: prevalence and performance of meta-regression models," Empirical Economics, Springer, vol. 56(3), pages 797-830, March.
    16. John D. Levendis, 2018. "Time Series Econometrics," Springer Texts in Business and Economics, Springer, number 978-3-319-98282-3, June.
    17. Altman, Morris, 2020. "A more scientific approach to applied economics: Reconstructing statistical, analytical significance, and correlation analysis," Economic Analysis and Policy, Elsevier, vol. 66(C), pages 315-324.
    18. Kim, Jae H. & Ji, Philip Inyeob, 2015. "Significance testing in empirical finance: A critical review and assessment," Journal of Empirical Finance, Elsevier, vol. 34(C), pages 1-14.
    19. Alexander Libman & Joachim Zweynert, 2014. "Ceremonial Science: The State of Russian Economics Seen Through the Lens of the Work of ‘Doctor of Science’ Candidates," Working Papers 337, Leibniz Institut für Ost- und Südosteuropaforschung (Institute for East and Southeast European Studies).
    20. Gunter, Ulrich & Önder, Irem & Smeral, Egon, 2019. "Scientific value of econometric tourism demand studies," Annals of Tourism Research, Elsevier, vol. 78(C), pages 1-1.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wly:apecpp:v:43:y:2021:i:3:p:1176-1191. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://doi.org/10.1002/(ISSN)2040-5804 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.