IDEAS home Printed from https://ideas.repec.org/a/oup/indcch/v31y2022i2p576-590..html
   My bibliography  Save this article

Reconsidering macroeconomic policy prescriptions with meta-analysis
[Statistical nonsignificance in empirical economics]

Author

Listed:
  • Sebastian Gechert

Abstract

This paper investigates recent developments in meta-analysis, the tool to quantitatively synthesize research in a certain body of literature. After providing a brief overview on how to do a meta-analysis and discussing recent methodological advancements, I review applied contributions to the field of macroeconomics. It turns out that meta-analyses have often questioned the conventional wisdom and established new consensuses in fiscal, monetary, and labor market policies by uncovering substantial publication bias and unexpected determining factors in many bodies of literature—in particular those dominated by policy conclusions in the neoclassical tradition like minimum wages, central bank strategies, financial regulation and the relative effects of tax and spending policies.

Suggested Citation

  • Sebastian Gechert, 2022. "Reconsidering macroeconomic policy prescriptions with meta-analysis [Statistical nonsignificance in empirical economics]," Industrial and Corporate Change, Oxford University Press and the Associazione ICC, vol. 31(2), pages 576-590.
    • Handle: RePEc:oup:indcch:v:31:y:2022:i:2:p:576-590.
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1093/icc/dtac005
    Download Restriction: Access to full text is restricted to subscribers.
    ---><---

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Havranek, Tomas & Rusnak, Marek & Sokolova, Anna, 2017. "Habit formation in consumption: A meta-analysis," European Economic Review, Elsevier, vol. 95(C), pages 142-167.
    2. Alberto Abadie, 2020. "Statistical Nonsignificance in Empirical Economics," American Economic Review: Insights, American Economic Association, vol. 2(2), pages 193-208, June.
    3. Thomas Herndon & Michael Ash & Robert Pollin, 2014. "Does high public debt consistently stifle economic growth? A critique of Reinhart and Rogoff," Cambridge Journal of Economics, Cambridge Political Economy Society, vol. 38(2), pages 257-279.
    4. Card, David & Krueger, Alan B, 1994. "Minimum Wages and Employment: A Case Study of the Fast-Food Industry in New Jersey and Pennsylvania," American Economic Review, American Economic Association, vol. 84(4), pages 772-793, September.
    5. Dosi, G. & Pereira, M.C. & Roventini, A. & Virgillito, M.E., 2019. "What if supply-side policies are not enough? The perverse interaction of flexibility and austerity," Journal of Economic Behavior & Organization, Elsevier, vol. 162(C), pages 360-388.
    6. Barro, Robert J, 1974. "Are Government Bonds Net Wealth?," Journal of Political Economy, University of Chicago Press, vol. 82(6), pages 1095-1117, Nov.-Dec..
    7. Nazila Alinaghi & W. Robert Reed, 2021. "Taxes and Economic Growth in OECD Countries: A Meta-analysis," Public Finance Review, , vol. 49(1), pages 3-40, January.
    8. Philip Arestis & Malcolm Sawyer, 2004. "On the Effectiveness of Monetary Policy and of Fiscal Policy," Review of Social Economy, Taylor & Francis Journals, vol. 62(4), pages 441-463.
    9. Giovanni Dosi & Richard R. Nelson, 2000. "An Introduction to Evolutionary Theories in Economics," Chapters, in: Innovation, Organization and Economic Dynamics, chapter 11, pages 327-346, Edward Elgar Publishing.
    10. Adair Turner, 2010. "The crisis, conventional economic wisdom, and public policy -super-1," Industrial and Corporate Change, Oxford University Press and the Associazione ICC, vol. 19(5), pages 1317-1329, October.
    11. Sebastian Gechert & Tomas Havranek & Zuzana Irsova & Dominika Kolcunova, 2022. "Measuring Capital-Labor Substitution: The Importance of Method Choices and Publication Bias," Review of Economic Dynamics, Elsevier for the Society for Economic Dynamics, vol. 45, pages 55-82, July.
    12. Carmen M. Reinhart & Kenneth S. Rogoff, 2010. "Growth in a Time of Debt," American Economic Review, American Economic Association, vol. 100(2), pages 573-578, May.
    13. Havranek, Tomas & Horvath, Roman & Irsova, Zuzana & Rusnak, Marek, 2015. "Cross-country heterogeneity in intertemporal substitution," Journal of International Economics, Elsevier, vol. 96(1), pages 100-118.
    14. Furukawa, Chishio, 2019. "Publication Bias under Aggregation Frictions: Theory, Evidence, and a New Correction Method," EconStor Preprints 194798, ZBW - Leibniz Information Centre for Economics.
    15. Kluve, Jochen, 2010. "The effectiveness of European active labor market programs," Labour Economics, Elsevier, vol. 17(6), pages 904-918, December.
    16. Jonathan A. Parker, 2011. "On Measuring the Effects of Fiscal Policy in Recessions," Journal of Economic Literature, American Economic Association, vol. 49(3), pages 703-718, September.
    17. David Card & Jochen Kluve & Andrea Weber, 2010. "Active Labour Market Policy Evaluations: A Meta-Analysis," Economic Journal, Royal Economic Society, vol. 120(548), pages 452-477, November.
    18. Abel Brodeur & Nikolai Cook & Anthony Heyes, 2020. "Methods Matter: p-Hacking and Publication Bias in Causal Analysis in Economics," American Economic Review, American Economic Association, vol. 110(11), pages 3634-3660, November.
    19. Gechert, Sebastian & Heimberger, Philipp, 2022. "Do corporate tax cuts boost economic growth?," European Economic Review, Elsevier, vol. 147(C).
    20. Alptekin, Aynur & Levine, Paul, 2012. "Military expenditure and economic growth: A meta-analysis," European Journal of Political Economy, Elsevier, vol. 28(4), pages 636-650.
    21. T. D. Stanley, 2005. "Beyond Publication Bias," Journal of Economic Surveys, Wiley Blackwell, vol. 19(3), pages 309-345, July.
    22. Greg Kaplan & Giovanni L. Violante, 2014. "A Model of the Consumption Response to Fiscal Stimulus Payments," Econometrica, Econometric Society, vol. 82(4), pages 1199-1239, July.
    23. Roberto Veneziani & Luca Zamparelli & Daniele Tavani & Luca Zamparelli, 2017. "Endogenous Technical Change In Alternative Theories Of Growth And Distribution," Journal of Economic Surveys, Wiley Blackwell, vol. 31(5), pages 1272-1303, December.
    24. Shaikh, Anwar, 1974. "Laws of Production and Laws of Algebra: The Humbug Production Function," The Review of Economics and Statistics, MIT Press, vol. 56(1), pages 115-120, February.
    25. Isaiah Andrews & Maximilian Kasy, 2019. "Identification of and Correction for Publication Bias," American Economic Review, American Economic Association, vol. 109(8), pages 2766-2794, August.
    26. SeHyoun Ahn & Greg Kaplan & Benjamin Moll & Thomas Winberry & Christian Wolf, 2018. "When Inequality Matters for Macro and Macro Matters for Inequality," NBER Macroeconomics Annual, University of Chicago Press, vol. 32(1), pages 1-75.
    27. Jesus Felipe & John McCombie, 2020. "The illusions of calculating total factor productivity and testing growth models: from Cobb-Douglas to Solow and Romer," Journal of Post Keynesian Economics, Taylor & Francis Journals, vol. 43(3), pages 470-513, July.
    28. Rannenberg, Ansgar, 2021. "State-dependent fiscal multipliers with preferences over safe assets," Journal of Monetary Economics, Elsevier, vol. 117(C), pages 1023-1040.
    29. Shafik Hebous, 2011. "The Effects Of Discretionary Fiscal Policy On Macroeconomic Aggregates: A Reappraisal," Journal of Economic Surveys, Wiley Blackwell, vol. 25(4), pages 674-707, September.
    30. Marek Rusnak & Tomas Havranek & Roman Horvath, 2013. "How to Solve the Price Puzzle? A Meta‐Analysis," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 45(1), pages 37-70, February.
    31. Tomas Havranek & Marek Rusnak, 2013. "Transmission Lags of Monetary Policy: A Meta-Analysis," International Journal of Central Banking, International Journal of Central Banking, vol. 9(4), pages 39-76, December.
    32. Doucouliagos, Hristos & Paldam, Martin & Stanley, T.D., 2018. "Skating on thin evidence: Implications for public policy," European Journal of Political Economy, Elsevier, vol. 54(C), pages 16-25.
    33. Tomáš Havránek, 2015. "Measuring Intertemporal Substitution: The Importance Of Method Choices And Selective Reporting," Journal of the European Economic Association, European Economic Association, vol. 13(6), pages 1180-1204, December.
    34. Andrew G. Haldane & Arthur E. Turrell, 2019. "Drawing on different disciplines: macroeconomic agent-based models," Journal of Evolutionary Economics, Springer, vol. 29(1), pages 39-66, March.
    35. Dosi, Giovanni & Fagiolo, Giorgio & Napoletano, Mauro & Roventini, Andrea, 2013. "Income distribution, credit and fiscal policies in an agent-based Keynesian model," Journal of Economic Dynamics and Control, Elsevier, vol. 37(8), pages 1598-1625.
    36. Hristos Doucouliagos & T. D. Stanley, 2009. "Publication Selection Bias in Minimum‐Wage Research? A Meta‐Regression Analysis," British Journal of Industrial Relations, London School of Economics, vol. 47(2), pages 406-428, June.
    37. Joseph E Stiglitz, 2018. "Where modern macroeconomics went wrong," Oxford Review of Economic Policy, Oxford University Press and Oxford Review of Economic Policy Limited, vol. 34(1-2), pages 70-106.
    38. Dominika Ehrenbergerova & Josef Bajzik & Tomas Havranek, 2023. "When Does Monetary Policy Sway House Prices? A Meta-Analysis," IMF Economic Review, Palgrave Macmillan;International Monetary Fund, vol. 71(2), pages 538-573, June.
    39. Pedro R.D. Bom & Jenny E. Ligthart, 2014. "What Have We Learned From Three Decades Of Research On The Productivity Of Public Capital?," Journal of Economic Surveys, Wiley Blackwell, vol. 28(5), pages 889-916, December.
    40. Paul De Grauwe & Cláudia Costa Storti & Cláudia Costa Storti, 2004. "The Effects of Monetary Policy: A Meta-Analysis," CESifo Working Paper Series 1224, CESifo.
    41. Eckhard Hein, 2017. "Post-Keynesian macroeconomics since the mid 1990s: main developments," European Journal of Economics and Economic Policies: Intervention, Edward Elgar Publishing, vol. 14(2), pages 131-172, September.
    42. Arestis, Philip, 1996. "Post-Keynesian Economics: Towards Coherence," Cambridge Journal of Economics, Cambridge Political Economy Society, vol. 20(1), pages 111-135, January.
    43. Leonhard Dobusch & Jakob Kapeller, 2012. "A Guide to Paradigmatic Self-Marginalization: Lessons for Post-Keynesian Economists," Review of Political Economy, Taylor & Francis Journals, vol. 24(3), pages 469-487, July.
    44. T.D. Stanley & Hristos Doucouliagos & Margaret Giles & Jost H. Heckemeyer & Robert J. Johnston & Patrice Laroche & Jon P. Nelson & Martin Paldam & Jacques Poot & Geoff Pugh & Randall S. Rosenberger & , 2013. "Meta-Analysis Of Economics Research Reporting Guidelines," Journal of Economic Surveys, Wiley Blackwell, vol. 27(2), pages 390-394, April.
    45. Tommaso Ferraresi & Andrea Roventini & Giorgio Fagiolo, 2015. "Fiscal Policies and Credit Regimes: A TVAR Approach," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 30(7), pages 1047-1072, November.
    46. Nijkamp, Peter & Poot, Jacques, 2004. "Meta-analysis of the effect of fiscal policies on long-run growth," European Journal of Political Economy, Elsevier, vol. 20(1), pages 91-124, March.
    47. Antràs Pol, 2004. "Is the U.S. Aggregate Production Function Cobb-Douglas? New Estimates of the Elasticity of Substitution," The B.E. Journal of Macroeconomics, De Gruyter, vol. 4(1), pages 1-36, April.
    48. Card, David & Krueger, Alan B, 1995. "Time-Series Minimum-Wage Studies: A Meta-analysis," American Economic Review, American Economic Association, vol. 85(2), pages 238-243, May.
    49. Sebastian Dullien, 2012. "Is new always better than old? On the treatment of fiscal policy in Keynesian models," Review of Keynesian Economics, Edward Elgar Publishing, vol. 1(0), pages 5-23.
    50. Asatryan, Zareh & Havlik, Annika & Heinemann, Friedrich & Nover, Justus, 2020. "Biases in fiscal multiplier estimates," European Journal of Political Economy, Elsevier, vol. 63(C).
    51. Sims, Christopher A, 1980. "Macroeconomics and Reality," Econometrica, Econometric Society, vol. 48(1), pages 1-48, January.
    52. Cantore, Cristiano & Levine, Paul & Pearlman, Joseph & Yang, Bo, 2015. "CES technology and business cycle fluctuations," Journal of Economic Dynamics and Control, Elsevier, vol. 61(C), pages 133-151.
    53. Daniel DeRock, 2021. "Hidden in Plain Sight: Unpaid Household Services and the Politics of GDP Measurement," New Political Economy, Taylor & Francis Journals, vol. 26(1), pages 20-35, January.
    54. Valerie A. Ramey, 2011. "Can Government Purchases Stimulate the Economy?," Journal of Economic Literature, American Economic Association, vol. 49(3), pages 673-685, September.
    55. Davide Furceri & Prakash Loungani & Jonathan D Ostry & Pietro Pizzuto, 2021. "The rise in inequality after pandemics: can fiscal support play a mitigating role? [Epidemics, inequality, and poverty in preindustrial and early industrial time]," Industrial and Corporate Change, Oxford University Press and the Associazione ICC, vol. 30(2), pages 445-457.
    56. Sebastian Gechert & Tomas Havranek & Zuzana Irsova & Dominika Kolcunova, 2022. "Measuring Capital-Labor Substitution: The Importance of Method Choices and Publication Bias," Review of Economic Dynamics, Elsevier for the Society for Economic Dynamics, vol. 45, pages 55-82, July.
    57. Eric Krassoi Peach & T. Stanley, 2009. "Efficiency Wages, Productivity and Simultaneity: A Meta-Regression Analysis," Journal of Labor Research, Springer, vol. 30(3), pages 262-268, September.
    58. Chletsos, Michael & Giotis, Georgios P., 2015. "The employment effect of minimum wage using 77 international studies since 1992: A meta-analysis," MPRA Paper 61321, University Library of Munich, Germany.
    59. Olivier Blanchard, 2018. "On the future of macroeconomic models," Oxford Review of Economic Policy, Oxford University Press and Oxford Review of Economic Policy Limited, vol. 34(1-2), pages 43-54.
    60. Simon Wren-Lewis, 2018. "Ending the microfoundations hegemony," Oxford Review of Economic Policy, Oxford University Press and Oxford Review of Economic Policy Limited, vol. 34(1-2), pages 55-69.
    61. Chris Doucouliagos & T.D. Stanley, 2013. "Are All Economic Facts Greatly Exaggerated? Theory Competition And Selectivity," Journal of Economic Surveys, Wiley Blackwell, vol. 27(2), pages 316-339, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Heimberger, Philipp, 2023. "The cyclical behaviour of fiscal policy: A meta-analysis," Economic Modelling, Elsevier, vol. 123(C).
    2. Villamizar-Villegas, Mauricio & Arango-Lozano, Lucía & Castelblanco, Geraldine & Fajardo-Baquero, Nicolás & Ruiz-Sánchez, María Alejandra, 2022. "The effects of Monetary Policy on Capital Flows A Meta-Analysis," Working papers 93, Red Investigadores de Economía.
    3. McCartney, Gerry & Hill O'Connor, Clementine & Laughlin, Sue & Robertson, Tony & Bunse, Lukas & Crighton, Matthew & McLeod, Aileen & Cochrane, Phoebe & Stuart, Francis & Black, Iain & McMaster, Robert, 2025. "Evidence review to support the development of a Wellbeing Economy strategy in Scotland," Working Paper Series 01/2025, Post-Growth Economics Network (PEN).
    4. Gechert, Sebastian & Mey, Bianka & Opatrny, Matej & Havranek, Tomas & Stanley, T. D. & Bom, Pedro R. D. & Doucouliagos, Hristos & Heimberger, Philipp & Irsova, Zuzana & Rachinger, Heiko J., 2023. "Conventional Wisdom, Meta-Analysis, and Research Revision in Economics," EconStor Preprints 280745, ZBW - Leibniz Information Centre for Economics.
    5. Gechert, Sebastian & Heimberger, Philipp, 2022. "Do corporate tax cuts boost economic growth?," European Economic Review, Elsevier, vol. 147(C).
    6. Philipp Heimberger, 2023. "Do higher public debt levels reduce economic growth?," Journal of Economic Surveys, Wiley Blackwell, vol. 37(4), pages 1061-1089, September.
    7. Philipp Heimberger, 2023. "Do higher public debt levels reduce economic growth?," Journal of Economic Surveys, Wiley Blackwell, vol. 37(4), pages 1061-1089, September.
    8. Villamizar-Villegas, Mauricio & Arango-Lozano, Lucía & Castelblanco, Geraldine & Fajardo-Baquero, Nicolás & Ruiz-Sanchez, Maria A., 2024. "The Effects of Monetary Policy on Capital Flows: An Emerging Market Survey," Emerging Markets Review, Elsevier, vol. 62(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Cazachevici, Alina & Havranek, Tomas & Horvath, Roman, 2020. "Remittances and economic growth: A meta-analysis," World Development, Elsevier, vol. 134(C).
    2. Dominika Ehrenbergerova & Josef Bajzik & Tomas Havranek, 2023. "When Does Monetary Policy Sway House Prices? A Meta-Analysis," IMF Economic Review, Palgrave Macmillan;International Monetary Fund, vol. 71(2), pages 538-573, June.
    3. Havranek, Tomas & Horvath, Roman & Zeynalov, Ayaz, 2016. "Natural Resources and Economic Growth: A Meta-Analysis," World Development, Elsevier, vol. 88(C), pages 134-151.
    4. Zuzana Irsova & Hristos Doucouliagos & Tomas Havranek & T. D. Stanley, 2024. "Meta‐analysis of social science research: A practitioner's guide," Journal of Economic Surveys, Wiley Blackwell, vol. 38(5), pages 1547-1566, December.
    5. Thi Mai Lan Nguyen, 2020. "Output Effects of Monetary Policy in Emerging and Developing Countries: Evidence from a Meta-Analysis," Emerging Markets Finance and Trade, Taylor & Francis Journals, vol. 56(1), pages 68-85, January.
    6. Fan Yang & Tomas Havranek & Zuzana Irsova & Jiri Novak, 2024. "Is research on hedge fund performance published selectively? A quantitative survey," Journal of Economic Surveys, Wiley Blackwell, vol. 38(4), pages 1085-1131, September.
    7. Ali Elminejad & Tomas Havranek & Roman Horvath & Zuzana Irsova, 2023. "Intertemporal Substitution in Labor Supply: A Meta-Analysis," Review of Economic Dynamics, Elsevier for the Society for Economic Dynamics, vol. 51, pages 1095-1113, December.
    8. Irsova, Zuzana & Bom, Pedro Ricardo Duarte & Havranek, Tomas & Rachinger, Heiko, 2023. "Spurious Precision in Meta-Analysis," MetaArXiv 3qp2w, Center for Open Science.
    9. Gechert, Sebastian & Heimberger, Philipp, 2022. "Do corporate tax cuts boost economic growth?," European Economic Review, Elsevier, vol. 147(C).
    10. Havranek, Tomas & Irsova, Zuzana & Janda, Karel & Zilberman, David, 2015. "Selective reporting and the social cost of carbon," Energy Economics, Elsevier, vol. 51(C), pages 394-406.
    11. Anton Astakhov & Tomas Havranek & Jiri Novak, 2019. "Firm Size And Stock Returns: A Quantitative Survey," Journal of Economic Surveys, Wiley Blackwell, vol. 33(5), pages 1463-1492, December.
    12. Kroupova, Katerina & Havranek, Tomas & Irsova, Zuzana, 2024. "Student Employment and Education: A Meta-Analysis," Economics of Education Review, Elsevier, vol. 100(C).
    13. Dominika Ehrenbergerova & Josef Bajzik, 2020. "The Effect of Monetary Policy on House Prices - How Strong is the Transmission?," Working Papers 2020/14, Czech National Bank.
    14. Zigraiova, Diana & Havranek, Tomas & Irsova, Zuzana & Novak, Jiri, 2021. "How puzzling is the forward premium puzzle? A meta-analysis," European Economic Review, Elsevier, vol. 134(C).
    15. Anna Sokolova & Todd Sorensen, 2021. "Monopsony in Labor Markets: A Meta-Analysis," ILR Review, Cornell University, ILR School, vol. 74(1), pages 27-55, January.
    16. Cazachevici, Alina & Havranek, Tomas & Horvath, Roman, 2019. "Remittances and Economic Growth: A Quantitative Survey," MPRA Paper 96823, University Library of Munich, Germany.
    17. Tomas Havranek & Anna Sokolova, 2020. "Do Consumers Really Follow a Rule of Thumb? Three Thousand Estimates from 144 Studies Say 'Probably Not'," Review of Economic Dynamics, Elsevier for the Society for Economic Dynamics, vol. 35, pages 97-122, January.
    18. Sebastian Gechert & Tomas Havranek & Zuzana Irsova & Dominika Kolcunova, 2022. "Measuring Capital-Labor Substitution: The Importance of Method Choices and Publication Bias," Review of Economic Dynamics, Elsevier for the Society for Economic Dynamics, vol. 45, pages 55-82, July.
    19. Lan Nguyen, Thi Mai & Papyrakis, Elissaios & van Bergeijk, Peter A.G., 2021. "Publication bias in the price effects of monetary policy: A meta-regression analysis for emerging and developing economies," International Review of Economics & Finance, Elsevier, vol. 71(C), pages 567-583.
    20. Irsova, Zuzana & Bom, Pedro R. D. & Havranek, Tomas & Rachinger, Heiko, 2023. "Spurious Precision in Meta-Analysis of Observational Research," MetaArXiv 3qp2w_v1, Center for Open Science.

    More about this item

    JEL classification:

    • E50 - Macroeconomics and Monetary Economics - - Monetary Policy, Central Banking, and the Supply of Money and Credit - - - General
    • E60 - Macroeconomics and Monetary Economics - - Macroeconomic Policy, Macroeconomic Aspects of Public Finance, and General Outlook - - - General
    • J30 - Labor and Demographic Economics - - Wages, Compensation, and Labor Costs - - - General

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:oup:indcch:v:31:y:2022:i:2:p:576-590.. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Oxford University Press (email available below). General contact details of provider: https://academic.oup.com/icc .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.