IDEAS home Printed from https://ideas.repec.org/a/eee/joepsy/v66y2018icp33-44.html
   My bibliography  Save this article

Lower-rated publications do lower academics’ judgments of publication lists: Evidence from a survey experiment of economists

Author

Listed:
  • Powdthavee, Nattavudh
  • Riyanto, Yohanes E.
  • Knetsch, Jack L.

Abstract

Publications in leading journals are widely known to have a positive impact on economists’ judgments of the value of authors’ contributions and professional reputations. While conjectures that publications in lower-rated journals likely have a negative impact on such judgments are common, there have been virtually no direct tests of their validity. Our intent is to provide results from such a test, one that involved asking economists from 44 universities throughout the world to rate either a publication list with only higher-rated journals or a list with all of these but with additional publications in lower-rated journals. Our primary finding was that, holding other things constant, adding publications in lower-rated journals to what is typically considered a good publication record does have a significant negative impact on economists’ judgments of the value of the author’s contribution. Most implications of this bias suggest negative impacts on social welfare.

Suggested Citation

  • Powdthavee, Nattavudh & Riyanto, Yohanes E. & Knetsch, Jack L., 2018. "Lower-rated publications do lower academics’ judgments of publication lists: Evidence from a survey experiment of economists," Journal of Economic Psychology, Elsevier, vol. 66(C), pages 33-44.
    • Handle: RePEc:eee:joepsy:v:66:y:2018:i:c:p:33-44
    DOI: 10.1016/j.joep.2018.04.003
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S016748701830059X
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.joep.2018.04.003?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. John P. Conley & Mario J. Crucini & Robert A. Driskill & Ali Sina Önder, 2013. "The Effects Of Publication Lags On Life-Cycle Research Productivity In Economics," Economic Inquiry, Western Economic Association International, vol. 51(2), pages 1251-1276, April.
    2. Kyle Jurado & Sydney C. Ludvigson & Serena Ng, 2015. "Measuring Uncertainty," American Economic Review, American Economic Association, vol. 105(3), pages 1177-1216, March.
    3. Kalaitzidakis, P. & Mamuneas, T.P. & Stengos, T., 2003. "Rankings of Academic Journals and Institutions," Working Papers 2003-8, University of Guelph, Department of Economics and Finance.
    4. repec:bla:jfinan:v:53:y:1998:i:1:p:351-363 is not listed on IDEAS
    5. Combes, Pierre-Philippe & Linnemer, Laurent & Visser, Michael, 2008. "Publish or peer-rich? The role of skills and networks in hiring economics professors," Labour Economics, Elsevier, vol. 15(3), pages 423-441, June.
    6. Kahn, Shulamit, 1993. "Gender Differences in Academic Career Paths of Economists," American Economic Review, American Economic Association, vol. 83(2), pages 52-56, May.
    7. Denis Bouyssou & Thierry Marchant, 2011. "Ranking scientists and departments in a consistent manner," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 62(9), pages 1761-1769, September.
    8. Němec Daniel, 2015. "Measuring Inefficiency of the Czech Labour Market," Review of Economic Perspectives, Sciendo, vol. 15(2), pages 197-220, June.
    9. Pantelis Kalaitzidakis & Theofanis P. Mamuneas & Thanasis Stengos, 2003. "Rankings of Academic Journals and Institutions in Economics," Journal of the European Economic Association, MIT Press, vol. 1(6), pages 1346-1366, December.
    10. Wijeyakulasuriya D. A. & Wickremasinghe W. N., 2015. "Measuring Extreme Market Risk: The Sri Lankan Context," Asia-Pacific Journal of Risk and Insurance, De Gruyter, vol. 9(2), pages 185-201, July.
    11. Richard Dusansky & Clayton J. Vernon, 1998. "Rankings of U.S. Economics Departments," Journal of Economic Perspectives, American Economic Association, vol. 12(1), pages 157-170, Winter.
    12. Gaut, Fred, 2015. "Measuring innovation in all sectors of the economy," MERIT Working Papers 2015-038, United Nations University - Maastricht Economic and Social Research Institute on Innovation and Technology (MERIT).
    13. David Card & Stefano DellaVigna, 2013. "Nine Facts about Top Journals in Economics," Journal of Economic Literature, American Economic Association, vol. 51(1), pages 144-161, March.
    14. ., 2015. "Legal measures for finalizing bargains," Chapters, in: Joint Venture Strategies, chapter 9, pages 132-143, Edward Elgar Publishing.
    15. J. C. R. Alcantud & R. Andrés Calle & J. M. Cascón, 2015. "Pairwise Dichotomous Cohesiveness Measures," Group Decision and Negotiation, Springer, vol. 24(5), pages 833-854, September.
    16. OLIVIER CADOT & Lili Yan Ing, 2015. "Non-Tariff Measures: Not All that Bad," Working Papers PB-2015-02, Economic Research Institute for ASEAN and East Asia (ERIA).
    17. Paul W. Grimes & Charles A. Register, 1997. "Career Publications and Academic Job Rank: Evidence from the Class of 1968," The Journal of Economic Education, Taylor & Francis Journals, vol. 28(1), pages 82-92, March.
    18. A. Nifo & G. Vecchione, 2015. "Measuring Institutional Quality in Italy," Rivista economica del Mezzogiorno, Società editrice il Mulino, issue 1-2, pages 157-182.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Vitus Püttmann & Jens Ruhose & Stephan L. Thomsen, 2023. "Academics’ Attitudes Toward Engaging in Public Discussions: Experimental Evidence on the Impact of Engagement Conditions," Research in Higher Education, Springer;Association for Institutional Research, vol. 64(5), pages 765-788, August.
    2. Aistleitner, Matthias & Kapeller, Jakob & Kronberger, Dominik, 2023. "The authors of economics journals revisited: evidence from a large-scale replication of Hodgson and Rothman (1999)," Journal of Institutional Economics, Cambridge University Press, vol. 19(1), pages 86-101, February.
    3. Payson Steven, 2019. "Cite This Economics Paper! It Is Time for the House of Cards to Fall Down," Open Economics, De Gruyter, vol. 2(1), pages 1-18, January.
    4. Lars Hornuf & Gül Yüksel, 2022. "The Performance of Socially Responsible Investments: A Meta-Analysis," CESifo Working Paper Series 9724, CESifo.
    5. Joanna Tyrowicz & Lucas Augusto van der Velde & Magdalena Smyk, 2024. "Gender-neutral hiring of junior scholars," GRAPE Working Papers 94, GRAPE Group for Research in Applied Economics.
    6. Ramani, Ravi S. & Aguinis, Herman & Coyle-Shapiro, Jacqueline A.M., 2022. "Defining, measuring, and rewarding scholarly impact: mind the level of analysis," LSE Research Online Documents on Economics 117286, London School of Economics and Political Science, LSE Library.
    7. Syed Hasan & Robert Breunig, 2021. "Article length and citation outcomes," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(9), pages 7583-7608, September.
    8. Aistleitner, Matthias & Kapeller, Jakob & Kronberger, Dominik, 2022. "The authors of economics journals revisited: Evidence from a large-scale replication of Hodgson & Rothman (1999)," ifso working paper series 20, University of Duisburg-Essen, Institute for Socioeconomics (ifso).
    9. David I. Stern & Richard S. J. Tol, 2021. "Depth and breadth relevance in citation metrics," Economic Inquiry, Western Economic Association International, vol. 59(3), pages 961-977, July.
    10. M. Ryan Haley & M. Kevin McGee, 2023. "A flexible functional method for jointly valuing journal visibility and author citation count," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(6), pages 3337-3346, June.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Powdthavee, Nattavudh & Riyanto, Yohanes E. & Knetsch, Jack L., 2017. "Impact of Lower Rated Journals on Economists' Judgments of Publication Lists: Evidence from a Survey Experiment," IZA Discussion Papers 10752, Institute of Labor Economics (IZA).
    2. Önder Ali Sina & Yilmazkuday Hakan, 2020. "Thirty-Five Years of Peer-Reviewed Publishing by North American Economics PhDs: Quantity, Quality, and Beyond," Open Economics, De Gruyter, vol. 3(1), pages 70-85, January.
    3. Ali Sina Önder & Sascha Schweitzer & Hakan Yilmazkuday, 2021. "Field Distance and Quality in Economists’ Collaborations," Working Papers in Economics & Finance 2021-04, University of Portsmouth, Portsmouth Business School, Economics and Finance Subject Group.
    4. Önder, Ali Sina & Schweitzer, Sascha & Yilmazkuday, Hakan, 2021. "Specialization, field distance, and quality in economists’ collaborations," Journal of Informetrics, Elsevier, vol. 15(4).
    5. James J. Heckman & Sidharth Moktan, 2020. "Publishing and Promotion in Economics: The Tyranny of the Top Five," Journal of Economic Literature, American Economic Association, vol. 58(2), pages 419-470, June.
    6. José Alberto Molina & Alfredo Ferrer & David Iñiguez & Alejandro Rivero & Gonzalo Ruiz & Alfonso Tarancón, 2020. "Network analysis to measure academic performance in economics," Empirical Economics, Springer, vol. 58(3), pages 995-1018, March.
    7. Irion, Kristina & Helberger, Natali, 2017. "Smart TV and the online media sector: User privacy in view of changing market realities," Telecommunications Policy, Elsevier, vol. 41(3), pages 170-184.
    8. Ali Sina Önder & Sascha Schweitzer, 2017. "Catching up or falling behind? Promising changes and persistent patterns across cohorts of economics PhDs in German-speaking countries from 1991 to 2008," Scientometrics, Springer;Akadémiai Kiadó, vol. 110(3), pages 1297-1331, March.
    9. Melody Lo & M. C. Sunny Wong & Franklin G. Mixon, 2008. "Ranking Economics Journals, Economics Departments, and Economists Using Teaching‐Focused Research Productivity," Southern Economic Journal, John Wiley & Sons, vol. 74(3), pages 894-906, January.
    10. Dolado, Juan J. & Felgueroso, Florentino & Almunia, Miguel, 2005. "Do Men and Women-Economists Choose the Same Research Fields? Evidence from Top-50 Departments," IZA Discussion Papers 1859, Institute of Labor Economics (IZA).
    11. Yong Bao & Melody Lo & Franklin G. Mixon, 2010. "General-interest versus specialty journals: Using intellectual influence of econometrics research to rank economics journals and articles," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 25(2), pages 345-353.
    12. repec:bla:germec:v:9:y:2008:i::p:431-456 is not listed on IDEAS
    13. Baltagi, Badi H., 2007. "Worldwide Econometrics Rankings: 1989–2005," Econometric Theory, Cambridge University Press, vol. 23(5), pages 952-1012, October.
    14. Ma, Chunbo & Stern, David I., 2006. "Environmental and ecological economics: A citation analysis," Ecological Economics, Elsevier, vol. 58(3), pages 491-506, June.
    15. Joseph Macri & Dipendra Sinha, 2006. "Rankings Methodology for International Comparisons of Institutions and Individuals: an Application to Economics in Australia and New Zealand," Journal of Economic Surveys, Wiley Blackwell, vol. 20(1), pages 111-156, February.
    16. Tombazos, Christis G., 2005. "A revisionist perspective of European research in economics," European Economic Review, Elsevier, vol. 49(2), pages 251-277, February.
    17. Rauber Michael & Ursprung Heinrich W., 2008. "Life Cycle and Cohort Productivity in Economic Research: The Case of Germany," German Economic Review, De Gruyter, vol. 9(4), pages 431-456, December.
    18. Püttmann, Vitus & Thomsen, Stephan L. & Trunzer, Johannes, 2020. "Zur Relevanz von Ausstattungsunterschieden für Forschungsleistungsvergleiche: Ein Diskussionsbeitrag für die Wirtschaftswissenschaften in Deutschland," Hannover Economic Papers (HEP) dp-679, Leibniz Universität Hannover, Wirtschaftswissenschaftliche Fakultät, revised Mar 2021.
    19. William C. Horrace & Christopher F. Parmeter, 2017. "Accounting for Multiplicity in Inference on Economics Journal Rankings," Southern Economic Journal, John Wiley & Sons, vol. 84(1), pages 337-347, July.
    20. J. Atsu Amegashie, 2020. "Citations And Incentives In Academic Contests," Economic Inquiry, Western Economic Association International, vol. 58(3), pages 1233-1244, July.

    More about this item

    Keywords

    Lower ranked journals; Publication; Judgment bias; Less-is-better effect;
    All these keywords.

    JEL classification:

    • A1 - General Economics and Teaching - - General Economics
    • M5 - Business Administration and Business Economics; Marketing; Accounting; Personnel Economics - - Personnel Economics

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:joepsy:v:66:y:2018:i:c:p:33-44. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/joep .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.