IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v100y2014i2d10.1007_s11192-014-1249-z.html
   My bibliography  Save this article

How expensive is Big Science? Consequences of using simple publication counts in performance assessment of large scientific facilities

Author

Listed:
  • Olof Hallonsten

    (University of Gothenburg)

Abstract

Although the nuclear era and the Cold War superpower competition have long since passed, governments are still investing in Big Science, although these large facilities are nowadays mostly geared towards areas of use closer to utility. Investments in Big Science are also motivated not only by promises of scientific breakthroughs but also by expectations (and demands) of measurable impact, and with an emerging global market of competing user-oriented Big Science facilities, quantitative measures of productivity and quality have become mainstream. Among these are rather simple and one-sided publication counts. This article uses publication counts and figures of expenditure for three cases that are disparate but all represent the state-of-the-art of Big Science of their times, discussing at depth the problems of using simple publication counts as a measure of performance in science. Showing, quite trivially, that Big Science is very expensive, the article also shows the absurd consequences of consistently using simple publication counts to display productivity and quality of Big Science, and concludes that such measures should be deemed irrelevant for analyses on the level of organizations in science and replaced by qualitative assessment of the content of the science produced.

Suggested Citation

  • Olof Hallonsten, 2014. "How expensive is Big Science? Consequences of using simple publication counts in performance assessment of large scientific facilities," Scientometrics, Springer;Akadémiai Kiadó, vol. 100(2), pages 483-496, August.
    • Handle: RePEc:spr:scient:v:100:y:2014:i:2:d:10.1007_s11192-014-1249-z
    DOI: 10.1007/s11192-014-1249-z
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-014-1249-z
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-014-1249-z?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Wilhelm Agrell, 2012. "Framing prospects and risk in the public promotion of ESS Scandinavia," Science and Public Policy, Oxford University Press, vol. 39(4), pages 429-438, July.
    2. Aant Elzinga, 2012. "Features of the current science policy regime: Viewed in historical perspective," Science and Public Policy, Oxford University Press, vol. 39(4), pages 416-428, August.
    3. Olof Hallonsten & Thomas Heinze, 2012. "Institutional persistence through gradual organizational adaptation: Analysis of national laboratories in the USA and Germany," Science and Public Policy, Oxford University Press, vol. 39(4), pages 450-463, July.
    4. Merle Jacob & Olof Hallonsten, 2012. "The persistence of big science and megascience in research and innovation policy," Science and Public Policy, Oxford University Press, vol. 39(4), pages 411-415, July.
    5. Olof Hallonsten, 2013. "Introducing ‘facilitymetrics’: a first review and analysis of commonly used measures of scientific leadership among synchrotron radiation facilities worldwide," Scientometrics, Springer;Akadémiai Kiadó, vol. 96(2), pages 497-513, August.
    6. Martin, Ben R. & Irvine, John, 1984. "CERN: Past performance and future prospects : I. CERN's position in world high-energy physics," Research Policy, Elsevier, vol. 13(4), pages 183-210, August.
    7. Catherine Westfall, 2012. "Institutional persistence and the material transformation of the US national labs: The curious story of the advent of the Advanced Photon Source," Science and Public Policy, Oxford University Press, vol. 39(4), pages 439-449, July.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Qiao, Lili & Mu, Rongping & Chen, Kaihua, 2016. "Scientific effects of large research infrastructures in China," Technological Forecasting and Social Change, Elsevier, vol. 112(C), pages 102-112.
    2. Richard Heidler & Olof Hallonsten, 2015. "Qualifying the performance evaluation of Big Science beyond productivity, impact and costs," Scientometrics, Springer;Akadémiai Kiadó, vol. 104(1), pages 295-312, July.
    3. Wenchao Xu & Yanmei Xu & Junfeng Li, 2017. "A Study of RI Clusters Based on Symbiosis Theory," Sustainability, MDPI, vol. 9(3), pages 1-13, March.
    4. Edmilson J. T. Manganote & Peter A. Schulz & Carlos Henrique Brito Cruz, 2016. "Effect of high energy physics large collaborations on higher education institutions citations and rankings," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(2), pages 813-826, November.
    5. György Csomós & Zsófia Viktória Vida & Balázs Lengyel, 2020. "Exploring the changing geographical pattern of international scientific collaborations through the prism of cities," PLOS ONE, Public Library of Science, vol. 15(11), pages 1-20, November.
    6. D’Ippolito, Beatrice & Rüling, Charles-Clemens, 2019. "Research collaboration in Large Scale Research Infrastructures: Collaboration types and policy implications," Research Policy, Elsevier, vol. 48(5), pages 1282-1296.
    7. Andrea Bastianin & Paolo Castelnovo & Massimo Florio & Anna Giunta, 2022. "Big science and innovation: gestation lag from procurement to patents for CERN suppliers," The Journal of Technology Transfer, Springer, vol. 47(2), pages 531-555, April.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Olof Hallonsten, 2013. "Introducing ‘facilitymetrics’: a first review and analysis of commonly used measures of scientific leadership among synchrotron radiation facilities worldwide," Scientometrics, Springer;Akadémiai Kiadó, vol. 96(2), pages 497-513, August.
    2. Richard Heidler & Olof Hallonsten, 2015. "Qualifying the performance evaluation of Big Science beyond productivity, impact and costs," Scientometrics, Springer;Akadémiai Kiadó, vol. 104(1), pages 295-312, July.
    3. Young-Sun Jang & Young Joo Ko, 2019. "How latecomers catch up to leaders in high-energy physics as Big Science: transition from national system to international collaboration," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(1), pages 437-480, April.
    4. D’Ippolito, Beatrice & Rüling, Charles-Clemens, 2019. "Research collaboration in Large Scale Research Infrastructures: Collaboration types and policy implications," Research Policy, Elsevier, vol. 48(5), pages 1282-1296.
    5. Kamilla Kohn Rådberg & Hans Löfsten, 2024. "The entrepreneurial university and development of large-scale research infrastructure: exploring the emerging university function of collaboration and leadership," The Journal of Technology Transfer, Springer, vol. 49(1), pages 334-366, February.
    6. Wenchao Xu & Yanmei Xu & Junfeng Li, 2017. "A Study of RI Clusters Based on Symbiosis Theory," Sustainability, MDPI, vol. 9(3), pages 1-13, March.
    7. Patrick S. Roberts & Jon Schmid, 2022. "Government‐led innovation acceleration: Case studies of US federal government innovation and technology acceleration organizations," Review of Policy Research, Policy Studies Organization, vol. 39(3), pages 353-378, May.
    8. Christoph March & Ina Schieferdecker, 2021. "Technological Sovereignty as Ability, Not Autarky," CESifo Working Paper Series 9139, CESifo.
    9. Eugenio Petrovich, 2022. "Bibliometrics in Press. Representations and uses of bibliometric indicators in the Italian daily newspapers," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(5), pages 2195-2233, May.
    10. Anna-Lena Rüland & Nicolas Rüffin & Katharina Cramer & Prosper Ngabonziza & Manoj Saxena & Stefan Skupien, 2023. "Science diplomacy from the Global South: the case of intergovernmental science organizations," Science and Public Policy, Oxford University Press, vol. 50(4), pages 782-793.
    11. Castelnovo, Paolo & Florio, Massimo & Forte, Stefano & Rossi, Lucio & Sirtori, Emanuela, 2018. "The economic impact of technological procurement for large-scale research infrastructures: Evidence from the Large Hadron Collider at CERN," Research Policy, Elsevier, vol. 47(9), pages 1853-1867.
    12. Stanislav Zaichenko, 2018. "The human resource dimension of science-based technology transfer: lessons from Russian RTOs and innovative enterprises," The Journal of Technology Transfer, Springer, vol. 43(2), pages 368-388, April.
    13. Fabio S. V. Silva & Peter A. Schulz & Everard C. M. Noyons, 2019. "Co-authorship networks and research impact in large research facilities: benchmarking internal reports and bibliometric databases," Scientometrics, Springer;Akadémiai Kiadó, vol. 118(1), pages 93-108, January.
    14. A. J. Nederhof & E. Wijk, 1999. "Profiling institutes: Identifying high research performance and social relevance in the social and behavioral sciences," Scientometrics, Springer;Akadémiai Kiadó, vol. 44(3), pages 487-506, March.
    15. Soete, Luc, 2019. "Science, technology and innovation studies at a crossroad: SPRU as case study," Research Policy, Elsevier, vol. 48(4), pages 849-857.
    16. Ben R. Martin, 2015. "R&D Policy Instruments: A Critical Review of What We Do & Don't Know," Working Papers wp476, Centre for Business Research, University of Cambridge.
    17. Del Bo, Chiara F., 2016. "The rate of return to investment in R&D: The case of research infrastructures," Technological Forecasting and Social Change, Elsevier, vol. 112(C), pages 26-37.
    18. Laurent Bach & Sandrine Wolff, 2017. "The BETA-EvaRIO impact evaluation method: towards a bridging approach?," Post-Print hal-02167827, HAL.
    19. Carrazza, Stefano & Ferrara, Alfio & Salini, Silvia, 2016. "Research infrastructures in the LHC era: A scientometric approach," Technological Forecasting and Social Change, Elsevier, vol. 112(C), pages 121-133.
    20. Massimo FLORIO & Francesco GIFFONI, 2019. "L’impatto sociale della produzione di scienza su larga scala: come governarlo?," Departmental Working Papers 2019-05, Department of Economics, Management and Quantitative Methods at Università degli Studi di Milano.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:100:y:2014:i:2:d:10.1007_s11192-014-1249-z. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.