IDEAS home Printed from https://ideas.repec.org/a/eee/infome/v1y2007i2p103-114.html
   My bibliography  Save this article

The unintended consequences of metrics in technology evaluation

Author

Listed:
  • Kostoff, Ronald N.
  • Geisler, Elie

Abstract

This paper describes science and technology (S&T) metrics, especially impact of metrics on strategic management. The main messages to be conveyed from this paper are: (1) metrics play many roles in supporting management of the S&T enterprise; (2) metrics can influence S&T development incentives; (3) incorrect selection and implementation of metrics can have negative unintended consequences on the research and research documentation generated and (4) before implementing metrics, an organization should identify and evaluate the intended and unintended consequences of the specific metrics’ implementation, and identify the impact of these consequences on the organization's core mission.

Suggested Citation

  • Kostoff, Ronald N. & Geisler, Elie, 2007. "The unintended consequences of metrics in technology evaluation," Journal of Informetrics, Elsevier, vol. 1(2), pages 103-114.
    • Handle: RePEc:eee:infome:v:1:y:2007:i:2:p:103-114
    DOI: 10.1016/j.joi.2007.02.002
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S175115770700034X
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.joi.2007.02.002?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Shobha S. Das & Andrew H. Van de Ven, 2000. "Competing with New Product Technologies: A Process Model of Strategy," Management Science, INFORMS, vol. 46(10), pages 1300-1316, October.
    2. J Sylvan Katz & Viv Cothey, 2006. "Web indicators for complex innovation systems," Research Evaluation, Oxford University Press, vol. 15(2), pages 85-95, August.
    3. Souitaris, Vangelis, 2002. "Technological trajectories as moderators of firm-level determinants of innovation," Research Policy, Elsevier, vol. 31(6), pages 877-898, August.
    4. Peter Weingart, 2005. "Impact of bibliometrics upon the science system: Inadvertent consequences?," Scientometrics, Springer;Akadémiai Kiadó, vol. 62(1), pages 117-131, January.
    5. Richard Klavans & Kevin W. Boyack, 2006. "Identifying a better measure of relatedness for mapping science," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 57(2), pages 251-263, January.
    6. Katz, J. Sylvan, 2006. "Indicators for complex innovation systems," Research Policy, Elsevier, vol. 35(7), pages 893-909, September.
    7. Brian Cozzarin & Jennifer Percival, 2006. "Complementarities between organisational strategies and innovation," Economics of Innovation and New Technology, Taylor & Francis Journals, vol. 15(3), pages 195-217.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. VAN CAMP, Jelle & BRAET, Johan, 2013. "Proposing a taxonomy for performance measurement systems' failures," Working Papers 2013004, University of Antwerp, Faculty of Business and Economics.
    2. Kobos, Peter H. & Malczynski, Leonard A. & Walker, La Tonya N. & Borns, David J. & Klise, Geoffrey T., 2018. "Timing is everything: A technology transition framework for regulatory and market readiness levels," Technological Forecasting and Social Change, Elsevier, vol. 137(C), pages 211-225.
    3. Péter Érdi & Kinga Makovi & Zoltán Somogyvári & Katherine Strandburg & Jan Tobochnik & Péter Volf & László Zalányi, 2013. "Prediction of emerging technologies based on analysis of the US patent citation network," Scientometrics, Springer;Akadémiai Kiadó, vol. 95(1), pages 225-242, April.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Dziallas, Marisa & Blind, Knut, 2019. "Innovation indicators throughout the innovation process: An extensive literature analysis," Technovation, Elsevier, vol. 80, pages 3-29.
    2. Li, Xin & Xie, Qianqian & Jiang, Jiaojiao & Zhou, Yuan & Huang, Lucheng, 2019. "Identifying and monitoring the development trends of emerging technologies using patent analysis and Twitter data mining: The case of perovskite solar cell technology," Technological Forecasting and Social Change, Elsevier, vol. 146(C), pages 687-705.
    3. Guillermo Armando Ronda-Pupo, 2017. "The effect of document types and sizes on the scaling relationship between citations and co-authorship patterns in management journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 110(3), pages 1191-1207, March.
    4. Andjelković, Miroslav & Tadić, Bosiljka & Maletić, Slobodan & Rajković, Milan, 2015. "Hierarchical sequencing of online social graphs," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 436(C), pages 582-595.
    5. S. Varun Shrivats & Sujit Bhattacharya, 2014. "Forecasting the trend of international scientific collaboration," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(3), pages 1941-1954, December.
    6. Abbasiharofteh, Milad & Kinne, Jan & Krüger, Miriam, 2021. "The strength of weak and strong ties in bridging geographic and cognitive distances," ZEW Discussion Papers 21-049, ZEW - Leibniz Centre for European Economic Research.
    7. Guillermo Armando Ronda-Pupo & Rodrigo Alda-Varas & Nelson Fenández-Vergara, 2021. "Cumulative advantage of the impact of the Latin American and Caribbean science system on JCR journals outside the region," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(11), pages 9291-9304, November.
    8. Guillermo Armando Ronda-Pupo, 2017. "The citation-based impact of complex innovation systems scales with the size of the system," Scientometrics, Springer;Akadémiai Kiadó, vol. 112(1), pages 141-151, July.
    9. Abdullah Gök & Alec Waterworth & Philip Shapira, 2015. "Use of web mining in studying innovation," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(1), pages 653-671, January.
    10. Sylvan Katz, 2012. "Science Policy, Complex Innovation Systems and Performance Measures," SPRU Working Paper Series 198, SPRU - Science Policy Research Unit, University of Sussex Business School.
    11. Benedetto Lepori & Isidro F. Aguillo & Marco Seeber, 2014. "Size of web domains and interlinking behavior of higher education institutions in Europe," Scientometrics, Springer;Akadémiai Kiadó, vol. 100(2), pages 497-518, August.
    12. Guillermo Armando Ronda-Pupo & J. Sylvan Katz, 2018. "The power law relationship between citation impact and multi-authorship patterns in articles in Information Science & Library Science journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(3), pages 919-932, March.
    13. Nasierowski Wojciech, 2019. "Reflections on Discussions About Technical Efficiency of Innovativeness of Countries," Foundations of Management, Sciendo, vol. 11(1), pages 165-176, January.
    14. José Manuel López‐Fernández & Mariluz Maté‐Sánchez‐Val & Francisco Manuel Somohano‐Rodriguez, 2021. "The effect of micro‐territorial networks on industrial small and medium enterprises' innovation: A case study in the Spanish region of Cantabria," Papers in Regional Science, Wiley Blackwell, vol. 100(1), pages 51-77, February.
    15. Rotolo, Daniele & Hicks, Diana & Martin, Ben R., 2015. "What is an emerging technology?," Research Policy, Elsevier, vol. 44(10), pages 1827-1843.
    16. J. Lobo & D. Strumsky & J. Rothwell, 2013. "Scaling of patenting with urban population size: evidence from global metropolitan areas," Scientometrics, Springer;Akadémiai Kiadó, vol. 96(3), pages 819-828, September.
    17. Calabrese, Armando & Capece, Guendalina & Costa, Roberta & Di Pillo, Francesca & Giuffrida, Stefania, 2018. "A ‘power law’ based method to reduce size-related bias in indicators of knowledge performance: An application to university research assessment," Journal of Informetrics, Elsevier, vol. 12(4), pages 1263-1281.
    18. Wooseok Jang & Yongtae Park & Hyeonju Seol, 2021. "Identifying emerging technologies using expert opinions on the future: A topic modeling and fuzzy clustering approach," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(8), pages 6505-6532, August.
    19. Jose Luis Ortega & Isidro Aguillo & Viv Cothey & Andrea Scharnhorst, 2008. "Maps of the academic web in the European Higher Education Area — an exploration of visual web indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 74(2), pages 295-308, February.
    20. Marek Dziura & Tomasz Rojek, 2021. "Management of the Company’s Innovation Development: The Case for Polish Enterprises," JRFM, MDPI, vol. 14(4), pages 1-16, April.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:infome:v:1:y:2007:i:2:p:103-114. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/joi .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.