IDEAS home Printed from https://ideas.repec.org/a/eee/infome/v1y2007i2p103-114.html
   My bibliography  Save this article

The unintended consequences of metrics in technology evaluation

Author

Listed:
  • Kostoff, Ronald N.
  • Geisler, Elie

Abstract

This paper describes science and technology (S&T) metrics, especially impact of metrics on strategic management. The main messages to be conveyed from this paper are: (1) metrics play many roles in supporting management of the S&T enterprise; (2) metrics can influence S&T development incentives; (3) incorrect selection and implementation of metrics can have negative unintended consequences on the research and research documentation generated and (4) before implementing metrics, an organization should identify and evaluate the intended and unintended consequences of the specific metrics’ implementation, and identify the impact of these consequences on the organization's core mission.

Suggested Citation

  • Kostoff, Ronald N. & Geisler, Elie, 2007. "The unintended consequences of metrics in technology evaluation," Journal of Informetrics, Elsevier, vol. 1(2), pages 103-114.
    • Handle: RePEc:eee:infome:v:1:y:2007:i:2:p:103-114
    DOI: 10.1016/j.joi.2007.02.002
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S175115770700034X
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.joi.2007.02.002?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Shobha S. Das & Andrew H. Van de Ven, 2000. "Competing with New Product Technologies: A Process Model of Strategy," Management Science, INFORMS, vol. 46(10), pages 1300-1316, October.
    2. J Sylvan Katz & Viv Cothey, 2006. "Web indicators for complex innovation systems," Research Evaluation, Oxford University Press, vol. 15(2), pages 85-95, August.
    3. Souitaris, Vangelis, 2002. "Technological trajectories as moderators of firm-level determinants of innovation," Research Policy, Elsevier, vol. 31(6), pages 877-898, August.
    4. Peter Weingart, 2005. "Impact of bibliometrics upon the science system: Inadvertent consequences?," Scientometrics, Springer;Akadémiai Kiadó, vol. 62(1), pages 117-131, January.
    5. Richard Klavans & Kevin W. Boyack, 2006. "Identifying a better measure of relatedness for mapping science," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 57(2), pages 251-263, January.
    6. Katz, J. Sylvan, 2006. "Indicators for complex innovation systems," Research Policy, Elsevier, vol. 35(7), pages 893-909, September.
    7. Brian Cozzarin & Jennifer Percival, 2006. "Complementarities between organisational strategies and innovation," Economics of Innovation and New Technology, Taylor & Francis Journals, vol. 15(3), pages 195-217.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. VAN CAMP, Jelle & BRAET, Johan, 2013. "Proposing a taxonomy for performance measurement systems' failures," Working Papers 2013004, University of Antwerp, Faculty of Business and Economics.
    2. Kobos, Peter H. & Malczynski, Leonard A. & Walker, La Tonya N. & Borns, David J. & Klise, Geoffrey T., 2018. "Timing is everything: A technology transition framework for regulatory and market readiness levels," Technological Forecasting and Social Change, Elsevier, vol. 137(C), pages 211-225.
    3. Péter Érdi & Kinga Makovi & Zoltán Somogyvári & Katherine Strandburg & Jan Tobochnik & Péter Volf & László Zalányi, 2013. "Prediction of emerging technologies based on analysis of the US patent citation network," Scientometrics, Springer;Akadémiai Kiadó, vol. 95(1), pages 225-242, April.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Dziallas, Marisa & Blind, Knut, 2019. "Innovation indicators throughout the innovation process: An extensive literature analysis," Technovation, Elsevier, vol. 80, pages 3-29.
    2. Goodall, Amanda H., 2009. "Highly cited leaders and the performance of research universities," Research Policy, Elsevier, vol. 38(7), pages 1079-1092, September.
    3. Naouri, Mohamed & Kuper, Marcel & Hartani, Tarik, 2020. "The power of translation: Innovation dialogues in the context of farmer-led innovation in the Algerian Sahara," Agricultural Systems, Elsevier, vol. 180(C).
    4. Maxim Kotsemir & Alexander Abroskin & Dirk Meissner, 2013. "Innovation concepts and typology – an evolutionary discussion," HSE Working papers WP BRP 05/STI/2013, National Research University Higher School of Economics.
    5. Elena Castro-Martínez & Fernando Jiménez-Sáez & Francisco Javier Ortega-Colomer, 2009. "Science and technology policies: A tale of political use, misuse and abuse of traditional R&D indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 80(3), pages 827-844, September.
    6. J. Sylvan Katz & Guillermo Armando Ronda-Pupo, 2019. "Cooperation, scale-invariance and complex innovation systems: a generalization," Scientometrics, Springer;Akadémiai Kiadó, vol. 121(2), pages 1045-1065, November.
    7. Yanyang Yan & Zhichao Wu, 2020. "Regional innovation distribution and its dynamic evolution: Policy impact and spillover effect—Based on the perspective of innovation motivation," PLOS ONE, Public Library of Science, vol. 15(7), pages 1-29, July.
    8. Li, Xin & Xie, Qianqian & Jiang, Jiaojiao & Zhou, Yuan & Huang, Lucheng, 2019. "Identifying and monitoring the development trends of emerging technologies using patent analysis and Twitter data mining: The case of perovskite solar cell technology," Technological Forecasting and Social Change, Elsevier, vol. 146(C), pages 687-705.
    9. Guillermo Armando Ronda-Pupo, 2017. "The effect of document types and sizes on the scaling relationship between citations and co-authorship patterns in management journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 110(3), pages 1191-1207, March.
    10. Andjelković, Miroslav & Tadić, Bosiljka & Maletić, Slobodan & Rajković, Milan, 2015. "Hierarchical sequencing of online social graphs," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 436(C), pages 582-595.
    11. Michail Fragkias & José Lobo & Deborah Strumsky & Karen C Seto, 2013. "Does Size Matter? Scaling of CO2 Emissions and U.S. Urban Areas," PLOS ONE, Public Library of Science, vol. 8(6), pages 1-8, June.
    12. S. Varun Shrivats & Sujit Bhattacharya, 2014. "Forecasting the trend of international scientific collaboration," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(3), pages 1941-1954, December.
    13. Abbasiharofteh, Milad & Kinne, Jan & Krüger, Miriam, 2021. "The strength of weak and strong ties in bridging geographic and cognitive distances," ZEW Discussion Papers 21-049, ZEW - Leibniz Centre for European Economic Research.
    14. Guillermo Armando Ronda-Pupo & Rodrigo Alda-Varas & Nelson Fenández-Vergara, 2021. "Cumulative advantage of the impact of the Latin American and Caribbean science system on JCR journals outside the region," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(11), pages 9291-9304, November.
    15. Manser, Kristina & Hillebrand, Bas & Driessen, Paul H. & Ziggers, Gerrit Willem & Bloemer, Josée M.M., 2015. "Activity sets in multi-organizational ecologies: a project-level perspective on sustainable energy innovations," Technological Forecasting and Social Change, Elsevier, vol. 90(PB), pages 444-455.
    16. Guillermo Armando Ronda-Pupo, 2017. "The citation-based impact of complex innovation systems scales with the size of the system," Scientometrics, Springer;Akadémiai Kiadó, vol. 112(1), pages 141-151, July.
    17. Abdullah Gök & Alec Waterworth & Philip Shapira, 2015. "Use of web mining in studying innovation," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(1), pages 653-671, January.
    18. Sylvan Katz, 2012. "Science Policy, Complex Innovation Systems and Performance Measures," SPRU Working Paper Series 198, SPRU - Science Policy Research Unit, University of Sussex Business School.
    19. Benedetto Lepori & Isidro F. Aguillo & Marco Seeber, 2014. "Size of web domains and interlinking behavior of higher education institutions in Europe," Scientometrics, Springer;Akadémiai Kiadó, vol. 100(2), pages 497-518, August.
    20. Gao, Xia & Guo, Xiaochuan & Sylvan, Katz J. & Guan, Jiancheng, 2010. "The Chinese innovation system during economic transition: A scale-independent view," Journal of Informetrics, Elsevier, vol. 4(4), pages 618-628.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:infome:v:1:y:2007:i:2:p:103-114. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/joi .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.