IDEAS home Printed from https://ideas.repec.org/a/gam/jpubli/v5y2017i3p21-d112616.html
   My bibliography  Save this article

Measuring Time-Dynamics and Time-Stability of Journal Rankings in Mathematics and Physics by Means of Fractional p -Variations

Author

Listed:
  • Antonia Ferrer-Sapena

    (Instituto de Diseño y Fabricación, Universitat Politècnica de València, 46022 Valencia, Spain
    All the authors contributed equally to this work.)

  • Susana Díaz-Novillo

    (Florida Universitaria, Rey En Jaume I, 2, Catarroja, 46470 Valencia, Spain
    All the authors contributed equally to this work.)

  • Enrique A. Sánchez-Pérez

    (Instituto Universitario de Matemática Pura y Aplicada, Universitat Politècnica de València, 46022 Valencia, Spain
    All the authors contributed equally to this work.)

Abstract

Journal rankings of specific research fields are often used for evaluation purposes, both of authors and institutions. These rankings can be defined by means of several methods, as expert assessment, scholarly-based agreements, or by the ordering induced by a numeric index associated to the prestige of the journals. In order to be efficient and accepted by the research community, it must preserve the ordering over time, at least up to a point. Otherwise, the procedure for defining the ranking must be revised to assure that it reflects the presumably stable characteristic “prestige” that it claims to be quantifying. A mathematical model based on fractional p -variations of the values of the order number of each journal in a time series of journal rankings is explained, and its main properties are shown. As an example, we study the evolution of two given ordered lists of journals through an eleven-year series. These journal ranks are defined by using the 2-year Impact Factor of Thomson-Reuters (nowadays Clarivate Analytics) lists for MATHEMATICS and PHYSICS, APPLIED from 2002 to 2013. As an application of our model, we define an index that precludes the use of journal ranks for evaluation purposes when some minimal requirements on the associated fractional p -variations are not satisfied. The final conclusion is that the list of mathematics does not satisfy the requirements on the p -variations, while the list of applied physics does.

Suggested Citation

  • Antonia Ferrer-Sapena & Susana Díaz-Novillo & Enrique A. Sánchez-Pérez, 2017. "Measuring Time-Dynamics and Time-Stability of Journal Rankings in Mathematics and Physics by Means of Fractional p -Variations," Publications, MDPI, vol. 5(3), pages 1-14, September.
    • Handle: RePEc:gam:jpubli:v:5:y:2017:i:3:p:21-:d:112616
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2304-6775/5/3/21/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2304-6775/5/3/21/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Mansilla, R. & Köppen, E. & Cocho, G. & Miramontes, P., 2007. "On the behavior of journal impact factor rank-order distribution," Journal of Informetrics, Elsevier, vol. 1(2), pages 155-160.
    2. Serenko, Alexander & Dohan, Michael, 2011. "Comparing the expert survey and citation impact journal ranking methods: Example from the field of Artificial Intelligence," Journal of Informetrics, Elsevier, vol. 5(4), pages 629-648.
    3. Benjamin M. Althouse & Jevin D. West & Carl T. Bergstrom & Theodore Bergstrom, 2009. "Differences in impact factor across fields and over time," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 60(1), pages 27-34, January.
    4. Dejian Yu & Wanru Wang & Shuai Zhang & Wenyu Zhang & Rongyu Liu, 2017. "A multiple-link, mutually reinforced journal-ranking model to measure the prestige of journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 111(1), pages 521-542, April.
    5. Per O Seglen, 1992. "How representative is the journal impact factor?," Research Evaluation, Oxford University Press, vol. 2(3), pages 143-149, December.
    6. Emanuel Kulczycki & Ewa A. Rozkosz, 2017. "Does an expert-based evaluation allow us to go beyond the Impact Factor? Experiences from building a ranking of national journals in Poland," Scientometrics, Springer;Akadémiai Kiadó, vol. 111(1), pages 417-442, April.
    7. Xu, Han & Martin, Eric & Mahidadia, Ashesh, 2014. "Contents and time sensitive document ranking of scientific literature," Journal of Informetrics, Elsevier, vol. 8(3), pages 546-561.
    8. A. Ferrer-Sapena & E. A. Sánchez-Pérez & L. M. González & F. Peset & R. Aleixandre-Benavent, 2015. "Mathematical properties of weighted impact factors based on measures of prestige of the citing journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 105(3), pages 2089-2108, December.
    9. Moussa, Salim & Touzani, Mourad, 2010. "Ranking marketing journals using the Google Scholar-based hg-index," Journal of Informetrics, Elsevier, vol. 4(1), pages 107-117.
    10. Éric Archambault & Vincent Larivière, 2009. "History of the journal impact factor: Contingencies and consequences," Scientometrics, Springer;Akadémiai Kiadó, vol. 79(3), pages 635-649, June.
    11. David L. Anderson & John Tressler, 2017. "Researcher rank stability across alternative output measurement schemes in the context of a time limited research evaluation: the New Zealand case," Applied Economics, Taylor & Francis Journals, vol. 49(45), pages 4542-4553, September.
    12. Fiala, Dalibor, 2012. "Time-aware PageRank for bibliographic networks," Journal of Informetrics, Elsevier, vol. 6(3), pages 370-388.
    13. Thed N. van Leeuwen & Henk F. Moed, 2002. "Development and application of journal impact measures in the Dutch science system," Scientometrics, Springer;Akadémiai Kiadó, vol. 53(2), pages 249-266, February.
    14. Maria Bordons & M. T. Fernández & Isabel Gómez, 2002. "Advantages and limitations in the use of impact factor measures for the assessment of research performance," Scientometrics, Springer;Akadémiai Kiadó, vol. 53(2), pages 195-206, February.
    15. Thed Leeuwen, 2012. "Discussing some basic critique on Journal Impact Factors: revision of earlier comments," Scientometrics, Springer;Akadémiai Kiadó, vol. 92(2), pages 443-455, August.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Juan Miguel Campanario, 2018. "Journals that Rise from the Fourth Quartile to the First Quartile in Six Years or Less: Mechanisms of Change and the Role of Journal Self-Citations," Publications, MDPI, vol. 6(4), pages 1-15, November.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. A. Ferrer-Sapena & J. M. Calabuig & L. M. García Raffi & E. A. Sánchez Pérez, 2020. "Where Should I Submit My Work for Publication? An Asymmetrical Classification Model to Optimize Choice," Journal of Classification, Springer;The Classification Society, vol. 37(2), pages 490-508, July.
    2. Battiston, Pietro & Sacco, Pier Luigi & Stanca, Luca, 2022. "Cover effects on citations uncovered: Evidence from Nature," Journal of Informetrics, Elsevier, vol. 16(2).
    3. Jerome K. Vanclay, 2012. "Impact factor: outdated artefact or stepping-stone to journal certification?," Scientometrics, Springer;Akadémiai Kiadó, vol. 92(2), pages 211-238, August.
    4. Tingcan Ma & Gui-Fang Wang & Ke Dong & Mukun Cao, 2012. "The Journal’s Integrated Impact Index: a new indicator for journal evaluation," Scientometrics, Springer;Akadémiai Kiadó, vol. 90(2), pages 649-658, February.
    5. Juan Miguel Campanario, 2011. "Empirical study of journal impact factors obtained using the classical two-year citation window versus a five-year citation window," Scientometrics, Springer;Akadémiai Kiadó, vol. 87(1), pages 189-204, April.
    6. Yu Zhang & Min Wang & Morteza Saberi & Elizabeth Chang, 2020. "Knowledge fusion through academic articles: a survey of definitions, techniques, applications and challenges," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(3), pages 2637-2666, December.
    7. Sangwal, Keshra, 2013. "Citation and impact factor distributions of scientific journals published in individual countries," Journal of Informetrics, Elsevier, vol. 7(2), pages 487-504.
    8. Dejian Yu & Wanru Wang & Shuai Zhang & Wenyu Zhang & Rongyu Liu, 2017. "A multiple-link, mutually reinforced journal-ranking model to measure the prestige of journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 111(1), pages 521-542, April.
    9. Eliseo Reategui & Alause Pires & Michel Carniato & Sergio Roberto Kieling Franco, 2020. "Evaluation of Brazilian research output in education: confronting international and national contexts," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(1), pages 427-444, October.
    10. Jing Li & Qiushuang Long & Xiaoli Lu & Dengsheng Wu, 2023. "Citation beneficiaries of discipline-specific mega-journals: who and how much," Palgrave Communications, Palgrave Macmillan, vol. 10(1), pages 1-10, December.
    11. Walters, William H., 2014. "Do Article Influence scores overestimate the citation impact of social science journals in subfields that are related to higher-impact natural science disciplines?," Journal of Informetrics, Elsevier, vol. 8(2), pages 421-430.
    12. Mingers, John & Yang, Liying, 2017. "Evaluating journal quality: A review of journal citation indicators and ranking in business and management," European Journal of Operational Research, Elsevier, vol. 257(1), pages 323-337.
    13. Torres-Salinas, Daniel & Valderrama-Baca, Pilar & Arroyo-Machado, Wenceslao, 2022. "Is there a need for a new journal metric? Correlations between JCR Impact Factor metrics and the Journal Citation Indicator—JCI," Journal of Informetrics, Elsevier, vol. 16(3).
    14. Eduardo Araujo Oliveira & Roberto Peicots-Filho & Daniella Reis Martelli & Isabel Gomes Quirino & Maria Christina Lopes Oliveira & Mariana Guerra Duarte & Sergio Veloso Pinheiro & Enrico Antonio Colos, 2013. "Is there a correlation between journal impact factor and researchers’ performance? A study comprising the fields of clinical nephrology and neurosciences," Scientometrics, Springer;Akadémiai Kiadó, vol. 97(2), pages 149-160, November.
    15. Waltman, Ludo & van Eck, Nees Jan, 2009. "Some comments on Egghe's derivation of the impact factor distribution," Journal of Informetrics, Elsevier, vol. 3(4), pages 363-366.
    16. Narongrit Sombatsompop & T. Markpin & N. Premkamolnetr, 2004. "A modified method for calculating the Impact Factors of journals in ISI Journal Citation Reports: Polymer Science Category in 1997–2001," Scientometrics, Springer;Akadémiai Kiadó, vol. 60(2), pages 217-235, June.
    17. Khaled Belkadhi & Adel Trabelsi, 2020. "Toward a stochastically robust normalized impact factor against fraud and scams," Scientometrics, Springer;Akadémiai Kiadó, vol. 124(3), pages 1871-1884, September.
    18. Waltman, L. & van Eck, N.J.P., 2009. "Some Comments on Egghe’s Derivation of the Impact Factor Distribution," ERIM Report Series Research in Management ERS-2009-016-LIS, Erasmus Research Institute of Management (ERIM), ERIM is the joint research institute of the Rotterdam School of Management, Erasmus University and the Erasmus School of Economics (ESE) at Erasmus University Rotterdam.
    19. Thelwall, Mike & Fairclough, Ruth, 2015. "Geometric journal impact factors correcting for individual highly cited articles," Journal of Informetrics, Elsevier, vol. 9(2), pages 263-272.
    20. Dašić Predrag, 2015. "State and Analysis of Scientific Journals in the Field of “Economic Sciences” for the Period 1995-2014," Economic Themes, Sciendo, vol. 53(4), pages 547-581, December.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jpubli:v:5:y:2017:i:3:p:21-:d:112616. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.