IDEAS home Printed from https://ideas.repec.org/a/eee/jomega/v28y2000i2p241-245.html
   My bibliography  Save this article

Is content analysis either practical or desirable for research evaluation?

Author

Listed:
  • Ormerod, R. J.

Abstract

This note responds to comments by Doyle (Omega, 1999;27:403-405) and Jones (Omega, 1999;27:397-401) on my contribution (Omega 1997;25:599-603) to the ongoing debate on judging the quality of research at business schools (a debate initiated by the same two authors and their co-authors). Both contributors have critically examined the use of Reisman and Kirschnick's work on the content analysis of MS/OR articles, each from a different perspective. Doyle sets out the analytical steps that would be required and argues that there are few, if any, gains to be made from the additional work involved in the content analysis. Jones argues that, even though content analysis has yet to be tried, peer review of journals and citation indices studies are to be preferred because they appear relatively more valid, reliable and practicable. In response I restate the case for analysing content, consider the specific arguments of Doyle and Jones, air other concerns, and conclude that content analysis should remain on the agenda despite the obvious difficulties. An analysis of the 1994 volume of the Journal of the Operational Research Society is described to illustrate how the use of content analysis can provide insight.

Suggested Citation

  • Ormerod, R. J., 2000. "Is content analysis either practical or desirable for research evaluation?," Omega, Elsevier, vol. 28(2), pages 241-245, April.
    • Handle: RePEc:eee:jomega:v:28:y:2000:i:2:p:241-245
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0305-0483(99)00049-3
    Download Restriction: Full text for ScienceDirect subscribers only
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Jones, M. J. & Brinn, T. & Pendlebury, M., 1996. "Judging the quality of research in business schools: A comment from accounting," Omega, Elsevier, vol. 24(5), pages 597-602, October.
    2. Doyle, J. R. & Arthurs, A. J. & Mcaulay, L. & Osborne, P. G., 1996. "Citation as effortful voting: A reply to ones, Brinn and Pendlebury," Omega, Elsevier, vol. 24(5), pages 603-606, October.
    3. Ormerod, R. J., 1997. "An observation on publication habits based on the analysis of MS/OR journals," Omega, Elsevier, vol. 25(5), pages 599-603, October.
    4. Michael H. Rothkopf, 1997. "Editorial: Which Universities Contribute to the Practice Literature? The Second Interfaces Ranking," Interfaces, INFORMS, vol. 27(4), pages 19-22, August.
    5. Jones, M. J. & Brinn, T. & Pendlebury, M., 1996. "Journal evaluation methodologies: A balanced response," Omega, Elsevier, vol. 24(5), pages 607-612, October.
    6. Jones, Michael John, 1999. "Critically evaluating an applications vs theory framework for research quality," Omega, Elsevier, vol. 27(3), pages 397-401, June.
    7. Doyle, John R., 1999. "Evaluating OR/MS research," Omega, Elsevier, vol. 27(3), pages 403-405, June.
    8. Uma G. Gupta, 1997. "Using Citation Analysis to Explore the Intellectual Base, Knowledge Dissemination, and Research Impact of Interfaces (1970–1992)," Interfaces, INFORMS, vol. 27(2), pages 85-101, April.
    9. Michael H. Rothkopf, 1996. "Editorial: Which Universities Contribute to the Practice Literature? The First Interfaces Ranking," Interfaces, INFORMS, vol. 26(2), pages 16-21, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Miśkiewicz, Janusz, 2013. "Effects of publications in proceedings on the measure of the core size of coauthors," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 392(20), pages 5119-5131.
    2. Iszan Hana Kaharudin & Mohammad Syuhaimi Ab-Rahman & Roslan Abd-Shukor & Azamin Zaharim & Mohd Jailani Mohd Nor & Ahmad Kamal Ariffin Mohd Ihsan & Shahrom Md Zain & Afiq Hipni & Kamisah Osman & Ruszym, 2022. "How Does Supervision Technique Affect Research? Towards Sustainable Performance: Publications and Students from Pure and Social Sciences," Sustainability, MDPI, vol. 14(9), pages 1-18, May.
    3. Vastag, Gyula & Montabon, Frank, 2002. "Journal characteristics, rankings and social acculturation in operations management," Omega, Elsevier, vol. 30(2), pages 109-126, April.
    4. R J Ormerod, 2010. "Research contribution: Citation and content analysis," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 61(4), pages 705-707, April.
    5. Meltem Denizel & Behlul Usdiken & Deniz Tuncalp, 2003. "Drift or Shift? Continuity, Change, and International Variation in Knowledge Production in OR/MS," Operations Research, INFORMS, vol. 51(5), pages 711-720, October.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Vastag, Gyula & Montabon, Frank, 2002. "Journal characteristics, rankings and social acculturation in operations management," Omega, Elsevier, vol. 30(2), pages 109-126, April.
    2. Holsapple, Clyde W. & Lee-Post, Anita, 2010. "Behavior-based analysis of knowledge dissemination channels in operations management," Omega, Elsevier, vol. 38(3-4), pages 167-178, June.
    3. Jones, Michael John, 1999. "Critically evaluating an applications vs theory framework for research quality," Omega, Elsevier, vol. 27(3), pages 397-401, June.
    4. Donohue, Joan M. & Fox, Jeremy B., 2000. "A multi-method evaluation of journals in the decision and management sciences by US academics," Omega, Elsevier, vol. 28(1), pages 17-36, February.
    5. Iszan Hana Kaharudin & Mohammad Syuhaimi Ab-Rahman & Roslan Abd-Shukor & Azamin Zaharim & Mohd Jailani Mohd Nor & Ahmad Kamal Ariffin Mohd Ihsan & Shahrom Md Zain & Afiq Hipni & Kamisah Osman & Ruszym, 2022. "How Does Supervision Technique Affect Research? Towards Sustainable Performance: Publications and Students from Pure and Social Sciences," Sustainability, MDPI, vol. 14(9), pages 1-18, May.
    6. Ormerod, R. J., 1997. "An observation on publication habits based on the analysis of MS/OR journals," Omega, Elsevier, vol. 25(5), pages 599-603, October.
    7. Rosenstreich, Daniela & Wooliscroft, Ben, 2009. "Measuring the impact of accounting journals using Google Scholar and the g-index," The British Accounting Review, Elsevier, vol. 41(4), pages 227-239.
    8. Brinn, Tony & Jones, Michael John & Pendlebury, Maurice, 2000. "Measuring research quality: peer review 1, citation indices 0," Omega, Elsevier, vol. 28(2), pages 237-239, April.
    9. Michael F. Gorman, 2016. "Editorial: The 11th Rothkopf Rankings: 28 Years of Tracking Applied Research in Academia," Interfaces, INFORMS, vol. 46(3), pages 264-276, June.
    10. Michael F. Gorman, 2019. "Editorial: The 12th Rothkopf Rankings," Interfaces, INFORMS, vol. 49(4), pages 295-303, July.
    11. J Mingers, 2008. "Exploring the dynamics of journal citations: Modelling with s-curves," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 59(8), pages 1013-1025, August.
    12. Ronald D. Fricker, 2012. "Editorial: The First Rothkopf Rankings of Nonacademic Organizations," Interfaces, INFORMS, vol. 42(6), pages 585-590, December.
    13. Jones, M. J. & Brinn, T. & Pendlebury, M., 1996. "Journal evaluation methodologies: A balanced response," Omega, Elsevier, vol. 24(5), pages 607-612, October.
    14. Doyle, John R., 1999. "Evaluating OR/MS research," Omega, Elsevier, vol. 27(3), pages 403-405, June.
    15. Mitchell, George, 1996. "Judging research quality and journals: A call for debate," Omega, Elsevier, vol. 24(5), pages 613-613, October.
    16. Michael H. Rothkopf, 2007. "Editorial: The Seventh Interfaces Ranking of Universities' Contributions to the Practice Literature," Interfaces, INFORMS, vol. 37(6), pages 566-569, December.
    17. Michael H. Rothkopf, 2005. "Editorial: The Sixth Interfaces Ranking of Universities’ Contributions to the Practice Literature," Interfaces, INFORMS, vol. 35(5), pages 425-428, October.
    18. Ronald D. Fricker, 2009. "Editorial: The Eighth Rothkopf Rankings of Universities' Contributions to the INFORMS Practice Literature," Interfaces, INFORMS, vol. 39(6), pages 533-539, December.
    19. Ronald D. Fricker, 2011. "Editorial: The Ninth Rothkopf Rankings of Universities' Contributions to the INFORMS Practice Literature," Interfaces, INFORMS, vol. 41(6), pages 590-598, December.
    20. Biehl, Markus & Kim, Henry & Wade, Michael, 2006. "Relationships among the academic business disciplines: a multi-method citation analysis," Omega, Elsevier, vol. 34(4), pages 359-371, August.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:jomega:v:28:y:2000:i:2:p:241-245. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/375/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.