IDEAS home Printed from https://ideas.repec.org/a/bla/popmgt/v28y2019i1p176-188.html
   My bibliography  Save this article

Scoring vs. Ranking: An Experimental Study of Idea Evaluation Processes

Author

Listed:
  • Zhijian Cui
  • Shijith Kumar PM
  • Dilney Gonçalves

Abstract

Using an online experiment, this study examines the efficacy of two idea evaluation processes: scoring vs. ranking in the context of innovation management. In the scoring process, the evaluators are asked to rate the quality of each idea by assigning it a score (e.g., from 0 to 10), while in the ranking process the evaluator simply orders all ideas according to their perceived qualities. Our results suggest that the scoring process strictly outperforms the ranking process in terms of the likelihood of selecting the highest‐quality ideas. This result remains robust, irrespective of the possibility of allowing ties in the ranking process. However, when the number of ideas to be evaluated is reduced from eight to three, the efficacies of the two idea evaluation processes become similar. Additionally, we find that the efficacy of the ranking process is significantly improved when additional information is provided, yet the efficacy of the scoring process does not change with more information. Based on the observations from the experimental data, we propose and test an explanatory model in which the information becomes a cue that directs the participants’ efforts to evaluate the ideas (i.e., the time taken for evaluation).

Suggested Citation

  • Zhijian Cui & Shijith Kumar PM & Dilney Gonçalves, 2019. "Scoring vs. Ranking: An Experimental Study of Idea Evaluation Processes," Production and Operations Management, Production and Operations Management Society, vol. 28(1), pages 176-188, January.
    • Handle: RePEc:bla:popmgt:v:28:y:2019:i:1:p:176-188
    DOI: 10.1111/poms.12910
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/poms.12910
    Download Restriction: no
    File URL: https://libkey.io/10.1111/poms.12910?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Young In Koh & Sung H. Han & Junseong Park, 2022. "A systematic process for generating new blockchain-service business model ideas," Service Business, Springer;Pan-Pacific Business Association, vol. 16(1), pages 187-209, March.
    2. Stylianos Kavadias & Karl T. Ulrich, 2020. "Innovation and New Product Development: Reflections and Insights from the Research Published in the First 20 Years of Manufacturing & Service Operations Management," Manufacturing & Service Operations Management, INFORMS, vol. 22(1), pages 84-92, January.
    3. Dahlander, Linus & Beretta, Michela & Thomas, Arne & Kazemi, Shahab & Fenger, Morten H.J. & Frederiksen, Lars, 2023. "Weeding out or picking winners in open innovation? Factors driving multi-stage crowd selection on LEGO ideas," Research Policy, Elsevier, vol. 52(10).

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:popmgt:v:28:y:2019:i:1:p:176-188. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no bibliographic references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: http://onlinelibrary.wiley.com/journal/10.1111/(ISSN)1937-5956 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.