IDEAS home Printed from https://ideas.repec.org/a/sgm/pzwzuw/v12i45y2014p137-155.html
   My bibliography  Save this article

Ewaluacja projektow i abstraktow – wplyw indywidualnego stylu ewaluacji na oceny (Evaluation of grant proposals and abstracts – the influence of individual evaluation style on ratings)

Author

Listed:
  • Grzegorz Krol

    (Wydzial Zarzadzania, Uniwersytet Warszawskii)

  • Katarzyna Kinga Kowalczyk

    (Wydzial Zarzadzania, Uniwersytet Warszawski)

Abstract

In this article we present an outline of psychological model of evaluation and results of ratings analysis for 673 abstracts submitted to international conference and 323 grant proposals. In both cases one evaluator (reviewer) evaluated many proposals or abstracts, which allowed us to demonstrate individual evaluation style (for 18 grant proposals evaluators and 33 abstract reviewers), operationalized by leniency/severity level and degree of ratings differentiation. We showed as well relation between rank order and severity rating (first 15 abstracts were rated more severely than others). It is suggested that this relation can be explained by two alternative explanations: stabilization of implicit comparative standard and fatigue.

Suggested Citation

  • Grzegorz Krol & Katarzyna Kinga Kowalczyk, 2014. "Ewaluacja projektow i abstraktow – wplyw indywidualnego stylu ewaluacji na oceny (Evaluation of grant proposals and abstracts – the influence of individual evaluation style on ratings)," Problemy Zarzadzania, University of Warsaw, Faculty of Management, vol. 12(45), pages 137-155.
    • Handle: RePEc:sgm:pzwzuw:v:12:i:45:y:2014:p:137-155
    as

    Download full text from publisher

    File URL: http://pz.wz.uw.edu.pl/sites/default/files/artykuly/pz_1_2014_krol_kowalczyk.pdf
    Download Restriction: no
    File URL: http://pz.wz.uw.edu.pl/en
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Stefan D. Haigner & Stefan Jenewein & Hans-Christian Müller & Florian Wakolbinger, 2010. "The first shall be last: Serial position effects in the case contestants evaluate each other," Economics Bulletin, AccessEcon, vol. 30(4), pages 3170-3176.
    2. Victor Ginsburgh & Renato Flores Galvao, 1996. "The Queen Elisabeth Musical Competition: how fair is the final ranking," ULB Institutional Repository 2013/1713, ULB -- Universite Libre de Bruxelles.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Pannicke, Julia, 2015. "Abstimmungsverhalten im Bundesvision Song Contest: Regionale Nähe versus Qualität der Musik," Ilmenau Economics Discussion Papers 95, Ilmenau University of Technology, Institute of Economics.
    2. GINSBURGH, Victor & NOURY, Abdul, 2005. "Cultural voting : The Eurovision Song Contest," LIDAM Discussion Papers CORE 2005006, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
    3. David Schüller & Thorsten Upmann, 2013. "When Focal Points are Out of Focus: A Game-Theoretic Analysis of Come Dine with Me," CESifo Working Paper Series 4138, CESifo.
    4. Budzinski, Oliver & Kohlschreiber, Marie & Kuchinke, Björn & Pannicke, Julia, 2019. "Does music quality matter for audience voters in a music contest?," Ilmenau Economics Discussion Papers 122, Ilmenau University of Technology, Institute of Economics.
    5. Victor Ginsburgh & Sheila Weyers, 2008. "Quantitative approaches to valuation in the arts, with an application to movies," ULB Institutional Repository 2013/7292, ULB -- Universite Libre de Bruxelles.
    6. Ginsburgh, Victor & Noury, Abdul G., 2008. "The Eurovision Song Contest. Is voting political or cultural?," European Journal of Political Economy, Elsevier, vol. 24(1), pages 41-52, March.
    7. Clémence Christin, 2013. "Entry Deterrence Through Cooperative R&D Over-Investment," Recherches économiques de Louvain, De Boeck Université, vol. 79(2), pages 5-26.
    8. Sanne J. Joustra & Ruud H. Koning & Alex Krumer, 2021. "Order Effects in Elite Gymnastics," De Economist, Springer, vol. 169(1), pages 21-35, February.
    9. V. Ginsburgh & Sheila Weyers, 2014. "Nominees, winners, and losers," Journal of Cultural Economics, Springer;The Association for Cultural Economics International, vol. 38(4), pages 291-313, November.
    10. Victor A. Ginsburgh & Jan C. van Ours, 2003. "Expert Opinion and Compensation: Evidence from a Musical Competition," American Economic Review, American Economic Association, vol. 93(1), pages 289-296, March.
    11. Oliver Budzinski & Julia Pannicke, 2017. "Culturally biased voting in the Eurovision Song Contest: Do national contests differ?," Journal of Cultural Economics, Springer;The Association for Cultural Economics International, vol. 41(4), pages 343-378, November.
    12. Vollaard, Ben & van Ours, Jan C., 2022. "Bias in expert product reviews," Journal of Economic Behavior & Organization, Elsevier, vol. 202(C), pages 105-118.
    13. Kurt W. Rotthoff, 2015. "(Not Finding a) Sequential Order Bias in Elite Level Gymnastics," Southern Economic Journal, John Wiley & Sons, vol. 81(3), pages 724-741, January.
    14. Gu, Yiquan & Wenzel, Tobias, 2012. "Transparency, entry, and productivity," Economics Letters, Elsevier, vol. 115(1), pages 7-10.
    15. Victor Ginsburgh & Juan D. Moreno-Ternero, 2023. "The Eurovision Song Contest: voting rules, biases and rationality," Journal of Cultural Economics, Springer;The Association for Cultural Economics International, vol. 47(2), pages 247-277, June.
    16. Hillary N. Morgan & Kurt W. Rotthoff, 2014. "The Harder The Task, The Higher The Score: Findings Of A Difficulty Bias," Economic Inquiry, Western Economic Association International, vol. 52(3), pages 1014-1026, July.
    17. Kurt W. Rotthoff, 2015. "(Not Finding a) Sequential Order Bias in Elite Level Gymnastics," Southern Economic Journal, Southern Economic Association, vol. 81(3), pages 724-741, January.
    18. Budzinski, Oliver & Gänßle, Sophia & Weimar, Daniel, 2023. "Disentangling individual biases in jury voting: An empirical analysis of voting behavior in the Eurovision Song Contest," Ilmenau Economics Discussion Papers 171, Ilmenau University of Technology, Institute of Economics.
    19. Stühmeier Torben & Wenzel Tobias, 2012. "Regulating Advertising in the Presence of Public Service Broadcasting," Review of Network Economics, De Gruyter, vol. 11(2), pages 1-23, June.
    20. Krzysztof Kontek & Honorata Sosnowska, 2020. "Specific Tastes or Cliques of Jurors? How to Reduce the Level of Manipulation in Group Decisions?," Group Decision and Negotiation, Springer, vol. 29(6), pages 1057-1084, December.

    More about this item

    Keywords

    evaluation; implicit standards; reviews; leniency terror; ratings differentiation;
    All these keywords.

    JEL classification:

    • C18 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Methodolical Issues: General

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sgm:pzwzuw:v:12:i:45:y:2014:p:137-155. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: the person in charge (email available below). General contact details of provider: https://edirc.repec.org/data/somuwpl.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.