IDEAS home Printed from https://ideas.repec.org/p/tin/wpaper/20210042.html
   My bibliography  Save this paper

Bias in expert product reviews

Author

Listed:
  • Ben Vollaard

    (Tilburg University)

  • Jan C. van Ours

    (Erasmus University Rotterdam)

Abstract

Hard evidence on bias in expert reviews and its consequences for ratings is rare. This holds particularly true for conflicts of interest that are thought to be common in non-blind product reviews but are not readily observable: ad hoc relationships between reviewers and producers. We present a textbook case of a long-running expert product review in the food service industry for which we happen to know the reviewer's conflict of interest: being affiliated to one particular producer. As is typical, only insiders were aware of the possible source of bias in the review. The review resembles other non-blind tests of product quality. We obtained detailed data to map the consequences of the conflict of interest. We find evidence of a sizable bias in the reviewers' ratings. Our findings suggest that reviewers' ad hoc relationships with producers, often dismissed as `coming with the job', can be very harmful.

Suggested Citation

  • Ben Vollaard & Jan C. van Ours, 2021. "Bias in expert product reviews," Tinbergen Institute Discussion Papers 21-042/V, Tinbergen Institute.
    • Handle: RePEc:tin:wpaper:20210042
    as

    Download full text from publisher

    File URL: https://papers.tinbergen.nl/21042.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Dina Mayzlin & Yaniv Dover & Judith Chevalier, 2014. "Promotional Reviews: An Empirical Investigation of Online Review Manipulation," American Economic Review, American Economic Association, vol. 104(8), pages 2421-2455, August.
    2. Santosh Anagol & Shawn Cole & Shayak Sarkar, 2017. "Understanding the Advice of Commissions-Motivated Agents: Evidence from the Indian Life Insurance Market," The Review of Economics and Statistics, MIT Press, vol. 99(1), pages 1-15, March.
    3. Lin, Hsiou-wei & McNichols, Maureen F., 1998. "Underwriting relationships, analysts' earnings forecasts and investment recommendations," Journal of Accounting and Economics, Elsevier, vol. 25(1), pages 101-127, February.
    4. Dobrescu, Loretti I. & Luca, Michael & Motta, Alberto, 2013. "What makes a critic tick? Connected authors and the determinants of book reviews," Journal of Economic Behavior & Organization, Elsevier, vol. 96(C), pages 85-103.
    5. Victor A. Ginsburgh & Jan C. van Ours, 2003. "Expert Opinion and Compensation: Evidence from a Musical Competition," American Economic Review, American Economic Association, vol. 93(1), pages 289-296, March.
    6. Tommaso Colussi, 2018. "Social Ties in Academia: A Friend Is a Treasure," The Review of Economics and Statistics, MIT Press, vol. 100(1), pages 45-50, March.
    7. Alessandro Lizzeri, 1999. "Information Revelation and Certification Intermediaries," RAND Journal of Economics, The RAND Corporation, vol. 30(2), pages 214-231, Summer.
    8. Colussi, Tommaso, 2015. "Social Ties in Academia: A Friend is a Treasure," IZA Discussion Papers 9414, Institute of Labor Economics (IZA).
    9. Stefano Dellavigna & Johannes Hermle, 2017. "Does Conflict of Interest Lead to Biased Coverage? Evidence from Movie Reviews," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 84(4), pages 1510-1550.
    10. Dewenter Ralf & Heimeshoff Ulrich, 2014. "Media Bias and Advertising: Evidence from a German Car Magazine," Review of Economics, De Gruyter, vol. 65(1), pages 77-94, April.
    11. Michael Anderson & Jeremy Magruder, 2012. "Learning from the Crowd: Regression Discontinuity Estimates of the Effects of an Online Review Database," Economic Journal, Royal Economic Society, vol. 122(563), pages 957-989, September.
    12. Tom Hamami, 2019. "Network Effects, Bargaining Power, and Product Review Bias: Theory and Evidence," Journal of Industrial Economics, Wiley Blackwell, vol. 67(2), pages 372-407, June.
    13. Cecilia Rouse & Claudia Goldin, 2000. "Orchestrating Impartiality: The Impact of "Blind" Auditions on Female Musicians," American Economic Review, American Economic Association, vol. 90(4), pages 715-741, September.
    14. Reuter, Jonathan, 2009. "Does Advertising Bias Product Reviews? An Analysis of Wine Ratings," Journal of Wine Economics, Cambridge University Press, vol. 4(2), pages 125-151, January.
    15. Michaely, Roni & Womack, Kent L, 1999. "Conflict of Interest and the Credibility of Underwriter Analyst Recommendations," The Review of Financial Studies, Society for Financial Studies, vol. 12(4), pages 653-686.
    16. Véronique Chossat & Olivier Gergaud, 2003. "Expert Opinion and Gastronomy: The Recipe for Success," Journal of Cultural Economics, Springer;The Association for Cultural Economics International, vol. 27(2), pages 127-141, May.
    17. Imke Reimers & Joel Waldfogel, 2021. "Digitization and Pre-purchase Information: The Causal and Welfare Impacts of Reviews and Crowd Ratings," American Economic Review, American Economic Association, vol. 111(6), pages 1944-1971, June.
    18. Goldin, Claudia D. & Rouse, Cecilia, 2000. "Orchestrating Impartiality: The Impact of “Blind†Auditions on Female Musicians," Scholarly Articles 30703974, Harvard University Department of Economics.
    19. David A. Reinstein & Christopher M. Snyder, 2005. "The Influence Of Expert Reviews On Consumer Demand For Experience Goods: A Case Study Of Movie Critics," Journal of Industrial Economics, Wiley Blackwell, vol. 53(1), pages 27-51, March.
    20. Yubo Chen & Jinhong Xie, 2005. "Third-Party Product Review and Firm Marketing Strategy," Marketing Science, INFORMS, vol. 24(2), pages 218-240, February.
    21. Amanda Hollenbacher & David Yerger, 2001. "Third party evaluations and resale prices in the US used vehicle market," Applied Economics Letters, Taylor & Francis Journals, vol. 8(6), pages 415-418.
    22. S. Ravid & John Wald & Suman Basuroy, 2006. "Distributors and film critics: does it take two to Tango?," Journal of Cultural Economics, Springer;The Association for Cultural Economics International, vol. 30(3), pages 201-218, December.
    23. Danielle Li, 2017. "Expertise versus Bias in Evaluation: Evidence from the NIH," American Economic Journal: Applied Economics, American Economic Association, vol. 9(2), pages 60-92, April.
    24. Victor Ginsburgh & Renato Flores Galvao, 1996. "The Queen Elisabeth Musical Competition: how fair is the final ranking," ULB Institutional Repository 2013/1713, ULB -- Universite Libre de Bruxelles.
    25. Tom Hamami & James Bailey, 2021. "Expert product reviews and conflict of interest," Managerial and Decision Economics, John Wiley & Sons, Ltd., vol. 42(1), pages 170-176, January.
    26. Ashenfelter, Orley & Jones, Gregory V., 2013. "The Demand for Expert Opinion: Bordeaux Wine," Journal of Wine Economics, Cambridge University Press, vol. 8(3), pages 285-293, December.
    27. Michael Luca & Georgios Zervas, 2016. "Fake It Till You Make It: Reputation, Competition, and Yelp Review Fraud," Management Science, INFORMS, vol. 62(12), pages 3412-3427, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Asmat, Roberto & Borowiecki, Karol J. & Law, Marc T., 2023. "Do experts and laypersons differ? Some evidence from international classical music competitions," Journal of Economic Behavior & Organization, Elsevier, vol. 214(C), pages 270-290.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Tom Hamami & James Bailey, 2021. "Expert product reviews and conflict of interest," Managerial and Decision Economics, John Wiley & Sons, Ltd., vol. 42(1), pages 170-176, January.
    2. Stefano Dellavigna & Johannes Hermle, 2017. "Does Conflict of Interest Lead to Biased Coverage? Evidence from Movie Reviews," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 84(4), pages 1510-1550.
    3. Jonathan Reuter & Eric Zitzewitz, 2006. "Do Ads Influence Editors? Advertising and Bias in the Financial Media," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 121(1), pages 197-227.
    4. Warut Khern-am-nuai & Karthik Kannan & Hossein Ghasemkhani, 2018. "Extrinsic versus Intrinsic Rewards for Contributing Reviews in an Online Platform," Information Systems Research, INFORMS, vol. 29(4), pages 871-892, December.
    5. Brett Hollenbeck & Sridhar Moorthy & Davide Proserpio, 2019. "Advertising Strategy in the Presence of Reviews: An Empirical Analysis," Marketing Science, INFORMS, vol. 38(5), pages 793-811, September.
    6. Tom Hamami, 2019. "Network Effects, Bargaining Power, and Product Review Bias: Theory and Evidence," Journal of Industrial Economics, Wiley Blackwell, vol. 67(2), pages 372-407, June.
    7. Alex Albright & Peter Pedroni & Stephen Sheppard, 2018. "Uncorking Expert Reviews with Social Media: A Case Study Served with Wine," Department of Economics Working Papers 2018-03, Department of Economics, Williams College.
    8. Florysiak, David & Schandlbauer, Alexander, 2022. "Experts or charlatans? ICO analysts and white paper informativeness," Journal of Banking & Finance, Elsevier, vol. 139(C).
    9. Jacobsen, Grant D., 2015. "Consumers, experts, and online product evaluations: Evidence from the brewing industry," Journal of Public Economics, Elsevier, vol. 126(C), pages 114-123.
    10. Emmanuel Paroissien & Michael Visser, 2020. "The Causal Impact of Medals on Wine Producers' Prices and the Gains from Participating in Contests," American Journal of Agricultural Economics, John Wiley & Sons, vol. 102(4), pages 1135-1153, August.
    11. Sung H. Ham & Ingrid Koch & Noah Lim & Jiabin Wu, 2021. "Conflict of Interest in Third-Party Reviews: An Experimental Study," Management Science, INFORMS, vol. 67(12), pages 7535-7559, December.
    12. Kevin Chung & Keehyung Kim & Noah Lim, 2020. "Social Structures and Reputation in Expert Review Systems," Management Science, INFORMS, vol. 66(7), pages 3249-3276, July.
    13. Hui, Xiang & Klein, Tobias & Stahl, Konrad, 2022. "Learning from Online Ratings," CEPR Discussion Papers 17006, C.E.P.R. Discussion Papers.
    14. Ginsburgh, Victor & Radermecker, Anne-Sophie & Tommasi, Denni, 2019. "The effect of experts’ opinion on prices of art works: The case of Peter Brueghel the Younger," Journal of Economic Behavior & Organization, Elsevier, vol. 159(C), pages 36-50.
    15. Gesche, Tobias, 2018. "Reference Price Shifts and Customer Antagonism: Evidence from Reviews for Online Auctions," VfS Annual Conference 2018 (Freiburg, Breisgau): Digital Economy 181650, Verein für Socialpolitik / German Economic Association.
    16. Zhuang, Mengzhou & Cui, Geng & Peng, Ling, 2018. "Manufactured opinions: The effect of manipulating online product reviews," Journal of Business Research, Elsevier, vol. 87(C), pages 24-35.
    17. Di Lizia, Adam, 2024. "Social Influence in Online Reviews : Evidence from the Steam Store," The Warwick Economics Research Paper Series (TWERPS) 1505, University of Warwick, Department of Economics.
    18. Pierre Deschamps, 2018. "Gender Quotas in Hiring Committees: a Boon or a Bane for Women?," Working Papers hal-03393117, HAL.
    19. Ralf Dewenter & Ulrich Heimeshoff & Tobias Thomas, 2016. "Media Coverage and Car Manufacturers' Sales," Economics Bulletin, AccessEcon, vol. 36(2), pages 976-982.
    20. Ralf Dewenter & Ulrich Heimeshoff, 2015. "Do expert reviews really drive demand? Evidence from a German car magazine," Applied Economics Letters, Taylor & Francis Journals, vol. 22(14), pages 1150-1153, September.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:tin:wpaper:20210042. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Tinbergen Office +31 (0)10-4088900 (email available below). General contact details of provider: https://edirc.repec.org/data/tinbenl.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.