IDEAS home Printed from https://ideas.repec.org/p/ces/ceswps/_11422.html
   My bibliography  Save this paper

Bad Apples on Rotten Tomatoes: Critics, Crowds, and Gender Bias in Product Ratings

Author

Listed:
  • Luis Aguiar

Abstract

Consumers considering the purchase of experience goods can rely on both critics and crowd-based evaluations to guide their decisions. Due to divergent incentives, however, critics and crowd assessments may incorporate different information. In the context of the movie industry, I investigate whether crowd reviewers provide gender-neutral product evaluations relative to professional critics. I classify movies as male or female based on the gender composition of their cast and estimate how the gender gap in movie rating scores differs across critics and crowds. Results show that while critics tend to assess both male and female movies similarly, the gender gap in ratings increases substantially under crowd-based ratings and at the expense of female movies. Notably, female movies receive a higher proportion of extreme low ratings, predominantly from male crowd reviewers. Using a rating design change implemented by the review-aggregating platform Rotten Tomatoes, results indicate that this overall increase in gender inequality is driven by a selected group of online reviewers rather than by a general bias against movies with more prominent female presence. These findings have important implications for the gate-keeping role of review-aggregating platforms in reducing bias against female representation in product ratings.

Suggested Citation

  • Luis Aguiar, 2024. "Bad Apples on Rotten Tomatoes: Critics, Crowds, and Gender Bias in Product Ratings," CESifo Working Paper Series 11422, CESifo.
  • Handle: RePEc:ces:ceswps:_11422
    as

    Download full text from publisher

    File URL: https://www.cesifo.org/DocDL/cesifo1_wp11422.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Imke Reimers & Joel Waldfogel, 2021. "Digitization and Pre-purchase Information: The Causal and Welfare Impacts of Reviews and Crowd Ratings," American Economic Review, American Economic Association, vol. 111(6), pages 1944-1971, June.
    2. Dina Mayzlin & Yaniv Dover & Judith Chevalier, 2014. "Promotional Reviews: An Empirical Investigation of Online Review Manipulation," American Economic Review, American Economic Association, vol. 104(8), pages 2421-2455, August.
    3. David A. Reinstein & Christopher M. Snyder, 2005. "The Influence Of Expert Reviews On Consumer Demand For Experience Goods: A Case Study Of Movie Critics," Journal of Industrial Economics, Wiley Blackwell, vol. 53(1), pages 27-51, March.
    4. Hong Luo & Laurina Zhang, 2022. "Scandal, Social Movement, and Change: Evidence from #MeToo in Hollywood," Management Science, INFORMS, vol. 68(2), pages 1278-1296, February.
    5. Fowdur, Lona & Kadiyali, Vrinda & Prince, Jeffrey, 2012. "Racial bias in expert quality assessment: A study of newspaper movie reviews," Journal of Economic Behavior & Organization, Elsevier, vol. 84(1), pages 292-307.
    6. Sherry He & Brett Hollenbeck & Davide Proserpio, 2022. "The Market for Fake Reviews," Marketing Science, INFORMS, vol. 41(5), pages 896-921, September.
    7. Ethan Mollick & Ramana Nanda, 2016. "Wisdom or Madness? Comparing Crowds with Expert Evaluation in Funding the Arts," Management Science, INFORMS, vol. 62(6), pages 1533-1553, June.
    8. J. Aislinn Bohren & Alex Imas & Michael Rosenberg, 2019. "The Dynamics of Discrimination: Theory and Evidence," American Economic Review, American Economic Association, vol. 109(10), pages 3395-3436, October.
    9. Anja Lambrecht & Catherine Tucker, 2019. "Algorithmic Bias? An Empirical Study of Apparent Gender-Based Discrimination in the Display of STEM Career Ads," Management Science, INFORMS, vol. 65(7), pages 2966-2981, July.
    10. David Godes & Dina Mayzlin, 2004. "Using Online Conversations to Study Word-of-Mouth Communication," Marketing Science, INFORMS, vol. 23(4), pages 545-560, June.
    11. Aguiar, Luis & Waldfogel, Joel & Waldfogel, Sarah, 2021. "Playlisting favorites: Measuring platform bias in the music industry," International Journal of Industrial Organization, Elsevier, vol. 78(C).
    12. Cecilia Rouse & Claudia Goldin, 2000. "Orchestrating Impartiality: The Impact of "Blind" Auditions on Female Musicians," American Economic Review, American Economic Association, vol. 90(4), pages 715-741, September.
    13. Hinnosaar, Marit, 2019. "Gender inequality in new media: Evidence from Wikipedia," Journal of Economic Behavior & Organization, Elsevier, vol. 163(C), pages 262-276.
    14. Holbrook, Morris B, 1999. "Popular Appeal versus Expert Judgments of Motion Pictures," Journal of Consumer Research, Journal of Consumer Research Inc., vol. 26(2), pages 144-155, September.
    15. Joel Waldfogel, 2023. "The Welfare Effect of Gender-Inclusive Intellectual Property Creation: Evidence from Books," NBER Working Papers 30987, National Bureau of Economic Research, Inc.
    16. Bikhchandani, Sushil & Hirshleifer, David & Welch, Ivo, 1992. "A Theory of Fads, Fashion, Custom, and Cultural Change in Informational Cascades," Journal of Political Economy, University of Chicago Press, vol. 100(5), pages 992-1026, October.
    17. Davide Proserpio & Isamar Troncoso & Francesca Valsesia, 2021. "Does Gender Matter? The Effect of Management Responses on Reviewing Behavior," Marketing Science, INFORMS, vol. 40(6), pages 1199-1213, November.
    18. Xinxin Li & Lorin M. Hitt, 2008. "Self-Selection and Information Role of Online Product Reviews," Information Systems Research, INFORMS, vol. 19(4), pages 456-474, December.
    19. Michael Luca & Georgios Zervas, 2016. "Fake It Till You Make It: Reputation, Competition, and Yelp Review Fraud," Management Science, INFORMS, vol. 62(12), pages 3412-3427, December.
    20. David Godes & José C. Silva, 2012. "Sequential and Temporal Dynamics of Online Opinion," Marketing Science, INFORMS, vol. 31(3), pages 448-473, May.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Luis Aguiar & Imke Reimers & Joel Waldfogel, 2024. "Platforms and the transformation of the content industries," Journal of Economics & Management Strategy, Wiley Blackwell, vol. 33(2), pages 317-326, March.
    2. Weijia (Daisy) Dai & Ginger Jin & Jungmin Lee & Michael Luca, 2018. "Aggregation of consumer ratings: an application to Yelp.com," Quantitative Marketing and Economics (QME), Springer, vol. 16(3), pages 289-339, September.
    3. Hui, Xiang & Klein, Tobias & Stahl, Konrad, 2022. "Learning from Online Ratings," CEPR Discussion Papers 17006, C.E.P.R. Discussion Papers.
    4. Sungsik Park & Woochoel Shin & Jinhong Xie, 2021. "The Fateful First Consumer Review," Marketing Science, INFORMS, vol. 40(3), pages 481-507, May.
    5. Vollaard, Ben & van Ours, Jan C., 2022. "Bias in expert product reviews," Journal of Economic Behavior & Organization, Elsevier, vol. 202(C), pages 105-118.
    6. Tao Lu & May Yuan & Chong (Alex) Wang & Xiaoquan (Michael) Zhang, 2022. "Histogram Distortion Bias in Consumer Choices," Management Science, INFORMS, vol. 68(12), pages 8963-8978, December.
    7. Dominik Gutt & Jürgen Neumann & Steffen Zimmermann & Dennis Kundisch & Jianqing Chen, 2018. "Design of Review Systems - A Strategic Instrument to shape Online Review Behavior and Economic Outcomes," Working Papers Dissertations 42, Paderborn University, Faculty of Business Administration and Economics.
    8. Cheng Zhao & Chong Alex Wang, 2023. "A cross-site comparison of online review manipulation using Benford’s law," Electronic Commerce Research, Springer, vol. 23(1), pages 365-406, March.
    9. Georgios Zervas & Davide Proserpio & John W. Byers, 2021. "A first look at online reputation on Airbnb, where every stay is above average," Marketing Letters, Springer, vol. 32(1), pages 1-16, March.
    10. Warut Khern-am-nuai & Karthik Kannan & Hossein Ghasemkhani, 2018. "Extrinsic versus Intrinsic Rewards for Contributing Reviews in an Online Platform," Information Systems Research, INFORMS, vol. 29(4), pages 871-892, December.
    11. Koukova, Nevena T. & Wang, Rebecca Jen-Hui & Isaac, Mathew S., 2023. "“If you loved our product”: Do conditional review requests harm retailer loyalty?," Journal of Retailing, Elsevier, vol. 99(1), pages 85-101.
    12. Davide Proserpio & Georgios Zervas, 2017. "Online Reputation Management: Estimating the Impact of Management Responses on Consumer Reviews," Marketing Science, INFORMS, vol. 36(5), pages 645-665, September.
    13. Heeseung Andrew Lee & Angela Aerry Choi & Tianshu Sun & Wonseok Oh, 2021. "Reviewing Before Reading? An Empirical Investigation of Book-Consumption Patterns and Their Effects on Reviews and Sales," Information Systems Research, INFORMS, vol. 32(4), pages 1368-1389, December.
    14. Alexei Parahonyak & Nick Vikander, 2024. "Strategic Use of Product Delays to Shape Word-of-Mouth Communication," Economics Series Working Papers 1032, University of Oxford, Department of Economics.
    15. Ishita Chakraborty & Minkyung Kim & K. Sudhir, 2019. "Attribute Sentiment Scoring With Online Text Reviews : Accounting for Language Structure and Attribute Self-Selection," Cowles Foundation Discussion Papers 2176R2, Cowles Foundation for Research in Economics, Yale University, revised Jun 2021.
    16. Chong (Alex) Wang & Xiaoquan (Michael) Zhang & Il-Horn Hann, 2018. "Socially Nudged: A Quasi-Experimental Study of Friends’ Social Influence in Online Product Ratings," Information Systems Research, INFORMS, vol. 29(3), pages 641-655, September.
    17. Balázs Kovács, 2024. "The Turing test of online reviews: Can we tell the difference between human-written and GPT-4-written online reviews?," Marketing Letters, Springer, vol. 35(4), pages 651-666, December.
    18. Pei-Yu Chen & Yili Hong & Ying Liu, 2018. "The Value of Multidimensional Rating Systems: Evidence from a Natural Experiment and Randomized Experiments," Management Science, INFORMS, vol. 64(10), pages 4629-4647, October.
    19. Ishita Chakraborty & Minkyung Kim & K. Sudhir, 2019. "Attribute Sentiment Scoring With Online Text Reviews : Accounting for Language Structure and Attribute Self-Selection," Cowles Foundation Discussion Papers 2176R, Cowles Foundation for Research in Economics, Yale University, revised Sep 2020.
    20. Young Joon Park & Jaewoo Joo & Charin Polpanumas & Yeujun Yoon, 2021. "“Worse Than What I Read?” The External Effect of Review Ratings on the Online Review Generation Process: An Empirical Analysis of Multiple Product Categories Using Amazon.com Review Data," Sustainability, MDPI, vol. 13(19), pages 1-22, September.

    More about this item

    Keywords

    digital platforms; product ratings; critics; crowds; gender bias; movie industry;
    All these keywords.

    JEL classification:

    • D83 - Microeconomics - - Information, Knowledge, and Uncertainty - - - Search; Learning; Information and Knowledge; Communication; Belief; Unawareness
    • L15 - Industrial Organization - - Market Structure, Firm Strategy, and Market Performance - - - Information and Product Quality
    • L82 - Industrial Organization - - Industry Studies: Services - - - Entertainment; Media
    • O33 - Economic Development, Innovation, Technological Change, and Growth - - Innovation; Research and Development; Technological Change; Intellectual Property Rights - - - Technological Change: Choices and Consequences; Diffusion Processes

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ces:ceswps:_11422. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Klaus Wohlrabe (email available below). General contact details of provider: https://edirc.repec.org/data/cesifde.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.