IDEAS home Printed from https://ideas.repec.org/a/inm/orisre/v35y2024i3p1052-1073.html
   My bibliography  Save this article

Does Help Help? An Empirical Analysis of Social Desirability Bias in Ratings

Author

Listed:
  • Jinyang Zheng

    (Daniels School of Business, Purdue University, West Lafayette, Indiana 47907)

  • Guopeng Yin

    (School of Information, University of International Business and Economics, Beijing 100029, China)

  • Yong Tan

    (Michael G. Foster School of Business, University of Washington, Seattle, Washington 98195)

  • Jianing Ding

    (Daniels School of Business, Purdue University, West Lafayette, Indiana 47907)

Abstract

Review-in-review (RIR) is a feature that allows viewers to generate positive or negative evaluations for primary quality evaluations of a product (e.g., ratings and reviews). This feature has the potential to reshape primary quality evaluations; specifically, it can cause social desirability bias in ratings, as raters (i.e., reviewers) who desire social recognition might be driven to provide ratings that are expected to gain more “helpful” and avoid unhelpful RIRs. This study aims to isolate this bias. Specifically, we develop and estimate a partially ordinal discrete choice model that allows rating responses to reflect a mixture of a conditional multinomial discrete choice model that captures the RIR-induced social desirability incentive and an ordinal discrete choice model that reflects the baseline incentive of quality perception. From the estimation results, we find evidence that individuals rate, in part, to satisfy social desirability, designing the rating to be more helpful, less unhelpful, and generate more text replies. This suggests a social desirability bias in ratings attributable to the expected RIRs. The raters, on average, attribute approximately 7.4% of the rating likelihood to the social desirability incentive, but the attribution varies across individuals, depending on their social characteristics. We further conduct various simulations under counterfactual RIR system designs to present the social desirability bias in ratings caused by each system and provide guidance on how to design an RIR system to alleviate such bias. Our robustness check suggests the presence of RIR-induced social desirability bias in the sentiment of the review.

Suggested Citation

  • Jinyang Zheng & Guopeng Yin & Yong Tan & Jianing Ding, 2024. "Does Help Help? An Empirical Analysis of Social Desirability Bias in Ratings," Information Systems Research, INFORMS, vol. 35(3), pages 1052-1073, September.
  • Handle: RePEc:inm:orisre:v:35:y:2024:i:3:p:1052-1073
    DOI: 10.1287/isre.2020.0406
    as

    Download full text from publisher

    File URL: http://dx.doi.org/10.1287/isre.2020.0406
    Download Restriction: no

    File URL: https://libkey.io/10.1287/isre.2020.0406?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Ling Huang & Martin D. Smith, 2014. "The Dynamic Efficiency Costs of Common-Pool Resource Exploitation," American Economic Review, American Economic Association, vol. 104(12), pages 4071-4103, December.
    2. Bajari, Patrick & Hong, Han & Krainer, John & Nekipelov, Denis, 2010. "Estimating Static Models of Strategic Interactions," Journal of Business & Economic Statistics, American Statistical Association, vol. 28(4), pages 469-482.
    3. Lynch, John G, Jr, 1982. "On the External Validity of Experiments in Consumer Research," Journal of Consumer Research, Journal of Consumer Research Inc., vol. 9(3), pages 225-239, December.
    4. Zhang, Jason Q. & Craciun, Georgiana & Shin, Dongwoo, 2010. "When does electronic word-of-mouth matter? A study of consumer product reviews," Journal of Business Research, Elsevier, vol. 63(12), pages 1336-1341, December.
    5. Chen, Xiaohong, 2007. "Large Sample Sieve Estimation of Semi-Nonparametric Models," Handbook of Econometrics, in: J.J. Heckman & E.E. Leamer (ed.), Handbook of Econometrics, edition 1, volume 6, chapter 76, Elsevier.
    6. Mochen Yang & Yuqing Ren & Gediminas Adomavicius, 2019. "Understanding User-Generated Content and Customer Engagement on Facebook Business Pages," Information Systems Research, INFORMS, vol. 30(3), pages 839-855, September.
    7. Geetha, M. & Singha, Pratap & Sinha, Sumedha, 2017. "Relationship between customer sentiment and online customer ratings for hotels - An empirical analysis," Tourism Management, Elsevier, vol. 61(C), pages 43-54.
    8. Moore, Ellen M & Bearden, William O & Teel, Jesse E, 1985. "Use of Labeling and Assertions of Dependency in Appeals for Consumer Support," Journal of Consumer Research, Journal of Consumer Research Inc., vol. 12(1), pages 90-96, June.
    9. Wendy W. Moe & David A. Schweidel, 2012. "Online Product Opinions: Incidence, Evaluation, and Evolution," Marketing Science, INFORMS, vol. 31(3), pages 372-386, May.
    10. Dokyun Lee & Kartik Hosanagar & Harikesh S. Nair, 2018. "Advertising Content and Consumer Engagement on Social Media: Evidence from Facebook," Management Science, INFORMS, vol. 64(11), pages 5105-5131, November.
    11. Hema Yoganarasimhan, 2016. "Estimation of Beauty Contest Auctions," Marketing Science, INFORMS, vol. 35(1), pages 27-54, January.
    12. Richins, Marsha L & Dawson, Scott, 1992. "A Consumer Values Orientation for Materialism and Its Measurement: Scale Development and Validation," Journal of Consumer Research, Journal of Consumer Research Inc., vol. 19(3), pages 303-316, December.
    13. Chunrong Ai & Xiaohong Chen, 2003. "Efficient Estimation of Models with Conditional Moment Restrictions Containing Unknown Functions," Econometrica, Econometric Society, vol. 71(6), pages 1795-1843, November.
    14. Fisher, Robert J, 1993. "Social Desirability Bias and the Validity of Indirect Questioning," Journal of Consumer Research, Journal of Consumer Research Inc., vol. 20(2), pages 303-315, September.
    15. Chrysanthos Dellarocas & Charles A. Wood, 2008. "The Sound of Silence in Online Feedback: Estimating Trading Risks in the Presence of Reporting Bias," Management Science, INFORMS, vol. 54(3), pages 460-476, March.
    16. Liangfei Qiu & Subodha Kumar, 2017. "Understanding Voluntary Knowledge Provision and Content Contribution Through a Social-Media-Based Prediction Market: A Field Experiment," Information Systems Research, INFORMS, vol. 28(3), pages 529-546, September.
    17. Yi-Chun (Chad) Ho & Junjie Wu & Yong Tan, 2017. "Disconfirmation Effect on Online Rating Behavior: A Structural Model," Information Systems Research, INFORMS, vol. 28(3), pages 626-642, September.
    18. Carlson, Les & Grossbart, Sanford, 1988. "Parental Style and Consumer Socialization of Children," Journal of Consumer Research, Journal of Consumer Research Inc., vol. 15(1), pages 77-94, June.
    19. Chong (Alex) Wang & Xiaoquan (Michael) Zhang & Il-Horn Hann, 2018. "Socially Nudged: A Quasi-Experimental Study of Friends’ Social Influence in Online Product Ratings," Information Systems Research, INFORMS, vol. 29(3), pages 641-655, September.
    20. Pan, Yue & Zhang, Jason Q., 2011. "Born Unequal: A Study of the Helpfulness of User-Generated Product Reviews," Journal of Retailing, Elsevier, vol. 87(4), pages 598-612.
    21. Xinxin Li & Lorin M. Hitt, 2008. "Self-Selection and Information Role of Online Product Reviews," Information Systems Research, INFORMS, vol. 19(4), pages 456-474, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Kunpeng Zhang & Wendy Moe, 2021. "Measuring Brand Favorability Using Large-Scale Social Media Data," Information Systems Research, INFORMS, vol. 32(4), pages 1128-1139, December.
    2. Foster, Joshua, 2022. "How rating mechanisms shape user search, quality inference and engagement in online platforms: Experimental evidence," Journal of Business Research, Elsevier, vol. 142(C), pages 791-807.
    3. Dominik Gutt & Jürgen Neumann & Steffen Zimmermann & Dennis Kundisch & Jianqing Chen, 2018. "Design of Review Systems - A Strategic Instrument to shape Online Review Behavior and Economic Outcomes," Working Papers Dissertations 42, Paderborn University, Faculty of Business Administration and Economics.
    4. Christoph Schneider & Markus Weinmann & Peter N.C. Mohr & Jan vom Brocke, 2021. "When the Stars Shine Too Bright: The Influence of Multidimensional Ratings on Online Consumer Ratings," Management Science, INFORMS, vol. 67(6), pages 3871-3898, June.
    5. Changseung Yoo & Eunae Yoo & Lu (Lucy) Yan & Alfonso Pedraza-Martinez, 2024. "Speak with One Voice? Examining Content Coordination and Social Media Engagement During Disasters," Information Systems Research, INFORMS, vol. 35(2), pages 551-569, June.
    6. Hui, Xiang & Klein, Tobias & Stahl, Konrad, 2022. "Learning from Online Ratings," CEPR Discussion Papers 17006, C.E.P.R. Discussion Papers.
    7. Xiang Hui & Tobias J. Klein & Konrad Stahl, 2021. "When and Why Do Buyers Rate in Online Markets?," CRC TR 224 Discussion Paper Series crctr224_2021_267v1, University of Bonn and University of Mannheim, Germany.
    8. Cheng Zhao & Chong Alex Wang, 2023. "A cross-site comparison of online review manipulation using Benford’s law," Electronic Commerce Research, Springer, vol. 23(1), pages 365-406, March.
    9. Marios Kokkodis & Theodoros Lappas, 2020. "Your Hometown Matters: Popularity-Difference Bias in Online Reputation Platforms," Information Systems Research, INFORMS, vol. 31(2), pages 412-430, June.
    10. Xue Bai & James R. Marsden & William T. Ross & Gang Wang, 2020. "A Note on the Impact of Daily Deals on Local Retailers’ Online Reputation: Mediation Effects of the Consumer Experience," Information Systems Research, INFORMS, vol. 31(4), pages 1132-1143, December.
    11. Juan Feng & Xin Li & Xiaoquan (Michael) Zhang, 2019. "Online Product Reviews-Triggered Dynamic Pricing: Theory and Evidence," Information Systems Research, INFORMS, vol. 30(4), pages 1107-1123, December.
    12. King, Robert Allen & Racherla, Pradeep & Bush, Victoria D., 2014. "What We Know and Don't Know About Online Word-of-Mouth: A Review and Synthesis of the Literature," Journal of Interactive Marketing, Elsevier, vol. 28(3), pages 167-183.
    13. Wei Chen & Bin Gu & Qiang Ye & Kevin Xiaoguo Zhu, 2019. "Measuring and Managing the Externality of Managerial Responses to Online Customer Reviews," Service Science, INFORMS, vol. 30(1), pages 81-96, March.
    14. Arslan Aziz & Hui Li & Rahul Telang, 2023. "The Consequences of Rating Inflation on Platforms: Evidence from a Quasi-Experiment," Information Systems Research, INFORMS, vol. 34(2), pages 590-608, June.
    15. Pei-Yu Chen & Yili Hong & Ying Liu, 2018. "The Value of Multidimensional Rating Systems: Evidence from a Natural Experiment and Randomized Experiments," Management Science, INFORMS, vol. 64(10), pages 4629-4647, October.
    16. Erfan Rezvani & Christian Rojas, 2022. "Firm responsiveness to consumers' reviews: The effect on online reputation," Journal of Economics & Management Strategy, Wiley Blackwell, vol. 31(4), pages 898-922, November.
    17. Janina Seutter & Jürgen Neumann, 2024. "Reviewing the simple things – How ease of evaluation affects online rating behavior," Working Papers Dissertations 120, Paderborn University, Faculty of Business Administration and Economics.
    18. Qian Wang & Michael Chau & Chih-Hung Peng & Eric W. T. Ngai, 2022. "Using the Anchoring Effect and the Cultural Dimensions Theory to Study Customers’ Online Rating Behaviors," Information Systems Frontiers, Springer, vol. 24(5), pages 1451-1463, October.
    19. Yue Guan & Yong Tan & Qiang Wei & Guoqing Chen, 2023. "When Images Backfire: The Effect of Customer-Generated Images on Product Rating Dynamics," Information Systems Research, INFORMS, vol. 34(4), pages 1641-1663, December.
    20. Zhihong Ke & De Liu & Daniel J. Brass, 2020. "Do Online Friends Bring Out the Best in Us? The Effect of Friend Contributions on Online Review Provision," Information Systems Research, INFORMS, vol. 31(4), pages 1322-1336, December.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:orisre:v:35:y:2024:i:3:p:1052-1073. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.