Does Help Help? An Empirical Analysis of Social Desirability Bias in Ratings

Review-in-review (RIR) is a feature that allows viewers to generate positive or negative evaluations for primary quality evaluations of a product (e.g., ratings and reviews). This feature has the potential to reshape primary quality evaluations; specifically, it can cause social desirability bias in ratings, as raters (i.e., reviewers) who desire social recognition might be driven to provide ratings that are expected to gain more “helpful” and avoid unhelpful RIRs. This study aims to isolate this bias. Specifically, we develop and estimate a partially ordinal discrete choice model that allows rating responses to reflect a mixture of a conditional multinomial discrete choice model that captures the RIR-induced social desirability incentive and an ordinal discrete choice model that reflects the baseline incentive of quality perception. From the estimation results, we find evidence that individuals rate, in part, to satisfy social desirability, designing the rating to be more helpful, less unhelpful, and generate more text replies. This suggests a social desirability bias in ratings attributable to the expected RIRs. The raters, on average, attribute approximately 7.4% of the rating likelihood to the social desirability incentive, but the attribution varies across individuals, depending on their social characteristics. We further conduct various simulations under counterfactual RIR system designs to present the social desirability bias in ratings caused by each system and provide guidance on how to design an RIR system to alleviate such bias. Our robustness check suggests the presence of RIR-induced social desirability bias in the sentiment of the review.