IDEAS home Printed from https://ideas.repec.org/a/inm/orijoc/v33y2021i1p246-261.html
   My bibliography  Save this article

Calibration of Voting-Based Helpfulness Measurement for Online Reviews: An Iterative Bayesian Probability Approach

Author

Listed:
  • Xunhua Guo

    (Research Center for Contemporary Management, School of Economics and Management, Tsinghua University, Beijing 100084, China)

  • Guoqing Chen

    (Research Center for Contemporary Management, School of Economics and Management, Tsinghua University, Beijing 100084, China)

  • Cong Wang

    (Guanghua School of Management, Peking University, Beijing 100871, China, School of Economics and Management, Tsinghua University, Beijing 100084, China)

  • Qiang Wei

    (Research Center for Contemporary Management, School of Economics and Management, Tsinghua University, Beijing 100084, China)

  • Zunqiang Zhang

    (Research Center for Contemporary Management, School of Economics and Management, Tsinghua University, Beijing 100084, China)

Abstract

Voting mechanisms are widely adopted for evaluating the quality and credibility of user-generated content, such as online product reviews. For the reviews that do not receive sufficient votes, techniques and models are developed to automatically assess their helpfulness levels. Existing methods serving this purpose are mostly centered on feature analysis, ignoring the information conveyed in the frequencies and patterns of user votes. Consequently, the accuracy of helpfulness measurement is limited. Inspired by related findings from prediction theories and consumer behavior research, we propose a novel approach characterized by the technique of iterative Bayesian distribution estimation, aiming to more accurately measure the helpfulness levels of reviews used for training prediction models. Using synthetic data and a real-world data set involving 1.67 million reviews and 5.18 million votes from Amazon, a simulation experiment and a two-stage data experiment show that the proposed approach outperforms existing methods on accuracy measures. Moreover, an out-of-sample user study is conducted on Amazon Mechanical Turk. The results further illustrate the predictive power of the new approach. Practically, the research contributes to e-commerce by providing an enhanced method for exploiting the value of user-generated content. Academically, we contribute to the design science literature with a novel approach that may be adapted to a wide range of research topics, such as recommender systems and social media analytics.

Suggested Citation

  • Xunhua Guo & Guoqing Chen & Cong Wang & Qiang Wei & Zunqiang Zhang, 2021. "Calibration of Voting-Based Helpfulness Measurement for Online Reviews: An Iterative Bayesian Probability Approach," INFORMS Journal on Computing, INFORMS, vol. 33(1), pages 246-261, January.
  • Handle: RePEc:inm:orijoc:v:33:y:2021:i:1:p:246-261
    DOI: 10.1287/ijoc.2019.0951
    as

    Download full text from publisher

    File URL: https://doi.org/10.1287/ijoc.2019.0951
    Download Restriction: no

    File URL: https://libkey.io/10.1287/ijoc.2019.0951?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. repec:cup:judgdm:v:5:y:2010:i:5:p:411-419 is not listed on IDEAS
    2. Alton Y.K. Chua & Snehasish Banerjee, 2015. "Understanding review helpfulness as a function of reviewer reputation, review rating, and review depth," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 66(2), pages 354-362, February.
    3. Dipayan Biswas & Guangzhi Zhao & Donald R. Lehmann, 2011. "The Impact of Sequential Data on Consumer Confidence in Relative Judgments," Journal of Consumer Research, Journal of Consumer Research Inc., vol. 37(5), pages 874-887.
    4. Yubo Chen & Jinhong Xie, 2008. "Online Consumer Review: Word-of-Mouth as a New Element of Marketing Communication Mix," Management Science, INFORMS, vol. 54(3), pages 477-491, March.
    5. Pan, Yue & Zhang, Jason Q., 2011. "Born Unequal: A Study of the Helpfulness of User-Generated Product Reviews," Journal of Retailing, Elsevier, vol. 87(4), pages 598-612.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Xiao-Jun Wang & Tao Liu & Weiguo Fan, 2023. "TGVx: Dynamic Personalized POI Deep Recommendation Model," INFORMS Journal on Computing, INFORMS, vol. 35(4), pages 786-796, July.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Guha Majumder, Madhumita & Dutta Gupta, Sangita & Paul, Justin, 2022. "Perceived usefulness of online customer reviews: A review mining approach using machine learning & exploratory data analysis," Journal of Business Research, Elsevier, vol. 150(C), pages 147-164.
    2. Hossin Md Altab & Mu Yinping & Hosain Md Sajjad & Adasa Nkrumah Kofi Frimpong & Michelle Frempomaa Frempong & Stephen Sarfo Adu-Yeboah, 2022. "Understanding Online Consumer Textual Reviews and Rating: Review Length With Moderated Multiple Regression Analysis Approach," SAGE Open, , vol. 12(2), pages 21582440221, June.
    3. Bin Guo & Shasha Zhou, 2017. "What makes population perception of review helpfulness: an information processing perspective," Electronic Commerce Research, Springer, vol. 17(4), pages 585-608, December.
    4. Xiaomo Liu & G. Alan Wang & Weiguo Fan & Zhongju Zhang, 2020. "Finding Useful Solutions in Online Knowledge Communities: A Theory-Driven Design and Multilevel Analysis," Information Systems Research, INFORMS, vol. 31(3), pages 731-752, September.
    5. Li, Yiming & Li, Gang & Tayi, Giri Kumar & Cheng, T.C.E., 2019. "Omni-channel retailing: Do offline retailers benefit from online reviews?," International Journal of Production Economics, Elsevier, vol. 218(C), pages 43-61.
    6. Wang, Feng & Liu, Xuefeng & Fang, Eric (Er), 2015. "User Reviews Variance, Critic Reviews Variance, and Product Sales: An Exploration of Customer Breadth and Depth Effects," Journal of Retailing, Elsevier, vol. 91(3), pages 372-389.
    7. Yani Wang & Jun Wang & Tang Yao, 2019. "What makes a helpful online review? A meta-analysis of review characteristics," Electronic Commerce Research, Springer, vol. 19(2), pages 257-284, June.
    8. Dominik Gutt & Jürgen Neumann & Steffen Zimmermann & Dennis Kundisch & Jianqing Chen, 2018. "Design of Review Systems - A Strategic Instrument to shape Online Review Behavior and Economic Outcomes," Working Papers Dissertations 42, Paderborn University, Faculty of Business Administration and Economics.
    9. Meek, Stephanie & Wilk, Violetta & Lambert, Claire, 2021. "A big data exploration of the informational and normative influences on the helpfulness of online restaurant reviews," Journal of Business Research, Elsevier, vol. 125(C), pages 354-367.
    10. Yi Feng & Yunqiang Yin & Dujuan Wang & Lalitha Dhamotharan & Joshua Ignatius & Ajay Kumar, 2023. "Diabetic patient review helpfulness: unpacking online drug treatment reviews by text analytics and design science approach," Annals of Operations Research, Springer, vol. 328(1), pages 387-418, September.
    11. Zunqiang Zhang & Guoqing Chen & Jin Zhang & Xunhua Guo & Qiang Wei, 2016. "Providing Consistent Opinions from Online Reviews: A Heuristic Stepwise Optimization Approach," INFORMS Journal on Computing, INFORMS, vol. 28(2), pages 236-250, May.
    12. Wanshu Niu & Liqiang Huang & Xixi Li & Jie Zhang & Mingliang Chen, 2023. "Beyond the review information: an investigation of individual- and group-based presentation forms of review information," Information Technology and Management, Springer, vol. 24(2), pages 159-176, June.
    13. Wu, Xiaoyue & Jin, Liyin & Xu, Qian, 2021. "Expertise Makes Perfect: How the Variance of a Reviewer's Historical Ratings Influences the Persuasiveness of Online Reviews," Journal of Retailing, Elsevier, vol. 97(2), pages 238-250.
    14. Muhammad Rifki Shihab & Audry Pragita Putri, 2019. "Negative online reviews of popular products: understanding the effects of review proportion and quality on consumers’ attitude and intention to buy," Electronic Commerce Research, Springer, vol. 19(1), pages 159-187, March.
    15. Jifeng Luo & Ying Rong & Huan Zheng, 2020. "Impacts of logistics information on sales: Evidence from Alibaba," Naval Research Logistics (NRL), John Wiley & Sons, vol. 67(8), pages 646-669, December.
    16. Nisar, Tahir M. & Prabhakar, Guru, 2018. "Trains and Twitter: Firm generated content, consumer relationship management and message framing," Transportation Research Part A: Policy and Practice, Elsevier, vol. 113(C), pages 318-334.
    17. Sangjae Lee & Kun Chang Lee & Joon Yeon Choeh, 2020. "Using Bayesian Network to Predict Online Review Helpfulness," Sustainability, MDPI, vol. 12(17), pages 1-17, August.
    18. Wenyi Tay & Xiuzhen Zhang & Sarvnaz Karimi, 2020. "Beyond mean rating: Probabilistic aggregation of star ratings based on helpfulness," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 71(7), pages 784-799, July.
    19. Srikanth Parameswaran & Pubali Mukherjee & Rohit Valecha, 2023. "I Like My Anonymity: An Empirical Investigation of the Effect of Multidimensional Review Text and Role Anonymity on Helpfulness of Employer Reviews," Information Systems Frontiers, Springer, vol. 25(2), pages 853-870, April.
    20. Lutz, Bernhard & Pröllochs, Nicolas & Neumann, Dirk, 2022. "Are longer reviews always more helpful? Disentangling the interplay between review length and line of argumentation," Journal of Business Research, Elsevier, vol. 144(C), pages 888-901.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:orijoc:v:33:y:2021:i:1:p:246-261. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.