IDEAS home Printed from https://ideas.repec.org/a/eee/techno/v134y2024ics0166497224000786.html
   My bibliography  Save this article

Anomaly detection in consumer review analytics for idea generation in product innovation: Comparing machine learning and deep learning techniques

Author

Listed:
  • Cui, Xiling
  • Zhu, Zhongshan
  • Liu, Libo
  • Zhou, Qiang
  • Liu, Qiang

Abstract

With the development of big data analytics, consumers' online reviews are becoming increasingly useful for product innovation with hidden innovative ideas that can be extracted. However, these ideas may be only hidden in a small part of the massive reviews. This study aims to investigate the potential of using anomaly detection technology to identify unique reviews for more effective innovation generation. Three classical anomaly detection approaches (including both machine and deep learning) were explored, namely, isolation forest, density-based cluster analysis, and autoencoder methods. Using the consumer reviews on Dyson Vacuum cleaner from Xiaohongshu (one of the most popular social media platforms in China), we tested and compared the application of these three approaches in detecting innovation-relevant reviews. The results indicate that the two machine learning approaches, aka., density-based cluster analysis and isolation forest are too sensitive to the length of the reviews. The deep learning method, autoencoder, on the contrary, shows good stability and capability to detect the unique reviews from the whole dataset. Furthermore, the experts’ rating also confirms the effectiveness of autoencoder in identifying innovation-relevant reviews. Future studies and implications are then discussed.

Suggested Citation

  • Cui, Xiling & Zhu, Zhongshan & Liu, Libo & Zhou, Qiang & Liu, Qiang, 2024. "Anomaly detection in consumer review analytics for idea generation in product innovation: Comparing machine learning and deep learning techniques," Technovation, Elsevier, vol. 134(C).
    • Handle: RePEc:eee:techno:v:134:y:2024:i:c:s0166497224000786
    DOI: 10.1016/j.technovation.2024.103028
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0166497224000786
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.technovation.2024.103028?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Thierry Rayna & Ludmila Striukova, 2021. "Involving Consumers: The Role of Digital Technologies in Promoting ‘Prosumption’ and User Innovation," Journal of the Knowledge Economy, Springer;Portland International Center for Management of Engineering and Technology (PICMET), vol. 12(1), pages 218-237, March.
    2. Aaron K. Chatterji & Kira R. Fabrizio, 2014. "Using users: When does external knowledge enhance corporate product innovation?," Strategic Management Journal, Wiley Blackwell, vol. 35(10), pages 1427-1445, October.
    3. Marcel E. A. Weber & Mathieu C. D. P. Weggeman & Joan E. Van Aken, 2012. "Developing What Customers Really Need: Involving Customers In Innovations," International Journal of Innovation and Technology Management (IJITM), World Scientific Publishing Co. Pte. Ltd., vol. 9(03), pages 1-15.
    4. Christian Janiesch & Patrick Zschech & Kai Heinrich, 2021. "Machine learning and deep learning," Electronic Markets, Springer;IIM University of St. Gallen, vol. 31(3), pages 685-695, September.
    5. Hui Zhang & Huguang Rao & Junzheng Feng, 2018. "Product innovation based on online review data mining: a case study of Huawei phones," Electronic Commerce Research, Springer, vol. 18(1), pages 3-22, March.
    6. Saura, Jose Ramon & Palacios-Marqués, Daniel & Ribeiro-Soriano, Domingo, 2023. "Exploring the boundaries of open innovation: Evidence from social media mining," Technovation, Elsevier, vol. 119(C).
    7. Hoyer, Wayne D. & Kroschke, Mirja & Schmitt, Bernd & Kraume, Karsten & Shankar, Venkatesh, 2020. "Transforming the Customer Experience Through New Technologies," Journal of Interactive Marketing, Elsevier, vol. 51(C), pages 57-71.
    8. Mehran Azimi & Anup Agrawal, 2021. "Is Positive Sentiment in Corporate Annual Reports Informative? Evidence from Deep Learning [Cash holdings and credit risk]," The Review of Asset Pricing Studies, Society for Financial Studies, vol. 11(4), pages 762-805.
    9. Kimmy Wa Chan & Stella Yiyan Li & Jian Ni & John JianJun Zhu, 2021. "What Feedback Matters? The Role of Experience in Motivating Crowdsourcing Innovation," Production and Operations Management, Production and Operations Management Society, vol. 30(1), pages 103-126, January.
    10. Urbinati, Andrea & Bogers, Marcel & Chiesa, Vittorio & Frattini, Federico, 2019. "Creating and capturing value from Big Data: A multiple-case study analysis of provider companies," Technovation, Elsevier, vol. 84, pages 21-36.
    11. Iqbal, Rahat & Doctor, Faiyaz & More, Brian & Mahmud, Shahid & Yousuf, Usman, 2020. "Big data analytics: Computational intelligence techniques and application areas," Technological Forecasting and Social Change, Elsevier, vol. 153(C).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jen-Yu Lee & Tien-Thinh Nguyen & Hong-Giang Nguyen & Jen-Yao Lee, 2022. "Towards Predictive Crude Oil Purchase: A Case Study in the USA and Europe," Energies, MDPI, vol. 15(11), pages 1-15, May.
    2. Mostafa Bigdeli & Mahsa Akbari, 2024. "Machine-learning-based Classification of Customers’ Behavioural Model in Instagram," Paradigm, , vol. 28(2), pages 223-240, December.
    3. Caroline Danièle Mothe & Thuc Uyen Nguyen-Thi, 2017. "Persistent openness and environmental innovation: An empirical analysis of French manufacturing firms," Post-Print hal-01609129, HAL.
    4. Eduard Hartwich & Alexander Rieger & Johannes Sedlmeir & Dominik Jurek & Gilbert Fridgen, 2023. "Machine economies," Electronic Markets, Springer;IIM University of St. Gallen, vol. 33(1), pages 1-13, December.
    5. Bertschek, Irene & Kesler, Reinhold, 2022. "Let the user speak: Is feedback on Facebook a source of firms’ innovation?," Information Economics and Policy, Elsevier, vol. 60(C).
    6. Chen, Xi & Zhang, Stephen X., 2024. "Too much of two good things: The curvilinear effects of self-efficacy and market validation in new ventures," Journal of Business Research, Elsevier, vol. 183(C).
    7. Najla Alharbi & Bashayer Alkalifah & Ghaida Alqarawi & Murad A. Rassam, 2024. "Countering Social Media Cybercrime Using Deep Learning: Instagram Fake Accounts Detection," Future Internet, MDPI, vol. 16(10), pages 1-22, October.
    8. Rui Ma & Jia Wang & Wei Zhao & Hongjie Guo & Dongnan Dai & Yuliang Yun & Li Li & Fengqi Hao & Jinqiang Bai & Dexin Ma, 2022. "Identification of Maize Seed Varieties Using MobileNetV2 with Improved Attention Mechanism CBAM," Agriculture, MDPI, vol. 13(1), pages 1-16, December.
    9. Maniyassouwe Amana & Pingfeng Liu & Mona Alariqi, 2022. "Value Creation and Capture with Big Data in Smart Phones Companies," Sustainability, MDPI, vol. 14(23), pages 1-22, November.
    10. Carlos J. Serrano & Rosemarie Ziedonis, 2018. "How Redeployable are Patent Assets? Evidence from Failed Startups," NBER Working Papers 24526, National Bureau of Economic Research, Inc.
    11. Yuk, Hyeyeon & Garrett, Tony C., 2023. "Does customer participation moderate the effects of innovation on cost-based financial performance? An examination of different forms of customer participation," Journal of Business Research, Elsevier, vol. 156(C).
    12. Brewis, Claire & Dibb, Sally & Meadows, Maureen, 2023. "Leveraging big data for strategic marketing: A dynamic capabilities model for incumbent firms," Technological Forecasting and Social Change, Elsevier, vol. 190(C).
    13. Zon-Yau Lee & Mei-Tai Chu & Shiuann-Shuoh Chen & Chih-Hung Tsai, 2018. "Identifying Comprehensive Key Criteria of Sustainable Development for Traditional Manufacturing in Taiwan," Sustainability, MDPI, vol. 10(9), pages 1-19, September.
    14. Suyun Chen & Yu Ji, 2022. "Do Corporate Social Responsibility Categories Distinctly Influence Innovation? A Resource-Based Theory Perspective," Sustainability, MDPI, vol. 14(6), pages 1-25, March.
    15. Yuanyuan Guo & Yanqing Wang & Chaoyou Wang, 2019. "Exploring the Salient Attributes of Short-Term Rental Experience: An Analysis of Online Reviews from Chinese Guests," Sustainability, MDPI, vol. 11(16), pages 1-19, August.
    16. Dylan Norbert Gono & Herlina Napitupulu & Firdaniza, 2023. "Silver Price Forecasting Using Extreme Gradient Boosting (XGBoost) Method," Mathematics, MDPI, vol. 11(18), pages 1-15, September.
    17. De Bruyn, Arnaud & Viswanathan, Vijay & Beh, Yean Shan & Brock, Jürgen Kai-Uwe & von Wangenheim, Florian, 2020. "Artificial Intelligence and Marketing: Pitfalls and Opportunities," Journal of Interactive Marketing, Elsevier, vol. 51(C), pages 91-105.
    18. Shaheer, Noman & Kim, Kijong & Li, Sali, 2022. "Internationalization of Digital Innovations: A Rapidly Evolving Research Stream," Journal of International Management, Elsevier, vol. 28(4).
    19. Cheng Yang & Fuhao Sun & Yujie Zou & Zhipeng Lv & Liang Xue & Chao Jiang & Shuangyu Liu & Bochao Zhao & Haoyang Cui, 2024. "A Survey of Photovoltaic Panel Overlay and Fault Detection Methods," Energies, MDPI, vol. 17(4), pages 1-37, February.
    20. Kamoonpuri, Sana Zehra & Sengar, Anita, 2023. "Hi, May AI help you? An analysis of the barriers impeding the implementation and use of artificial intelligence-enabled virtual assistants in retail," Journal of Retailing and Consumer Services, Elsevier, vol. 72(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:techno:v:134:y:2024:i:c:s0166497224000786. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.sciencedirect.com/science/journal/01664972 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.