IDEAS home Printed from https://ideas.repec.org/a/eee/techno/v134y2024ics0166497224000786.html
   My bibliography  Save this article

Anomaly detection in consumer review analytics for idea generation in product innovation: Comparing machine learning and deep learning techniques

Author

Listed:
  • Cui, Xiling
  • Zhu, Zhongshan
  • Liu, Libo
  • Zhou, Qiang
  • Liu, Qiang

Abstract

With the development of big data analytics, consumers' online reviews are becoming increasingly useful for product innovation with hidden innovative ideas that can be extracted. However, these ideas may be only hidden in a small part of the massive reviews. This study aims to investigate the potential of using anomaly detection technology to identify unique reviews for more effective innovation generation. Three classical anomaly detection approaches (including both machine and deep learning) were explored, namely, isolation forest, density-based cluster analysis, and autoencoder methods. Using the consumer reviews on Dyson Vacuum cleaner from Xiaohongshu (one of the most popular social media platforms in China), we tested and compared the application of these three approaches in detecting innovation-relevant reviews. The results indicate that the two machine learning approaches, aka., density-based cluster analysis and isolation forest are too sensitive to the length of the reviews. The deep learning method, autoencoder, on the contrary, shows good stability and capability to detect the unique reviews from the whole dataset. Furthermore, the experts’ rating also confirms the effectiveness of autoencoder in identifying innovation-relevant reviews. Future studies and implications are then discussed.

Suggested Citation

  • Cui, Xiling & Zhu, Zhongshan & Liu, Libo & Zhou, Qiang & Liu, Qiang, 2024. "Anomaly detection in consumer review analytics for idea generation in product innovation: Comparing machine learning and deep learning techniques," Technovation, Elsevier, vol. 134(C).
    • Handle: RePEc:eee:techno:v:134:y:2024:i:c:s0166497224000786
    DOI: 10.1016/j.technovation.2024.103028
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0166497224000786
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.technovation.2024.103028?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Thierry Rayna & Ludmila Striukova, 2021. "Involving Consumers: The Role of Digital Technologies in Promoting ‘Prosumption’ and User Innovation," Journal of the Knowledge Economy, Springer;Portland International Center for Management of Engineering and Technology (PICMET), vol. 12(1), pages 218-237, March.
    2. Aaron K. Chatterji & Kira R. Fabrizio, 2014. "Using users: When does external knowledge enhance corporate product innovation?," Strategic Management Journal, Wiley Blackwell, vol. 35(10), pages 1427-1445, October.
    3. Marcel E. A. Weber & Mathieu C. D. P. Weggeman & Joan E. Van Aken, 2012. "Developing What Customers Really Need: Involving Customers In Innovations," International Journal of Innovation and Technology Management (IJITM), World Scientific Publishing Co. Pte. Ltd., vol. 9(03), pages 1-15.
    4. Christian Janiesch & Patrick Zschech & Kai Heinrich, 2021. "Machine learning and deep learning," Electronic Markets, Springer;IIM University of St. Gallen, vol. 31(3), pages 685-695, September.
    5. Hui Zhang & Huguang Rao & Junzheng Feng, 2018. "Product innovation based on online review data mining: a case study of Huawei phones," Electronic Commerce Research, Springer, vol. 18(1), pages 3-22, March.
    6. Saura, Jose Ramon & Palacios-Marqués, Daniel & Ribeiro-Soriano, Domingo, 2023. "Exploring the boundaries of open innovation: Evidence from social media mining," Technovation, Elsevier, vol. 119(C).
    7. Hoyer, Wayne D. & Kroschke, Mirja & Schmitt, Bernd & Kraume, Karsten & Shankar, Venkatesh, 2020. "Transforming the Customer Experience Through New Technologies," Journal of Interactive Marketing, Elsevier, vol. 51(C), pages 57-71.
    8. Mehran Azimi & Anup Agrawal, 2021. "Is Positive Sentiment in Corporate Annual Reports Informative? Evidence from Deep Learning [Cash holdings and credit risk]," The Review of Asset Pricing Studies, Society for Financial Studies, vol. 11(4), pages 762-805.
    9. Kimmy Wa Chan & Stella Yiyan Li & Jian Ni & John JianJun Zhu, 2021. "What Feedback Matters? The Role of Experience in Motivating Crowdsourcing Innovation," Production and Operations Management, Production and Operations Management Society, vol. 30(1), pages 103-126, January.
    10. Urbinati, Andrea & Bogers, Marcel & Chiesa, Vittorio & Frattini, Federico, 2019. "Creating and capturing value from Big Data: A multiple-case study analysis of provider companies," Technovation, Elsevier, vol. 84, pages 21-36.
    11. Iqbal, Rahat & Doctor, Faiyaz & More, Brian & Mahmud, Shahid & Yousuf, Usman, 2020. "Big data analytics: Computational intelligence techniques and application areas," Technological Forecasting and Social Change, Elsevier, vol. 153(C).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jen-Yu Lee & Tien-Thinh Nguyen & Hong-Giang Nguyen & Jen-Yao Lee, 2022. "Towards Predictive Crude Oil Purchase: A Case Study in the USA and Europe," Energies, MDPI, vol. 15(11), pages 1-15, May.
    2. Caroline Danièle Mothe & Thuc Uyen Nguyen-Thi, 2017. "Persistent openness and environmental innovation: An empirical analysis of French manufacturing firms," Post-Print hal-01609129, HAL.
    3. Eduard Hartwich & Alexander Rieger & Johannes Sedlmeir & Dominik Jurek & Gilbert Fridgen, 2023. "Machine economies," Electronic Markets, Springer;IIM University of St. Gallen, vol. 33(1), pages 1-13, December.
    4. Rainer Alt, 2021. "Electronic Markets on robotics," Electronic Markets, Springer;IIM University of St. Gallen, vol. 31(3), pages 465-471, September.
    5. Bertschek, Irene & Kesler, Reinhold, 2022. "Let the user speak: Is feedback on Facebook a source of firms’ innovation?," Information Economics and Policy, Elsevier, vol. 60(C).
    6. Anke Joubert & Matthias Murawski & Markus Bick, 2023. "Measuring the Big Data Readiness of Developing Countries – Index Development and its Application to Africa," Information Systems Frontiers, Springer, vol. 25(1), pages 327-350, February.
    7. Chen, Xi & Zhang, Stephen X., 2024. "Too much of two good things: The curvilinear effects of self-efficacy and market validation in new ventures," Journal of Business Research, Elsevier, vol. 183(C).
    8. Michele Gorgoglione & Achille Claudio Garavelli & Umberto Panniello & Angelo Natalicchio, 2023. "Information Retrieval Technologies and Big Data Analytics to Analyze Product Innovation in the Music Industry," Sustainability, MDPI, vol. 15(1), pages 1-16, January.
    9. Agnieszka Izabela Baruk, 2021. "Relationships between Final Purchasers and Offerors in the Context of Their Perception by Final Purchasers," Energies, MDPI, vol. 14(11), pages 1-15, June.
    10. Najla Alharbi & Bashayer Alkalifah & Ghaida Alqarawi & Murad A. Rassam, 2024. "Countering Social Media Cybercrime Using Deep Learning: Instagram Fake Accounts Detection," Future Internet, MDPI, vol. 16(10), pages 1-22, October.
    11. Mendez-Picazo, María-Teresa & Galindo-Martin, Miguel-Angel & Perez-Pujol, Rafael-Sergio, 2024. "Direct and indirect effects of digital transformation on sustainable development in pre- and post-pandemic periods," Technological Forecasting and Social Change, Elsevier, vol. 200(C).
    12. Zhucui Jing & Ying Zheng & Hongli Guo, 2023. "A Study of the Impact of Digital Competence and Organizational Agility on Green Innovation Performance of Manufacturing Firms—The Moderating Effect Based on Knowledge Inertia," Administrative Sciences, MDPI, vol. 13(12), pages 1-17, December.
    13. Chen, Pengyu & Chu, Zhongzhu & Zhao, Miao, 2024. "The Road to corporate sustainability: The importance of artificial intelligence," Technology in Society, Elsevier, vol. 76(C).
    14. Abdulwahhab, Ali H. & Abdulaal, Alaa Hussein & Thary Al-Ghrairi, Assad H. & Mohammed, Ali Abdulwahhab & Valizadeh, Morteza, 2024. "Detection of epileptic seizure using EEG signals analysis based on deep learning techniques," Chaos, Solitons & Fractals, Elsevier, vol. 181(C).
    15. Abhirup Khanna & Bhawna Yadav Lamba & Sapna Jain & Vadim Bolshev & Dmitry Budnikov & Vladimir Panchenko & Alexandr Smirnov, 2023. "Biodiesel Production from Jatropha: A Computational Approach by Means of Artificial Intelligence and Genetic Algorithm," Sustainability, MDPI, vol. 15(12), pages 1-33, June.
    16. Rui Ma & Jia Wang & Wei Zhao & Hongjie Guo & Dongnan Dai & Yuliang Yun & Li Li & Fengqi Hao & Jinqiang Bai & Dexin Ma, 2022. "Identification of Maize Seed Varieties Using MobileNetV2 with Improved Attention Mechanism CBAM," Agriculture, MDPI, vol. 13(1), pages 1-16, December.
    17. Maniyassouwe Amana & Pingfeng Liu & Mona Alariqi, 2022. "Value Creation and Capture with Big Data in Smart Phones Companies," Sustainability, MDPI, vol. 14(23), pages 1-22, November.
    18. Carlos J. Serrano & Rosemarie Ziedonis, 2018. "How Redeployable are Patent Assets? Evidence from Failed Startups," NBER Working Papers 24526, National Bureau of Economic Research, Inc.
    19. Li, Lei & Lin, Jiabao & Ouyang, Ye & Luo, Xin (Robert), 2022. "Evaluating the impact of big data analytics usage on the decision-making quality of organizations," Technological Forecasting and Social Change, Elsevier, vol. 175(C).
    20. Yuk, Hyeyeon & Garrett, Tony C., 2023. "Does customer participation moderate the effects of innovation on cost-based financial performance? An examination of different forms of customer participation," Journal of Business Research, Elsevier, vol. 156(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:techno:v:134:y:2024:i:c:s0166497224000786. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.sciencedirect.com/science/journal/01664972 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.