IDEAS home Printed from https://ideas.repec.org/a/inm/orisre/v31y2020i3p675-691.html
   My bibliography  Save this article

Eye-Tracking-Based Classification of Information Search Behavior Using Machine Learning: Evidence from Experiments in Physical Shops and Virtual Reality Shopping Environments

Author

Listed:
  • Jella Pfeiffer

    (Justus Liebig University Giessen, 35394 Gießen, Germany)

  • Thies Pfeiffer

    (University of Applied Sciences Emden/Leer, 26723 Emden, Germany)

  • Martin Meißner

    (Zeppelin University, 88045 Friedrichshafen, Germany)

  • Elisa Weiß

    (Karlsruhe Institute of Technology, 76133 Karlsruhe, Germany)

Abstract

Classifying information search behavior helps tailor recommender systems to individual customers’ shopping motives. But how can we identify these motives without requiring users to exert too much effort? Our research goal is to demonstrate that eye tracking can be used at the point of sale to do so. We focus on two frequently investigated shopping motives: goal-directed and exploratory search. To train and test a prediction model, we conducted two eye-tracking experiments in front of supermarket shelves. The first experiment was carried out in immersive virtual reality; the second, in physical reality—in other words, as a field study in a real supermarket. We conducted a virtual reality study, because recently launched virtual shopping environments suggest that there is great interest in using this technology as a retail channel. Our empirical results show that support vector machines allow the correct classification of search motives with 80% accuracy in virtual reality and 85% accuracy in physical reality. Our findings also imply that eye movements allow shopping motives to be identified relatively early in the search process: our models achieve 70% prediction accuracy after only 15 seconds in virtual reality and 75% in physical reality. Applying an ensemble method increases the prediction accuracy substantially, to about 90%. Consequently, the approach that we propose could be used for the satisfiable classification of consumers in practice. Furthermore, both environments’ best predictor variables overlap substantially. This finding provides evidence that in virtual reality, information search behavior might be similar to the one used in physical reality. Finally, we also discuss managerial implications for retailers and companies that are planning to use our technology to personalize a consumer assistance system.

Suggested Citation

  • Jella Pfeiffer & Thies Pfeiffer & Martin Meißner & Elisa Weiß, 2020. "Eye-Tracking-Based Classification of Information Search Behavior Using Machine Learning: Evidence from Experiments in Physical Shops and Virtual Reality Shopping Environments," Information Systems Research, INFORMS, vol. 31(3), pages 675-691, September.
  • Handle: RePEc:inm:orisre:v:31:y:2020:i:3:p:675-691
    DOI: 10.1287/isre.2019.0907
    as

    Download full text from publisher

    File URL: https://doi.org/10.1287/isre.2019.0907
    Download Restriction: no

    File URL: https://libkey.io/10.1287/isre.2019.0907?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Cowan, Kirsten & Ketron, Seth, 2019. "A dual model of product involvement for effective virtual reality: The roles of imagination, co-creation, telepresence, and interactivity," Journal of Business Research, Elsevier, vol. 100(C), pages 483-492.
    2. Arnaud De Bruyn & John C. Liechty & Eelko K. R. E. Huizingh & Gary L. Lilien, 2008. "Offering Online Recommendations with Minimum Customer Input Through Conjoint-Based Decision Aids," Marketing Science, INFORMS, vol. 27(3), pages 443-460, 05-06.
    3. repec:cup:judgdm:v:12:y:2017:i:6:p:596-609 is not listed on IDEAS
    4. Francesca Bonetti & Gary Warnaby & Lee Quinn, 2018. "Augmented Reality and Virtual Reality in Physical and Online Retailing: A Review, Synthesis and Research Agenda," Progress in IS, in: Timothy Jung & M. Claudia tom Dieck (ed.), Augmented Reality and Virtual Reality, pages 119-132, Springer.
    5. Bigné, Enrique & Llinares, Carmen & Torrecilla, Carmen, 2016. "Elapsed time on first buying triggers brand choices within a category: A virtual reality-based study," Journal of Business Research, Elsevier, vol. 69(4), pages 1423-1427.
    6. Inman, J. Jeffrey & Nikolova, Hristina, 2017. "Shopper-Facing Retail Technology: A Retailer Adoption Decision Framework Incorporating Shopper Attitudes and Privacy Concerns," Journal of Retailing, Elsevier, vol. 93(1), pages 7-28.
    7. Sam K. Hui & Peter S. Fader & Eric T. Bradlow, 2009. "Path Data in Marketing: An Integrative Framework and Prospectus for Model Building," Marketing Science, INFORMS, vol. 28(2), pages 320-335, 03-04.
    8. Janiszewski, Chris, 1998. "The Influence of Display Characteristics on Visual Exploratory Search Behavior," Journal of Consumer Research, Journal of Consumer Research Inc., vol. 25(3), pages 290-301, December.
    9. Russo, J Edward & Leclerc, France, 1994. "An Eye-Fixation Analysis of Choice Processes for Consumer Nondurables," Journal of Consumer Research, Journal of Consumer Research Inc., vol. 21(2), pages 274-290, September.
    10. A. S. C. Ehrenberg & J. A. Bound, 1993. "Predictability and Prediction," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 156(2), pages 167-194, March.
    11. Boyd, D. Eric & Koles, Bernadett, 2019. "Virtual reality and its impact on B2B marketing: A value-in-use perspective," Journal of Business Research, Elsevier, vol. 100(C), pages 590-598.
    12. D. Eric Boyd & Bernadett Koles, 2019. "Virtual reality and its impact on B2B marketing: A value-in-use perspective," Post-Print hal-02160880, HAL.
    13. Savannah Wei Shi & Michel Wedel & F. G. M. (Rik) Pieters, 2013. "Information Acquisition During Online Decision Making: A Model-Based Exploration Using Eye-Tracking Data," Management Science, INFORMS, vol. 59(5), pages 1009-1026, May.
    14. Kahn, Barbara E., 2017. "Using Visual Design to Improve Customer Perceptions of Online Assortments," Journal of Retailing, Elsevier, vol. 93(1), pages 29-42.
    15. Scholz, Michael & Pfeiffer, Jella & Rothlauf, Franz, 2017. "Using PageRank for non-personalized default rankings in dynamic markets," European Journal of Operational Research, Elsevier, vol. 260(1), pages 388-401.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Pascal Oliver Heßler & Jella Pfeiffer & Sebastian Hafenbrädl, 2022. "When Self-Humanization Leads to Algorithm Aversion," Business & Information Systems Engineering: The International Journal of WIRTSCHAFTSINFORMATIK, Springer;Gesellschaft für Informatik e.V. (GI), vol. 64(3), pages 275-292, June.
    2. Meißner, Martin & Pfeiffer, Jella & Peukert, Christian & Dietrich, Holger & Pfeiffer, Thies, 2020. "How virtual reality affects consumer choice," Journal of Business Research, Elsevier, vol. 117(C), pages 219-231.
    3. Fu-Hsiang Chen & Ming-Fu Hsu & Kuang-Hua Hu, 2022. "Enterprise’s internal control for knowledge discovery in a big data environment by an integrated hybrid model," Information Technology and Management, Springer, vol. 23(3), pages 213-231, September.
    4. Ransome Epie Bawack & Samuel Fosso Wamba & Kevin Daniel André Carillo & Shahriar Akter, 2022. "Artificial intelligence in E-Commerce: a bibliometric study and literature review," Electronic Markets, Springer;IIM University of St. Gallen, vol. 32(1), pages 297-338, March.
    5. Wolfgang Ketter & Karsten Schroer & Konstantina Valogianni, 2023. "Information Systems Research for Smart Sustainable Mobility: A Framework and Call for Action," Information Systems Research, INFORMS, vol. 34(3), pages 1045-1065, September.
    6. Stefan Morana & Jella Pfeiffer & Marc T. P. Adam, 2020. "User Assistance for Intelligent Systems," Business & Information Systems Engineering: The International Journal of WIRTSCHAFTSINFORMATIK, Springer;Gesellschaft für Informatik e.V. (GI), vol. 62(3), pages 189-192, June.
    7. Nannan Xi & Juan Chen & Filipe Gama & Marc Riar & Juho Hamari, 2023. "The challenges of entering the metaverse: An experiment on the effect of extended reality on workload," Information Systems Frontiers, Springer, vol. 25(2), pages 659-680, April.
    8. Roman Beck & Jens Dibbern & Martin Wiener, 2022. "A Multi-Perspective Framework for Research on (Sustainable) Autonomous Systems," Business & Information Systems Engineering: The International Journal of WIRTSCHAFTSINFORMATIK, Springer;Gesellschaft für Informatik e.V. (GI), vol. 64(3), pages 265-273, June.
    9. Abdelmaged, Mohamed Adel Mahmoud, 2021. "Implementation of Virtual Reality in Healthcare, Entertainment, Tourism, Education, and Retail Sectors," MPRA Paper 110491, University Library of Munich, Germany.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Wedel, Michel & Bigné, Enrique & Zhang, Jie, 2020. "Virtual and augmented reality: Advancing research in consumer marketing," International Journal of Research in Marketing, Elsevier, vol. 37(3), pages 443-465.
    2. Pizzi, Gabriele & Vannucci, Virginia & Aiello, Gaetano, 2020. "Branding in the time of virtual reality: Are virtual store brand perceptions real?," Journal of Business Research, Elsevier, vol. 119(C), pages 502-510.
    3. Loureiro, Sandra Maria Correia & Guerreiro, João & Japutra, Arnold, 2021. "How escapism leads to behavioral intention in a virtual reality store with background music?," Journal of Business Research, Elsevier, vol. 134(C), pages 288-300.
    4. de Regt, Anouk & Barnes, Stuart J. & Plangger, Kirk, 2020. "The virtual reality value chain," Business Horizons, Elsevier, vol. 63(6), pages 737-748.
    5. Zuschke, Nick, 2020. "An analysis of process-tracing research on consumer decision-making," Journal of Business Research, Elsevier, vol. 111(C), pages 305-320.
    6. Kautish, Pradeep & Paul, Justin & Sharma, Rajesh, 2021. "The effect of assortment and fulfillment on shopping assistance and efficiency: An e-tail servicescape perspective," Journal of Retailing and Consumer Services, Elsevier, vol. 59(C).
    7. de Regt, Anouk & Plangger, Kirk & Barnes, Stuart J., 2021. "Virtual reality marketing and customer advocacy: Transforming experiences from story-telling to story-doing," Journal of Business Research, Elsevier, vol. 136(C), pages 513-522.
    8. Roose, Gudrun & Vermeir, Iris, 2023. "Putting spatial product presentation cues on the map: Review and research directions," Journal of Business Research, Elsevier, vol. 155(PA).
    9. Milica Mormann & Tom Griffiths & Chris Janiszewski & J. Edward Russo & Anocha Aribarg & Nathaniel J. S. Ashby & Rajesh Bagchi & Sudeep Bhatia & Aleksandra Kovacheva & Martin Meissner & Kellen J. Mrkva, 2020. "Time to pay attention to attention: using attention-based process traces to better understand consumer decision-making," Marketing Letters, Springer, vol. 31(4), pages 381-392, December.
    10. Martinovici, A., 2019. "Revealing attention - how eye movements predict brand choice and moment of choice," Other publications TiSEM 7dca38a5-9f78-4aee-bd81-c, Tilburg University, School of Economics and Management.
    11. Meißner, Martin & Oppewal, Harmen & Huber, Joel, 2020. "Surprising adaptivity to set size changes in multi-attribute repeated choice tasks," Journal of Business Research, Elsevier, vol. 111(C), pages 163-175.
    12. Raza, Ali & Wasim, Manahil & Ishaq, Muhammad Ishtiaq, 2024. "Virtual reality-based product displays to inspire consumers’ purchase intentions: An experimental study," Journal of Business Research, Elsevier, vol. 175(C).
    13. Huddleston, Patricia T. & Behe, Bridget K. & Driesener, Carl & Minahan, S., 2018. "Inside-outside: Using eye-tracking to investigate search-choice processes in the retail environment," Journal of Retailing and Consumer Services, Elsevier, vol. 43(C), pages 85-93.
    14. Drummond, Conor & McGrath, Helen & O'Toole, Thomas, 2023. "Beyond the platform: Social media as a multi-faceted resource in value creation for entrepreneurial firms in a collaborative network," Journal of Business Research, Elsevier, vol. 158(C).
    15. Xi, Nannan & Hamari, Juho, 2021. "Shopping in virtual reality: A literature review and future agenda," Journal of Business Research, Elsevier, vol. 134(C), pages 37-58.
    16. Epstein, Leonardo D. & Inostroza-Quezada, Ignacio E. & Goodstein, Ronald C. & Choi, S. Chan, 2021. "Dynamic effects of store promotions on purchase conversion: Expanding technology applications with innovative analytics," Journal of Business Research, Elsevier, vol. 128(C), pages 279-289.
    17. Kostyk, Alena & Sheng, Jie, 2023. "VR in customer-centered marketing: Purpose-driven design," Business Horizons, Elsevier, vol. 66(2), pages 225-236.
    18. Rahman, Syed Mahmudur & Carlson, Jamie & Gudergan, Siegfried P. & Wetzels, Martin & Grewal, Dhruv, 2022. "Perceived Omnichannel Customer Experience (OCX): Concept, measurement, and impact," Journal of Retailing, Elsevier, vol. 98(4), pages 611-632.
    19. Zuschke, Nick, 2020. "The impact of task complexity and task motivation on in-store marketing effectiveness: An eye tracking analysis," Journal of Business Research, Elsevier, vol. 116(C), pages 337-350.
    20. Mohamed Slim Ben Mimoun & Caroline Lancelot Miltgen & Boulbeba Slama, 2022. "Is the shopper always the king/queen? Study of omnichannel retail technology use and shopping orientations," Post-Print hal-03519230, HAL.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:orisre:v:31:y:2020:i:3:p:675-691. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.