IDEAS home Printed from https://ideas.repec.org/a/zag/market/v35y2023i2p223-249.html
   My bibliography  Save this article

Measuring the Effectiveness of Online Sales by Conducting A/B Testing

Author

Listed:
  • Miroslav Mandić

    (University of Zagreb, Faculty of Economics & Business)

  • Iva Gregurec

    (University of Zagreb, Faculty of Organization and Informatics)

  • UgljeÅ¡a Vujović

    (dm-drogerie markt Croatia)

Abstract

Purpose – The aim of this paper is to demonstrate the application of A/B testing for measuring the effectiveness of online sales in order to determine which changes to a website’s user interface have the greatest effect on the improvement of key performance indicators (KPIs) of online sales. Design/Methodology/Approach – A total of five A/B tests were conducted, four of which concerned the manipulation of user interface elements whereas one examined the difference in KPIs depending on the quality of the search engine used. Testing was conducted from January to July 2021 on a sample of a minimum of 7,000 visitors of the website of a company operating on the Croatian market. Findings and Implications – The conducted tests show that sometimes, as can be seen from the results of the first A/B test, the existing version of the website should be kept. However, as shown by the second, fourth, and fifth A/B test, changes to a website’s user interface can be of significant help in improving KPIs. The third A/B test highlighted the need for multiple tests of the same user interface element in order to achieve the full potential of an online business. Limitations – The general limitations of A/B testing, such as its focus on short-term goals, the lack of insights into real consumer behavior, and the use of CMS systems all affect the results of conducted research. Originality – As the analysis of existing research papers established the absence of a unified division of KPIs, their summarization and use for the purposes of the present research is the main contribution of this study.

Suggested Citation

  • Miroslav Mandić & Iva Gregurec & UgljeÅ¡a Vujović, 2023. "Measuring the Effectiveness of Online Sales by Conducting A/B Testing," Tržište/Market, Faculty of Economics and Business, University of Zagreb, vol. 35(2), pages 223-249.
    • Handle: RePEc:zag:market:v:35:y:2023:i:2:p:223-249
    as

    Download full text from publisher

    File URL: https://hrcak.srce.hr/file/452137
    Download Restriction: None
    ---><---

    References listed on IDEAS

    as
    1. Plaza, Beatriz, 2011. "Google Analytics for measuring website performance," Tourism Management, Elsevier, vol. 32(3), pages 477-481.
    2. Akrivi Krouska & Katerina Kabassi & Christos Troussas & Cleo Sgouropoulou, 2022. "Personalizing Environmental Awareness through Smartphones Using AHP and PROMETHEE II," Future Internet, MDPI, vol. 14(2), pages 1-16, February.
    3. Randall Lewis & David Reiley, 2014. "Online ads and offline sales: measuring the effect of retail advertising via a controlled experiment on Yahoo!," Quantitative Marketing and Economics (QME), Springer, vol. 12(3), pages 235-266, September.
    4. Tuck Siong Chung & Michel Wedel & Roland T. Rust, 2016. "Adaptive personalization using social networks," Journal of the Academy of Marketing Science, Springer, vol. 44(1), pages 66-87, January.
    5. Nick Hagar & Nicholas Diakopoulos, 2019. "Optimizing Content with A/B Headline Testing: Changing Newsroom Practices," Media and Communication, Cogitatio Press, vol. 7(1), pages 117-127.
    6. Vinodh Krishnaraju & Saji K Mathew & Vijayan Sugumaran, 2016. "Web personalization for user acceptance of technology: An empirical investigation of E-government services," Information Systems Frontiers, Springer, vol. 18(3), pages 579-595, June.
    7. D. Veena Parboteeah & Joseph S. Valacich & John D. Wells, 2009. "The Influence of Website Characteristics on a Consumer's Urge to Buy Impulsively," Information Systems Research, INFORMS, vol. 20(1), pages 60-78, March.
    8. Alexander Bleier & Maik Eisenbeiss, 2015. "Personalized Online Advertising Effectiveness: The Interplay of What, When, and Where," Marketing Science, INFORMS, vol. 34(5), pages 669-688, September.
    9. Avi Goldfarb & Catherine E. Tucker, 2015. "Standardization and the Effectiveness of Online Advertising," Management Science, INFORMS, vol. 61(11), pages 2707-2719, November.
    10. Mihaela I. MUNTEAN & Diana TARNAVEANU & Ana Raluca ION, 2016. "E-Commerce Performance. Shopping Cart Key Performance Indicators," Informatica Economica, Academy of Economic Studies - Bucharest, Romania, vol. 20(1), pages 62-71.
    11. Wenjie Li & Yuxiao Zhou & Shijian Luo & Yenan Dong, 2022. "Design Factors to Improve the Consistency and Sustainable User Experience of Responsive Interface Design," Sustainability, MDPI, vol. 14(15), pages 1-26, July.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Shun-Yang Lee & Julian Runge & Daniel Yoo & Yakov Bart & Anett Gyurak & J. W. Schneider, 2023. "COVID-19 Demand Shocks Revisited: Did Advertising Technology Help Mitigate Adverse Consequences for Small and Midsize Businesses?," Papers 2307.09035, arXiv.org, revised Jan 2024.
    2. Bayer, Emanuel & Srinivasan, Shuba & Riedl, Edward J. & Skiera, Bernd, 2020. "The impact of online display advertising and paid search advertising relative to offline advertising on firm performance and firm value," International Journal of Research in Marketing, Elsevier, vol. 37(4), pages 789-804.
    3. Georgios Filippou & Athanasios G. Georgiadis & Ashish Kumar Jha, 2024. "Establishing the link: Does web traffic from various marketing channels influence direct traffic source purchases?," Marketing Letters, Springer, vol. 35(1), pages 59-71, March.
    4. Garrett Johnson & Julian Runge & Eric Seufert, 2022. "Privacy-Centric Digital Advertising: Implications for Research," Customer Needs and Solutions, Springer;Institute for Sustainable Innovation and Growth (iSIG), vol. 9(1), pages 49-54, June.
    5. Hana Choi & Carl F. Mela & Santiago R. Balseiro & Adam Leary, 2020. "Online Display Advertising Markets: A Literature Review and Future Directions," Information Systems Research, INFORMS, vol. 31(2), pages 556-575, June.
    6. Jan Krämer & Daniel Schnurr & Michael Wohlfarth, 2019. "Winners, Losers, and Facebook: The Role of Social Logins in the Online Advertising Ecosystem," Management Science, INFORMS, vol. 65(4), pages 1678-1699, April.
    7. Douglas Olsen, G. & Pracejus, John W., 2020. "Customized advertising: Allowing consumers to directly tailor messages leads to better outcomes for the brand," Journal of Business Research, Elsevier, vol. 116(C), pages 245-257.
    8. Kelly D. Martin & Patrick E. Murphy, 2017. "The role of data privacy in marketing," Journal of the Academy of Marketing Science, Springer, vol. 45(2), pages 135-155, March.
    9. Martin, Kelly D. & Kim, Jisu J. & Palmatier, Robert W. & Steinhoff, Lena & Stewart, David W. & Walker, Beth A. & Wang, Yonggui & Weaven, Scott K., 2020. "Data Privacy in Retail," Journal of Retailing, Elsevier, vol. 96(4), pages 474-489.
    10. Stefan F. Bernritter & Paul E. Ketelaar & Francesca Sotgiu, 2021. "Behaviorally targeted location-based mobile marketing," Journal of the Academy of Marketing Science, Springer, vol. 49(4), pages 677-702, July.
    11. Ritu Agarwal & Michelle Dugas & Guodong (Gordon) Gao & P. K. Kannan, 2020. "Emerging technologies and analytics for a new era of value-centered marketing in healthcare," Journal of the Academy of Marketing Science, Springer, vol. 48(1), pages 9-23, January.
    12. Navdeep Sahni, 2015. "Effect of temporal spacing between advertising exposures: Evidence from online field experiments," Quantitative Marketing and Economics (QME), Springer, vol. 13(3), pages 203-247, September.
    13. Venugopal Gopalakrishna-Remani & Robert Paul Jones & Kerri M. Camp, 2019. "Levels of EMR Adoption in U.S. Hospitals: An Empirical Examination of Absorptive Capacity, Institutional Pressures, Top Management Beliefs, and Participation," Information Systems Frontiers, Springer, vol. 21(6), pages 1325-1344, December.
    14. Un-Kon Lee, 2021. "The Effect of Confirmation of Nation Brand Image in International Tourism Advertisement on Travel Intention of Foreign Tourists: The Case of Korean ITA for Chinese Tourists," SAGE Open, , vol. 11(1), pages 21582440209, January.
    15. Mahesh Balan U & Saji K. Mathew, 2021. "Personalize, Summarize or Let them Read? A Study on Online Word of Mouth Strategies and Consumer Decision Process," Information Systems Frontiers, Springer, vol. 23(3), pages 627-647, June.
    16. Mucunska Palevska, Valentina & Novkovska, Blagica, 2018. "The Participation Of Ict In Activities Of Economic Subjects In Small Economy," UTMS Journal of Economics, University of Tourism and Management, Skopje, Macedonia, vol. 9(2), pages 157-168.
    17. Randall Lewis & Dan Nguyen, 2015. "Display advertising’s competitive spillovers to consumer search," Quantitative Marketing and Economics (QME), Springer, vol. 13(2), pages 93-115, June.
    18. Lim, Joon Soo & Zhang, Jun, 2022. "Adoption of AI-driven personalization in digital news platforms: An integrative model of technology acceptance and perceived contingency," Technology in Society, Elsevier, vol. 69(C).
    19. Navdeep S. Sahni & Dan Zou & Pradeep K. Chintagunta, 2017. "Do Targeted Discount Offers Serve as Advertising? Evidence from 70 Field Experiments," Management Science, INFORMS, vol. 63(8), pages 2688-2705, August.
    20. Jean-Éric Pelet & Basma Taieb, 2022. "Context-aware optimization of mobile commerce website interfaces from the consumers’ perspective: Effects on behavioral intentions [Optimisation contextuelle des interfaces de sites Web de commerce," Post-Print hal-04138288, HAL.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:zag:market:v:35:y:2023:i:2:p:223-249. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Tanja Komarac (email available below). General contact details of provider: https://edirc.repec.org/data/fefzghr.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.