IDEAS home Printed from https://ideas.repec.org/a/eee/jbrese/v118y2020icp189-198.html
   My bibliography  Save this article

Running field experiments using Facebook split test

Author

Listed:
  • Orazi, Davide C.
  • Johnston, Allen C.

Abstract

Business researchers use experimental methods extensively due to their high internal validity. However, controlled laboratory and crowdsourcing settings often introduce issues of artificiality, data contamination, and low managerial relevance of the dependent variables. Field experiments can overcome these issues but are traditionally time- and resource-consuming. This primer presents an alternative experimental setting to conduct online field experiments in a time- and cost-effective way. It does so by introducing the Facebook A/B split test functionality, which allows for random assignment of manipulated variables embedded in ecologically-valid stimuli. We compare and contrast this method against laboratory settings and Amazon Mechanical Turk in terms of design flexibility, managerial relevance, data quality control, and sample representativeness. We then provide an empirical demonstration of how to set up, pre-test, run, and analyze FBST experiments.

Suggested Citation

  • Orazi, Davide C. & Johnston, Allen C., 2020. "Running field experiments using Facebook split test," Journal of Business Research, Elsevier, vol. 118(C), pages 189-198.
    • Handle: RePEc:eee:jbrese:v:118:y:2020:i:c:p:189-198
    DOI: 10.1016/j.jbusres.2020.06.053
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S014829632030429X
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.jbusres.2020.06.053?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. J Jeffrey Inman & Margaret C Campbell & Amna Kirmani & Linda L Price, 2018. "Our Vision for the Journal of Consumer Research: It’s All about the Consumer," Journal of Consumer Research, Journal of Consumer Research Inc., vol. 44(5), pages 955-959.
    2. repec:cup:judgdm:v:5:y:2010:i:5:p:411-419 is not listed on IDEAS
    3. repec:nas:journl:v:115:y:2018:p:e5254-e5255 is not listed on IDEAS
    4. Orazi, Davide C. & Pizzetti, Marta, 2015. "Revisiting fear appeals: A structural re-inquiry of the protection motivation model," International Journal of Research in Marketing, Elsevier, vol. 32(2), pages 223-225.
    5. Smith, Scott M. & Roster, Catherine A. & Golden, Linda L. & Albaum, Gerald S., 2016. "A multi-group analysis of online survey respondent data quality: Comparing a regular USA consumer panel to MTurk samples," Journal of Business Research, Elsevier, vol. 69(8), pages 3139-3148.
    6. Paolacci, Gabriele & Chandler, Jesse & Ipeirotis, Panagiotis G., 2010. "Running experiments on Amazon Mechanical Turk," Judgment and Decision Making, Cambridge University Press, vol. 5(5), pages 411-419, August.
    7. Arthur Schram, 2005. "Artificiality: The tension between internal and external validity in economic experiments," Journal of Economic Methodology, Taylor & Francis Journals, vol. 12(2), pages 225-237.
    8. Orazi, Davide C. & Lei, Jing & Bove, Liliana L., 2015. "The nature and framing of gambling consequences in advertising," Journal of Business Research, Elsevier, vol. 68(10), pages 2049-2056.
    9. Kirsten Passyn & Mita Sujan, 2006. "Self-Accountability Emotions and Fear Appeals: Motivating Behavior," Journal of Consumer Research, Journal of Consumer Research Inc., vol. 32(4), pages 583-589, March.
    10. Daly, Timothy M. & Nataraajan, Rajan, 2015. "Swapping bricks for clicks: Crowdsourcing longitudinal data on Amazon Turk," Journal of Business Research, Elsevier, vol. 68(12), pages 2603-2609.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Wallach, Karen Anne & Popovich, Deidre, 2023. "When Big Is Less than Small: Why dominant brands lack authenticity in their sustainability initiatives," Journal of Business Research, Elsevier, vol. 158(C).
    2. Lukas Maier & Christian V. Baccarella & Jörn H. Block & Timm F. Wagner & Kai-Ingo Voigt, 2023. "The Legitimization Effect of Crowdfunding Success: A Consumer Perspective," Entrepreneurship Theory and Practice, , vol. 47(4), pages 1389-1420, July.
    3. Chawla, Yash & Chodak, Grzegorz, 2021. "Social media marketing for businesses: Organic promotions of web-links on Facebook," Journal of Business Research, Elsevier, vol. 135(C), pages 49-65.
    4. Manuela Fritz & Michael Grimm & Ingmar Weber & Elad Yom-Tov & Benedictus Praditya, 2022. "Uncover your risk! Using Facebook to increase personal risk awareness and screening of type 2 diabetes in Indonesia," Working Papers 221, Bavarian Graduate Program in Economics (BGPE).
    5. Krallman, Alexandra & Barnes, Donald C. & Lastner, Matthew M. & Collier, Joel E., 2023. "You can’t touch this: Driving purchase justification for hedonic online purchases," Journal of Business Research, Elsevier, vol. 155(PB).
    6. Zhang, Jin & Xu, Xiaobing & Keh, Hean Tat, 2022. "I implement, they deliberate: The matching effects of point of view and mindset on consumer attitudes," Journal of Business Research, Elsevier, vol. 139(C), pages 397-410.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Efrat Dressler & Yevgeny Mugerman, 2023. "Doing the Right Thing? The Voting Power Effect and Institutional Shareholder Voting," Journal of Business Ethics, Springer, vol. 183(4), pages 1089-1112, April.
    2. Nicolas Jacquemet & Alexander G James & Stéphane Luchini & James J Murphy & Jason F Shogren, 2021. "Do truth-telling oaths improve honesty in crowd-working?," PLOS ONE, Public Library of Science, vol. 16(1), pages 1-18, January.
    3. Sharifi, Seyed Shahin & Aghazadeh, Hashem, 2016. "Discount reference moderates customers' reactions to discount frames after online service failure," Journal of Business Research, Elsevier, vol. 69(10), pages 4074-4080.
    4. Ina Garnefeld & Andreas Eggert & Markus Husemann-Kopetzky & Eva Böhm, 2019. "Exploring the link between payment schemes and customer fraud: a mental accounting perspective," Journal of the Academy of Marketing Science, Springer, vol. 47(4), pages 595-616, July.
    5. Arndt, Aaron D. & Ford, John B. & Babin, Barry J. & Luong, Vinh, 2022. "Collecting samples from online services: How to use screeners to improve data quality," International Journal of Research in Marketing, Elsevier, vol. 39(1), pages 117-133.
    6. Sung, Billy & Crawford, Robert & Teah, Min & Stankovic, Michelle & Phau, Ian, 2020. "The “timber box†effect for premium wines," Journal of Retailing and Consumer Services, Elsevier, vol. 54(C).
    7. Kostyk, Alena & Zhou, Wenkai & Hyman, Michael R., 2019. "Using surveytainment to counter declining survey data quality," Journal of Business Research, Elsevier, vol. 95(C), pages 211-219.
    8. Harrigan, Paul & Evers, Uwana & Miles, Morgan & Daly, Timothy, 2017. "Customer engagement with tourism social media brands," Tourism Management, Elsevier, vol. 59(C), pages 597-609.
    9. Parra, Carlos M. & Gupta, Manjul & Cadden, Trevor, 2022. "Towards an understanding of remote work exhaustion: A study on the effects of individuals’ big five personality traits," Journal of Business Research, Elsevier, vol. 150(C), pages 653-662.
    10. Proestakis, Antonios & Marandola, Ginevra & Lourenço, Joana S. & van Bavel, René, 2024. "Testing a policy intervention in the lab: differences between students and non-students in switching bank accounts," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 111(C).
    11. Yamada, Katsunori & Sato, Masayuki, 2013. "Another avenue for anatomy of income comparisons: Evidence from hypothetical choice experiments," Journal of Economic Behavior & Organization, Elsevier, vol. 89(C), pages 35-57.
    12. Sweldens, Steven & Puntoni, Stefano & Paolacci, Gabriele & Vissers, Maarten, 2014. "The bias in the bias: Comparative optimism as a function of event social undesirability," Organizational Behavior and Human Decision Processes, Elsevier, vol. 124(2), pages 229-244.
    13. S. Venus Jin & Aziz Muqaddam, 2019. "Product placement 2.0: “Do Brands Need Influencers, or Do Influencers Need Brands?”," Journal of Brand Management, Palgrave Macmillan, vol. 26(5), pages 522-537, September.
    14. Thieme, Lutz & Winkelhake, Olaf & Hartmann, Ulrich, 2014. "Fairness als universelle Norm? Empirische Evidenz ohne Manna [Fairness as a universal norm? Empiric evidence without manna]," Working Papers of the European Institute for Socioeconomics 12, European Institute for Socioeconomics (EIS), Saarbrücken.
    15. Hsu, Dan K. & Burmeister-Lamp, Katrin & Simmons, Sharon A. & Foo, Maw-Der & Hong, Michelle C. & Pipes, Jesse D., 2019. "“I know I can, but I don't fit”: Perceived fit, self-efficacy, and entrepreneurial intention," Journal of Business Venturing, Elsevier, vol. 34(2), pages 311-326.
    16. Xie, Junyi & Ifie, Kemefasu & Gruber, Thorsten, 2022. "The dual threat of COVID-19 to health and job security – Exploring the role of mindfulness in sustaining frontline employee-related outcomes," Journal of Business Research, Elsevier, vol. 146(C), pages 216-227.
    17. Lutz, Christoph & Newlands, Gemma, 2018. "Consumer segmentation within the sharing economy: The case of Airbnb," Journal of Business Research, Elsevier, vol. 88(C), pages 187-196.
    18. Bouma, J.A. & Nguyen, Binh & van der Heijden, Eline & Dijk, J.J., 2018. "Analysing Group Contract Design Using a Lab and a Lab-in-the-Field Threshold Public Good Experiment," Discussion Paper 2018-049, Tilburg University, Center for Economic Research.
    19. Mariconda, Simone & Lurati, Francesco, 2015. "Does familiarity breed stability? The role of familiarity in moderating the effects of new information on reputation judgments," Journal of Business Research, Elsevier, vol. 68(5), pages 957-964.
    20. Ginder, Whitney & Byun, Sang-Eun, 2022. "To trust or not to trust? The interplay between labor-related CSR claim type and prior CSR reputation of apparel retailers," Journal of Retailing and Consumer Services, Elsevier, vol. 65(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:jbrese:v:118:y:2020:i:c:p:189-198. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/jbusres .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.