IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2311.14698.html
   My bibliography  Save this paper

Business Policy Experiments using Fractional Factorial Designs: Consumer Retention on DoorDash

Author

Listed:
  • Yixin Tang
  • Yicong Lin
  • Navdeep S. Sahni

Abstract

This paper investigates an approach to both speed up business decision-making and lower the cost of learning through experimentation by factorizing business policies and employing fractional factorial experimental designs for their evaluation. We illustrate how this method integrates with advances in the estimation of heterogeneous treatment effects, elaborating on its advantages and foundational assumptions. We empirically demonstrate the implementation and benefits of our approach and assess its validity in evaluating consumer promotion policies at DoorDash, which is one of the largest delivery platforms in the US. Our approach discovers a policy with 5% incremental profit at 67% lower implementation cost.

Suggested Citation

  • Yixin Tang & Yicong Lin & Navdeep S. Sahni, 2023. "Business Policy Experiments using Fractional Factorial Designs: Consumer Retention on DoorDash," Papers 2311.14698, arXiv.org, revised Nov 2023.
    • Handle: RePEc:arx:papers:2311.14698
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2311.14698
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Eric M. Schwartz & Eric T. Bradlow & Peter S. Fader, 2017. "Customer Acquisition via Display Advertising Using Multi-Armed Bandit Experiments," Marketing Science, INFORMS, vol. 36(4), pages 500-522, July.
    2. Susan Athey & Raj Chetty & Guido W. Imbens & Hyunseung Kang, 2019. "The Surrogate Index: Combining Short-Term Proxies to Estimate Long-Term Treatment Effects More Rapidly and Precisely," NBER Working Papers 26463, National Bureau of Economic Research, Inc.
    3. Steven L. Scott, 2010. "A modern Bayesian look at the multi‐armed bandit," Applied Stochastic Models in Business and Industry, John Wiley & Sons, vol. 26(6), pages 639-658, November.
    4. Stefan Wager & Susan Athey, 2018. "Estimation and Inference of Heterogeneous Treatment Effects using Random Forests," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 113(523), pages 1228-1242, July.
    5. Elea McDonnell Feit & Ron Berman, 2019. "Test & Roll: Profit-Maximizing A/B Tests," Marketing Science, INFORMS, vol. 38(6), pages 1038-1058, November.
    6. Reiss, Peter C. & Wolak, Frank A., 2007. "Structural Econometric Modeling: Rationales and Examples from Industrial Organization," Handbook of Econometrics, in: J.J. Heckman & E.E. Leamer (ed.), Handbook of Econometrics, edition 1, volume 6, chapter 64, Elsevier.
    7. Green, Paul E & Srinivasan, V, 1978. "Conjoint Analysis in Consumer Research: Issues and Outlook," Journal of Consumer Research, Journal of Consumer Research Inc., vol. 5(2), pages 103-123, Se.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. A Stefano Caria & Grant Gordon & Maximilian Kasy & Simon Quinn & Soha Osman Shami & Alexander Teytelboym, 2024. "An Adaptive Targeted Field Experiment: Job Search Assistance for Refugees in Jordan," Journal of the European Economic Association, European Economic Association, vol. 22(2), pages 781-836.
    2. Gui Liberali & Alina Ferecatu, 2022. "Morphing for Consumer Dynamics: Bandits Meet Hidden Markov Models," Marketing Science, INFORMS, vol. 41(4), pages 769-794, July.
    3. Guido W. Imbens, 2020. "Potential Outcome and Directed Acyclic Graph Approaches to Causality: Relevance for Empirical Practice in Economics," Journal of Economic Literature, American Economic Association, vol. 58(4), pages 1129-1179, December.
    4. Aysegül Kayaoglu & Ghassan Baliki & Tilman Brück & Melodie Al Daccache & Dorothee Weiffen, 2023. "How to conduct impact evaluations in humanitarian and conflict settings," HiCN Working Papers 387, Households in Conflict Network.
    5. Dean Eckles & Maurits Kaptein, 2019. "Bootstrap Thompson Sampling and Sequential Decision Problems in the Behavioral Sciences," SAGE Open, , vol. 9(2), pages 21582440198, June.
    6. Mingyu Joo & Michael L. Thompson & Greg M. Allenby6, 2019. "Optimal Product Design by Sequential Experiments in High Dimensions," Management Science, INFORMS, vol. 65(7), pages 3235-3254, July.
    7. Hana Choi & Carl F. Mela & Santiago R. Balseiro & Adam Leary, 2020. "Online Display Advertising Markets: A Literature Review and Future Directions," Information Systems Research, INFORMS, vol. 31(2), pages 556-575, June.
    8. Dave Donaldson, 2022. "Blending Theory and Data: A Space Odyssey," Journal of Economic Perspectives, American Economic Association, vol. 36(3), pages 185-210, Summer.
    9. Ma, Liye & Sun, Baohong, 2020. "Machine learning and AI in marketing – Connecting computing power to human insights," International Journal of Research in Marketing, Elsevier, vol. 37(3), pages 481-504.
    10. Elea McDonnell Feit & Ron Berman, 2019. "Test & Roll: Profit-Maximizing A/B Tests," Marketing Science, INFORMS, vol. 38(6), pages 1038-1058, November.
    11. Athey, Susan & Imbens, Guido W., 2019. "Machine Learning Methods Economists Should Know About," Research Papers 3776, Stanford University, Graduate School of Business.
    12. Khashayar Khosravi & Greg Lewis & Vasilis Syrgkanis, 2019. "Non-Parametric Inference Adaptive to Intrinsic Dimension," Papers 1901.03719, arXiv.org, revised Jun 2019.
    13. Alina Ferecatu & Arnaud De Bruyn, 2022. "Understanding Managers’ Trade-Offs Between Exploration and Exploitation," Marketing Science, INFORMS, vol. 41(1), pages 139-165, January.
    14. Winfried Steiner & Harald Hruschka, 2002. "A Probabilistic One-Step Approach to the Optimal Product Line Design Problem Using Conjoint and Cost Data," Review of Marketing Science Working Papers 1-4-1003, Berkeley Electronic Press.
    15. Lechner, Michael, 2018. "Modified Causal Forests for Estimating Heterogeneous Causal Effects," IZA Discussion Papers 12040, Institute of Labor Economics (IZA).
    16. William Arbour, 2021. "Can Recidivism be Prevented from Behind Bars? Evidence from a Behavioral Program," Working Papers tecipa-683, University of Toronto, Department of Economics.
    17. Alexandre Belloni & Victor Chernozhukov & Denis Chetverikov & Christian Hansen & Kengo Kato, 2018. "High-dimensional econometrics and regularized GMM," CeMMAP working papers CWP35/18, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    18. Dimitris Bertsimas & Agni Orfanoudaki & Rory B. Weiner, 2020. "Personalized treatment for coronary artery disease patients: a machine learning approach," Health Care Management Science, Springer, vol. 23(4), pages 482-506, December.
    19. Manthos D. Delis & Sotirios Kokas & Steven Ongena, 2016. "Foreign Ownership and Market Power in Banking: Evidence from a World Sample," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 48(2-3), pages 449-483, March.
    20. Merja Halme & Kari Linden & Kimmo Kääriä, 2009. "Patients’ Preferences for Generic and Branded Over-the-Counter Medicines," The Patient: Patient-Centered Outcomes Research, Springer;International Academy of Health Preference Research, vol. 2(4), pages 243-255, December.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2311.14698. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.