IDEAS home Printed from https://ideas.repec.org/p/arx/papers/1606.00908.html
   My bibliography  Save this paper

A/B Testing of Auctions

Author

Listed:
  • Shuchi Chawla
  • Jason D. Hartline
  • Denis Nekipelov

Abstract

For many application areas A/B testing, which partitions users of a system into an A (control) and B (treatment) group to experiment between several application designs, enables Internet companies to optimize their services to the behavioral patterns of their users. Unfortunately, the A/B testing framework cannot be applied in a straightforward manner to applications like auctions where the users (a.k.a., bidders) submit bids before the partitioning into the A and B groups is made. This paper combines auction theoretic modeling with the A/B testing framework to develop methodology for A/B testing auctions. The accuracy of our method %, assuming the auction is directly comparable to ideal A/B testing where there is no interference between A and B. Our results are based on an extension and improved analysis of the inference method of Chawla et al. (2014).

Suggested Citation

  • Shuchi Chawla & Jason D. Hartline & Denis Nekipelov, 2016. "A/B Testing of Auctions," Papers 1606.00908, arXiv.org.
    • Handle: RePEc:arx:papers:1606.00908
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/1606.00908
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Emmanuel Guerre & Isabelle Perrigne & Quang Vuong, 2000. "Optimal Nonparametric Estimation of First-Price Auctions," Econometrica, Econometric Society, vol. 68(3), pages 525-574, May.
    2. Ilya Segal, 2003. "Optimal Pricing Mechanisms with Unknown Demand," American Economic Review, American Economic Association, vol. 93(3), pages 509-529, June.
    3. Athey, Susan & Haile, Philip A., 2007. "Nonparametric Approaches to Auctions," Handbook of Econometrics, in: J.J. Heckman & E.E. Leamer (ed.), Handbook of Econometrics, edition 1, volume 6, chapter 60, Elsevier.
    4. Baliga Sandeep & Vohra Rakesh, 2003. "Market Research and Market Design," The B.E. Journal of Theoretical Economics, De Gruyter, vol. 3(1), pages 1-27, August.
    5. Harry J. Paarsch & Han Hong, 2006. "An Introduction to the Structural Econometrics of Auction Data," MIT Press Books, The MIT Press, edition 1, volume 1, number 0262162350, April.
    6. Benjamin Edelman & Michael Ostrovsky & Michael Schwarz, 2007. "Internet Advertising and the Generalized Second-Price Auction: Selling Billions of Dollars Worth of Keywords," American Economic Review, American Economic Association, vol. 97(1), pages 242-259, March.
    7. Goldberg, Andrew V. & Hartline, Jason D. & Karlin, Anna R. & Saks, Michael & Wright, Andrew, 2006. "Competitive auctions," Games and Economic Behavior, Elsevier, vol. 55(2), pages 242-269, May.
    8. David H. Reiley, 2006. "Field experiments on the effects of reserve prices in auctions: more Magic on the Internet," RAND Journal of Economics, RAND Corporation, vol. 37(1), pages 195-211, March.
    9. Roger B. Myerson, 1981. "Optimal Auction Design," Mathematics of Operations Research, INFORMS, vol. 6(1), pages 58-73, February.
    10. Elodie Guerre & I. Perrigne & Q.H. Vuong, 2000. "Optimal nonparametric estimation of first-price auctions [[Estimation nonparamétrique optimale des enchères au premier prix]]," Post-Print hal-02697497, HAL.
    11. Jennifer Brown & John Morgan, 2009. "How Much Is a Dollar Worth? Tipping versus Equilibrium Coexistence on Competing Online Auction Sites," Journal of Political Economy, University of Chicago Press, vol. 117(4), pages 668-700, August.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Loertscher, Simon & Marx, Leslie M., 2020. "Asymptotically optimal prior-free clock auctions," Journal of Economic Theory, Elsevier, vol. 187(C).
    2. Devanur, Nikhil R. & Hartline, Jason D. & Yan, Qiqi, 2015. "Envy freedom and prior-free mechanism design," Journal of Economic Theory, Elsevier, vol. 156(C), pages 103-143.
    3. Hickman Brent R. & Hubbard Timothy P. & Sağlam Yiğit, 2012. "Structural Econometric Methods in Auctions: A Guide to the Literature," Journal of Econometric Methods, De Gruyter, vol. 1(1), pages 67-106, August.
    4. Hickman Brent R. & Hubbard Timothy P. & Sağlam Yiğit, 2012. "Structural Econometric Methods in Auctions: A Guide to the Literature," Journal of Econometric Methods, De Gruyter, vol. 1(1), pages 67-106, August.
    5. Dominic Coey & Bradley J. Larsen & Kane Sweeney & Caio Waisman, 2021. "Scalable Optimal Online Auctions," Marketing Science, INFORMS, vol. 40(4), pages 593-618, July.
    6. Lamy, Laurent, 2012. "The econometrics of auctions with asymmetric anonymous bidders," Journal of Econometrics, Elsevier, vol. 167(1), pages 113-132.
    7. Omid Rafieian & Hema Yoganarasimhan, 2021. "Targeting and Privacy in Mobile Advertising," Marketing Science, INFORMS, vol. 40(2), pages 193-218, March.
    8. Brasil, Eric Universo Rodrigues & Postali, Fernando Antonio Slaibe, 2014. "Informational rents in oil and gas concession auctions in Brazil," Energy Economics, Elsevier, vol. 46(C), pages 93-101.
    9. Syngjoo Choi & Lars Nesheim & Imran Rasul, 2016. "Reserve Price Effects In Auctions: Estimates From Multiple Regression-Discontinuity Designs," Economic Inquiry, Western Economic Association International, vol. 54(1), pages 294-314, January.
    10. Jun, Sung Jae & Zincenko, Federico, 2022. "Testing for risk aversion in first-price sealed-bid auctions," Journal of Econometrics, Elsevier, vol. 226(2), pages 295-320.
    11. Dimopoulos, Theodosios & Sacchetto, Stefano, 2014. "Preemptive bidding, target resistance, and takeover premiums," Journal of Financial Economics, Elsevier, vol. 114(3), pages 444-470.
    12. Leonardo Rezende, 2008. "Econometrics of auctions by least squares," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 23(7), pages 925-948.
    13. Michael Ostrovsky & Michael Schwarz, 2023. "Reserve Prices in Internet Advertising Auctions: A Field Experiment," Journal of Political Economy, University of Chicago Press, vol. 131(12), pages 3352-3376.
    14. Philip A Haile & Yuichi Kitamura, 2019. "Unobserved heterogeneity in auctions," The Econometrics Journal, Royal Economic Society, vol. 22(1), pages 1-19.
    15. Zincenko, Federico, 2018. "Nonparametric estimation of first-price auctions with risk-averse bidders," Journal of Econometrics, Elsevier, vol. 205(2), pages 303-335.
    16. Marmer, Vadim & Shneyerov, Artyom & Xu, Pai, 2013. "What model for entry in first-price auctions? A nonparametric approach," Journal of Econometrics, Elsevier, vol. 176(1), pages 46-58.
    17. Timothy P. Hubbard & Rene Kirkegaard, 2015. "Asymmetric Auctions with More Than Two Bidders," Working Papers 1502, University of Guelph, Department of Economics and Finance.
    18. Hortaçsu, Ali, 2011. "Recent progress in the empirical analysis of multi-unit auctions," International Journal of Industrial Organization, Elsevier, vol. 29(3), pages 345-349, May.
    19. Jiaming Mao & Jingzhi Xu, 2020. "Ensemble Learning with Statistical and Structural Models," Papers 2006.05308, arXiv.org.
    20. Shneyerov, Artyom & Wong, Adam Chi Leung, 2011. "Identification in first-price and Dutch auctions when the number of potential bidders is unobservable," Games and Economic Behavior, Elsevier, vol. 72(2), pages 574-582, June.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:1606.00908. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.