IDEAS home Printed from https://ideas.repec.org/p/eti/dpaper/22097.html
   My bibliography  Save this paper

Off-policy Evaluation with General Logging Policies: Implementation at Mercari

Author

Listed:
  • NARITA Yusuke
  • OKUMURA Kyohei
  • SHIMIZU Akihiro
  • YATA Kohei

Abstract

Off-policy evaluation (OPE) attempts to predict the performance of counterfactual policies using log data from a different policy. We extend its applicability by developing an OPE method for a class of both full support and deficient support logging policies in contextual-bandit settings. This class includes deterministic bandit (such as Upper Confidence Bound) as well as deterministic decision-making based on supervised and unsupervised learning. We prove that our method's prediction converges in probability to the true performance of a counterfactual policy as the sample size increases. We validate our method with experiments on partly and entirely deterministic logging policies. Finally, we apply it to evaluate coupon targeting policies by a major online platform and show how to improve the existing policy.

Suggested Citation

  • NARITA Yusuke & OKUMURA Kyohei & SHIMIZU Akihiro & YATA Kohei, 2022. "Off-policy Evaluation with General Logging Policies: Implementation at Mercari," Discussion papers 22097, Research Institute of Economy, Trade and Industry (RIETI).
    • Handle: RePEc:eti:dpaper:22097
    as

    Download full text from publisher

    File URL: https://www.rieti.go.jp/jp/publications/dp/22e097.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. David S. Lee & Thomas Lemieux, 2010. "Regression Discontinuity Designs in Economics," Journal of Economic Literature, American Economic Association, vol. 48(2), pages 281-355, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. KAMKOUM, Arnaud Cedric, 2023. "The Federal Reserve’s Response to the Global Financial Crisis and its Effects: An Interrupted Time-Series Analysis of the Impact of its Quantitative Easing Programs," Thesis Commons d7pvg, Center for Open Science.
    2. Cao, Chunfang & Li, Xiaoyang & Xia, Changyuan, 2021. "The complicit role of local government authorities in corporate bribery: Evidence from a tax collection reform in China," China Economic Review, Elsevier, vol. 65(C).
    3. Stephan Litschig, 2008. "Financing local development: Quasi-experimental evidence from municipalities in Brazil, 1980-1991," Economics Working Papers 1142, Department of Economics and Business, Universitat Pompeu Fabra, revised Jun 2012.
    4. Riccardo Crescenzi & Mara Giua, 2018. "One or Many Cohesion Policies of the European Union? On the Diverging Impacts of Cohesion Policy across Member States," SERC Discussion Papers 0230, Centre for Economic Performance, LSE.
    5. Chen, Yi & Zhao, Yi, 2022. "The timing of first marriage and subsequent life outcomes: Evidence from a natural experiment," Journal of Comparative Economics, Elsevier, vol. 50(3), pages 713-731.
    6. Stancanelli, Elena G. F., 2012. "Spouses' Retirement and Hours Outcomes: Evidence from Twofold Regression Discontinuity with Differences-in-Differences," IZA Discussion Papers 6791, Institute of Labor Economics (IZA).
    7. Christopher Ksoll & Seth Morgan & Kristine Bos & Randall Blair, "undated". "Evaluation of the Burkina Faso Agriculture Development Project: Baseline Report," Mathematica Policy Research Reports d56281560c084ba5af1233ed9, Mathematica Policy Research.
    8. Tianyan Hu & Sandra L. Decker & Shin-Yi Chou, 2014. "The Impact of Health Insurance Expansion on Physician Treatment Choice: Medicare Part D and Physician Prescribing," NBER Working Papers 20708, National Bureau of Economic Research, Inc.
    9. Fanny Landaud & Son Thierry Ly & Éric Maurin, 2020. "Competitive Schools and the Gender Gap in the Choice of Field of Study," Journal of Human Resources, University of Wisconsin Press, vol. 55(1), pages 278-308.
    10. Huang, Wei & Zhou, Yi, 2013. "Effects of education on cognition at older ages: Evidence from China's Great Famine," Social Science & Medicine, Elsevier, vol. 98(C), pages 54-62.
    11. Marini, Andrea, 2024. "Updating the retirement-consumption puzzle in Italy: who are the most affected?," Working Paper Series 2936, European Central Bank.
    12. Konstantinos Matakos & Riikka Savolainen & Orestis Troumpounis & Janne Tukiainen & Dimitrios Xefteris, 2024. "Electoral Institutions and Intraparty Cohesion," Journal of Political Economy Microeconomics, University of Chicago Press, vol. 2(4), pages 883-916.
    13. Ivan A Canay & Vishal Kamat, 2018. "Approximate Permutation Tests and Induced Order Statistics in the Regression Discontinuity Design," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 85(3), pages 1577-1608.
    14. Touria Jaaidane & Sophie Larribeau, 2023. "Partisanship and the effectiveness of Fiscal Equalization: Evidence from French Municipalities," Economics Working Paper Archive (University of Rennes & University of Caen) 2023-01, Center for Research in Economics and Management (CREM), University of Rennes, University of Caen and CNRS.
    15. Silvia Helena Barcellos, 2010. "Legalization and the Economic Status of Immigrants," Working Papers 754, RAND Corporation.
    16. De Paola, Maria & Scoppa, Vincenzo, 2015. "Procrastination, academic success and the effectiveness of a remedial program," Journal of Economic Behavior & Organization, Elsevier, vol. 115(C), pages 217-236.
    17. Arteaga, Irma & Heflin, Colleen & Gable, Sara, 2016. "The impact of aging out of WIC on food security in households with children," Children and Youth Services Review, Elsevier, vol. 69(C), pages 82-96.
    18. Frederik von Waldow & Heike Link, 2024. "Spatial Competition and Pass-through of Fuel Taxes: Evidence from a Quasi-natural Experiment in Germany," Discussion Papers of DIW Berlin 2086, DIW Berlin, German Institute for Economic Research.
    19. Chong Lu, 2022. "The effect of migration on rural residents’ intergenerational subjective social status mobility in China," Quality & Quantity: International Journal of Methodology, Springer, vol. 56(5), pages 3279-3308, October.
    20. Genqiang Lei & Xiaohong Huang & Penghui Xi, 2016. "The impact of transfer payments on urban-rural income gap: based on fuzzy RD analysis of China’s midwestern county data," China Finance and Economic Review, Springer, vol. 4(1), pages 1-17, December.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eti:dpaper:22097. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: TANIMOTO, Toko (email available below). General contact details of provider: https://edirc.repec.org/data/rietijp.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.