IDEAS home Printed from https://ideas.repec.org/p/ehl/lserod/122741.html
   My bibliography  Save this paper

Policy evaluation for temporal and/or spatial dependent experiments

Author

Listed:
  • Luo, Shikai
  • Yang, Ying
  • Shi, Chengchun
  • Yao, Fang
  • Ye, Jieping
  • Zhu, Hongtu

Abstract

The aim of this article is to establish a causal link between the policies implemented by technology companies and the outcomes they yield within intricate temporal and/or spatial dependent experiments. We propose a novel temporal/spatio-Temporal Varying Coefficient Decision Process model, capable of effectively capturing the evolving treatment effects in situations characterized by temporal and/or spatial dependence. Our methodology encompasses the decomposition of the average treatment effect into the direct effect (DE) and the indirect effect (IE). We subsequently devise comprehensive procedures for estimating and making inferences about both DE and IE. Additionally, we provide a rigorous analysis of the statistical properties of these procedures, such as asymptotic power. To substantiate the effectiveness of our approach, we carry out extensive simulations and real data analyses.

Suggested Citation

  • Luo, Shikai & Yang, Ying & Shi, Chengchun & Yao, Fang & Ye, Jieping & Zhu, Hongtu, 2024. "Policy evaluation for temporal and/or spatial dependent experiments," LSE Research Online Documents on Economics 122741, London School of Economics and Political Science, LSE Library.
    • Handle: RePEc:ehl:lserod:122741
    as

    Download full text from publisher

    File URL: http://eprints.lse.ac.uk/122741/
    File Function: Open access version.
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Clément de Chaisemartin & Xavier D'Haultfœuille, 2020. "Two-Way Fixed Effects Estimators with Heterogeneous Treatment Effects," American Economic Review, American Economic Association, vol. 110(9), pages 2964-2996, September.
    2. Iavor Bojinov & Neil Shephard, 2019. "Time Series Experiments and Causal Estimands: Exact Randomization Tests and Trading," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 114(528), pages 1665-1682, October.
    3. Michael E. Sobel & Martin A. Lindquist, 2014. "Causal Inference for fMRI Time Series Data With Systematic Errors of Measurement in a Balanced On/Off Study of Social Evaluative Threat," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 109(507), pages 967-976, September.
    4. Andrei Hagiu & Julian Wright, 2019. "The status of workers and platforms in the sharing economy," Journal of Economics & Management Strategy, Wiley Blackwell, vol. 28(1), pages 97-108, January.
    5. Fan Zhou & Shikai Luo & Xiaohu Qie & Jieping Ye & Hongtu Zhu, 2021. "Graph-Based Equilibrium Metrics for Dynamic Supply–Demand Systems With Applications to Ride-sourcing Platforms," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 116(536), pages 1688-1699, October.
    6. Charles F. Manski, 2013. "Identification of treatment response with social interactions," Econometrics Journal, Royal Economic Society, vol. 16(1), pages 1-23, February.
    7. Audrey Boruvka & Daniel Almirall & Katie Witkiewitz & Susan A. Murphy, 2018. "Assessing Time-Varying Causal Effect Moderation in Mobile Health," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 113(523), pages 1112-1121, July.
    8. Hudgens, Michael G. & Halloran, M. Elizabeth, 2008. "Toward Causal Inference With Interference," Journal of the American Statistical Association, American Statistical Association, vol. 103, pages 832-842, June.
    9. Yuchen Hu & Shuangning Li & Stefan Wager, 2021. "Average Direct and Indirect Causal Effects under Interference," Papers 2104.03802, arXiv.org, revised Jan 2022.
    10. Fan J. & Li R., 2001. "Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties," Journal of the American Statistical Association, American Statistical Association, vol. 96, pages 1348-1360, December.
    11. Ramesh Johari & Hannah Li & Inessa Liskovich & Gabriel Y. Weintraub, 2022. "Experimental Design in Two-Sided Platforms: An Analysis of Bias," Management Science, INFORMS, vol. 68(10), pages 7069-7089, October.
    12. Maxime C. Cohen & Michael D. Fiszer & Baek Jung Kim, 2022. "Frustration-Based Promotions: Field Experiments in Ride-Sharing," Management Science, INFORMS, vol. 68(4), pages 2432-2464, April.
    13. Yuchen Hu & Shuangning Li & Stefan Wager, 2022. "Average direct and indirect causal effects under interference [Estimating average causal effects under general interference, with application to a social network experiment]," Biometrika, Biometrika Trust, vol. 109(4), pages 1165-1172.
    14. Kostas Bimpikis & Ozan Candogan & Daniela Saban, 2019. "Spatial Pricing in Ride-Sharing Networks," Operations Research, INFORMS, vol. 67(3), pages 744-769, May.
    15. Xianyang Zhang & Guang Cheng, 2017. "Simultaneous Inference for High-Dimensional Linear Models," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 112(518), pages 757-768, April.
    16. L. Liu & M. G. Hudgens & S. Becker-Dreps, 2016. "On inverse probability-weighted estimators in the presence of interference," Biometrika, Biometrika Trust, vol. 103(4), pages 829-842.
    17. Carolina Perez-Heydrich & Michael G. Hudgens & M. Elizabeth Halloran & John D. Clemens & Mohammad Ali & Michael E. Emch, 2014. "Assessing effects of cholera vaccination in the presence of interference," Biometrics, The International Biometric Society, vol. 70(3), pages 731-741, September.
    18. Ruben Dezeure & Peter Bühlmann & Cun-Hui Zhang, 2017. "High-dimensional simultaneous inference with the bootstrap," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 26(4), pages 685-719, December.
    19. Stefan Wager & Kuang Xu, 2021. "Experimenting in Equilibrium," Management Science, INFORMS, vol. 67(11), pages 6694-6715, November.
    20. Daniel J. Luckett & Eric B. Laber & Anna R. Kahkoska & David M. Maahs & Elizabeth Mayer-Davis & Michael R. Kosorok, 2020. "Estimating Dynamic Treatment Regimes in Mobile Health Using V-Learning," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 115(530), pages 692-706, April.
    21. Baqun Zhang & Anastasios A. Tsiatis & Eric B. Laber & Marie Davidian, 2013. "Robust estimation of optimal dynamic treatment regimes for sequential treatment decisions," Biometrika, Biometrika Trust, vol. 100(3), pages 681-694.
    22. Victor Chernozhukov & Denis Chetverikov & Kengo Kato, 2012. "Gaussian approximations and multiplier bootstrap for maxima of sums of high-dimensional random vectors," Papers 1212.6906, arXiv.org, revised Jan 2018.
    23. Lee, Lung-fei, 2007. "Identification and estimation of econometric models with group interactions, contextual factors and fixed effects," Journal of Econometrics, Elsevier, vol. 140(2), pages 333-374, October.
    24. Peng Liao & Predrag Klasnja & Susan Murphy, 2021. "Off-Policy Estimation of Long-Term Average Outcomes With Applications to Mobile Health," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 116(533), pages 382-391, March.
    25. Chengchun Shi & Rui Song & Wenbin Lu & Runze Li, 2021. "Statistical Inference for High-Dimensional Models via Recursive Online-Score Estimation," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 116(535), pages 1307-1318, July.
    26. Hongtu Zhu & Jianqing Fan & Linglong Kong, 2014. "Spatially Varying Coefficient Model for Neuroimaging Data With Jump Discontinuities," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 109(507), pages 1084-1098, September.
    27. Shi, Chengchun & Zhang, Shengxing & Lu, Wenbin & Song, Rui, 2022. "Statistical inference of the value function for reinforcement learning in infinite-horizon settings," LSE Research Online Documents on Economics 110882, London School of Economics and Political Science, LSE Library.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Li, Ting & Shi, Chengchun & Lu, Zhaohua & Li, Yi & Zhu, Hongtu, 2024. "Evaluating dynamic conditional quantile treatment effects with applications in ridesharing," LSE Research Online Documents on Economics 122488, London School of Economics and Political Science, LSE Library.
    2. Luofeng Liao & Christian Kroer, 2024. "Statistical Inference and A/B Testing in Fisher Markets and Paced Auctions," Papers 2406.15522, arXiv.org, revised Aug 2024.
    3. Jinglong Zhao, 2024. "Experimental Design For Causal Inference Through An Optimization Lens," Papers 2408.09607, arXiv.org, revised Aug 2024.
    4. Shi, Chengchun & Wan, Runzhe & Song, Ge & Luo, Shikai & Zhu, Hongtu & Song, Rui, 2023. "A multiagent reinforcement learning framework for off-policy evaluation in two-sided markets," LSE Research Online Documents on Economics 117174, London School of Economics and Political Science, LSE Library.
    5. Yuehao Bai & Azeem M. Shaikh & Max Tabord-Meehan, 2024. "A Primer on the Analysis of Randomized Experiments and a Survey of some Recent Advances," Papers 2405.03910, arXiv.org.
    6. Yi Zhang & Kosuke Imai, 2023. "Individualized Policy Evaluation and Learning under Clustered Network Interference," Papers 2311.02467, arXiv.org, revised Feb 2024.
    7. Han, Kevin & Basse, Guillaume & Bojinov, Iavor, 2024. "Population interference in panel experiments," Journal of Econometrics, Elsevier, vol. 238(1).
    8. Denis Fougère & Nicolas Jacquemet, 2020. "Policy Evaluation Using Causal Inference Methods," SciencePo Working papers Main hal-03455978, HAL.
    9. Cyrus Samii & Ye Wang & Jonathan Sullivan & P. M. Aronow, 2023. "Inference in Spatial Experiments with Interference using the SpatialEffect Package," Journal of Agricultural, Biological and Environmental Statistics, Springer;The International Biometric Society;American Statistical Association, vol. 28(1), pages 138-156, March.
    10. Haoge Chang, 2023. "Design-based Estimation Theory for Complex Experiments," Papers 2311.06891, arXiv.org.
    11. Davide Viviano & Jelena Bradic, 2021. "Dynamic covariate balancing: estimating treatment effects over time with potential local projections," Papers 2103.01280, arXiv.org, revised Jan 2024.
    12. Georgia Papadogeorgou & Kosuke Imai & Jason Lyall & Fan Li, 2022. "Causal inference with spatio‐temporal data: Estimating the effects of airstrikes on insurgent violence in Iraq," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 84(5), pages 1969-1999, November.
    13. Luofeng Liao & Christian Kroer, 2023. "Statistical Inference and A/B Testing for First-Price Pacing Equilibria," Papers 2301.02276, arXiv.org, revised Jun 2023.
    14. Anish Agarwal & Sarah H. Cen & Devavrat Shah & Christina Lee Yu, 2022. "Network Synthetic Interventions: A Causal Framework for Panel Data Under Network Interference," Papers 2210.11355, arXiv.org, revised Oct 2023.
    15. Gao, Yuhe & Shi, Chengchun & Song, Rui, 2023. "Deep spectral Q-learning with application to mobile health," LSE Research Online Documents on Economics 119445, London School of Economics and Political Science, LSE Library.
    16. Alexandre Belloni & Victor Chernozhukov & Denis Chetverikov & Christian Hansen & Kengo Kato, 2018. "High-dimensional econometrics and regularized GMM," CeMMAP working papers CWP35/18, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    17. Yann Bramoullé & Habiba Djebbari & Bernard Fortin, 2020. "Peer Effects in Networks: A Survey," Annual Review of Economics, Annual Reviews, vol. 12(1), pages 603-629, August.
    18. Brian J. Reich & Shu Yang & Yawen Guan & Andrew B. Giffin & Matthew J. Miller & Ana Rappold, 2021. "A Review of Spatial Causal Inference Methods for Environmental and Epidemiological Applications," International Statistical Review, International Statistical Institute, vol. 89(3), pages 605-634, December.
    19. Joseph Puleo & Ashley Buchanan & Natallia Katenka & M. Elizabeth Halloran & Samuel R. Friedman & Georgios Nikolopoulos, 2024. "Assessing Spillover Effects of Medications for Opioid Use Disorder on HIV Risk Behaviors among a Network of People Who Inject Drugs," Stats, MDPI, vol. 7(2), pages 1-27, June.
    20. A. Giffin & B. J. Reich & S. Yang & A. G. Rappold, 2023. "Generalized propensity score approach to causal inference with spatial interference," Biometrics, The International Biometric Society, vol. 79(3), pages 2220-2231, September.

    More about this item

    Keywords

    A/B testing; policy evaluation; spatio-temporal dependent experiments; varying coefficient decision process;
    All these keywords.

    JEL classification:

    • C1 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ehl:lserod:122741. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: LSERO Manager (email available below). General contact details of provider: https://edirc.repec.org/data/lsepsuk.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.