IDEAS home Printed from https://ideas.repec.org/a/spr/annopr/v208y2013i1p383-41610.1007-s10479-012-1248-5.html
   My bibliography  Save this article

Batch mode reinforcement learning based on the synthesis of artificial trajectories

Author

Listed:
  • Raphael Fonteneau
  • Susan Murphy
  • Louis Wehenkel
  • Damien Ernst

Abstract

In this paper, we consider the batch mode reinforcement learning setting, where the central problem is to learn from a sample of trajectories a policy that satisfies or optimizes a performance criterion. We focus on the continuous state space case for which usual resolution schemes rely on function approximators either to represent the underlying control problem or to represent its value function. As an alternative to the use of function approximators, we rely on the synthesis of “artificial trajectories” from the given sample of trajectories, and show that this idea opens new avenues for designing and analyzing algorithms for batch mode reinforcement learning. Copyright Springer Science+Business Media New York 2013

Suggested Citation

  • Raphael Fonteneau & Susan Murphy & Louis Wehenkel & Damien Ernst, 2013. "Batch mode reinforcement learning based on the synthesis of artificial trajectories," Annals of Operations Research, Springer, vol. 208(1), pages 383-416, September.
    • Handle: RePEc:spr:annopr:v:208:y:2013:i:1:p:383-416:10.1007/s10479-012-1248-5
    DOI: 10.1007/s10479-012-1248-5
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1007/s10479-012-1248-5
    Download Restriction: Access to full text is restricted to subscribers.
    File URL: https://libkey.io/10.1007/s10479-012-1248-5?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Murphy S.A. & van der Laan M.J. & Robins J.M., 2001. "Marginal Mean Models for Dynamic Regimes," Journal of the American Statistical Association, American Statistical Association, vol. 96, pages 1410-1423, December.
    2. S. A. Murphy, 2003. "Optimal dynamic treatment regimes," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 65(2), pages 331-355, May.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Ruitu Xu & Yifei Min & Tianhao Wang & Zhaoran Wang & Michael I. Jordan & Zhuoran Yang, 2023. "Finding Regularized Competitive Equilibria of Heterogeneous Agent Macroeconomic Models with Reinforcement Learning," Papers 2303.04833, arXiv.org.
    2. Stefano Bromuri, 2019. "Dynamic heuristic acceleration of linearly approximated SARSA( $$\lambda $$ λ ): using ant colony optimization to learn heuristics dynamically," Journal of Heuristics, Springer, vol. 25(6), pages 901-932, December.
    3. Shosei Sakaguchi, 2024. "Policy Learning for Optimal Dynamic Treatment Regimes with Observational Data," Papers 2404.00221, arXiv.org, revised Dec 2024.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yiwang Zhou & Peter X.K. Song & Haoda Fu, 2021. "Net benefit index: Assessing the influence of a biomarker for individualized treatment rules," Biometrics, The International Biometric Society, vol. 77(4), pages 1254-1264, December.
    2. Xinyu Tang & Abdus S. Wahed, 2011. "Comparison of treatment regimes with adjustment for auxiliary variables," Journal of Applied Statistics, Taylor & Francis Journals, vol. 38(12), pages 2925-2938, March.
    3. Hongming Pu & Bo Zhang, 2021. "Estimating optimal treatment rules with an instrumental variable: A partial identification learning approach," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 83(2), pages 318-345, April.
    4. Yuqian Zhang & Weijie Ji & Jelena Bradic, 2021. "Dynamic treatment effects: high-dimensional inference under model misspecification," Papers 2111.06818, arXiv.org, revised Jun 2023.
    5. Han, Sukjin, 2021. "Identification in nonparametric models for dynamic treatment effects," Journal of Econometrics, Elsevier, vol. 225(2), pages 132-147.
    6. Ashesh Rambachan & Neil Shephard, 2019. "When do common time series estimands have nonparametric causal meaning?," Papers 1903.01637, arXiv.org, revised Jan 2025.
    7. Jelena Bradic & Weijie Ji & Yuqian Zhang, 2021. "High-dimensional Inference for Dynamic Treatment Effects," Papers 2110.04924, arXiv.org, revised May 2023.
    8. Anders Bredahl Kock & Martin Thyrsgaard, 2017. "Optimal sequential treatment allocation," Papers 1705.09952, arXiv.org, revised Aug 2018.
    9. Callaway, Brantly & Sant’Anna, Pedro H.C., 2021. "Difference-in-Differences with multiple time periods," Journal of Econometrics, Elsevier, vol. 225(2), pages 200-230.
    10. Alex Chin & Dean Eckles & Johan Ugander, 2022. "Evaluating Stochastic Seeding Strategies in Networks," Management Science, INFORMS, vol. 68(3), pages 1714-1736, March.
    11. Nina Zhou & Lu Wang & Daniel Almirall, 2023. "Estimating tree‐based dynamic treatment regimes using observational data with restricted treatment sequences," Biometrics, The International Biometric Society, vol. 79(3), pages 2260-2271, September.
    12. Orellana Liliana & Rotnitzky Andrea & Robins James M., 2010. "Dynamic Regime Marginal Structural Mean Models for Estimation of Optimal Dynamic Treatment Regimes, Part I: Main Content," The International Journal of Biostatistics, De Gruyter, vol. 6(2), pages 1-49, March.
    13. Shuxiao Chen & Bo Zhang, 2021. "Estimating and Improving Dynamic Treatment Regimes With a Time-Varying Instrumental Variable," Papers 2104.07822, arXiv.org.
    14. Jin Wang & Donglin Zeng & D. Y. Lin, 2022. "Semiparametric single-index models for optimal treatment regimens with censored outcomes," Lifetime Data Analysis: An International Journal Devoted to Statistical Methods and Applications for Time-to-Event Data, Springer, vol. 28(4), pages 744-763, October.
    15. Ji Liu, 2024. "Education legislations that equalize: a study of compulsory schooling law reforms in post-WWII United States," Palgrave Communications, Palgrave Macmillan, vol. 11(1), pages 1-12, December.
    16. Durlauf, Steven N. & Navarro, Salvador & Rivers, David A., 2016. "Model uncertainty and the effect of shall-issue right-to-carry laws on crime," European Economic Review, Elsevier, vol. 81(C), pages 32-67.
    17. Yusuke Narita, 2018. "Toward an Ethical Experiment," Cowles Foundation Discussion Papers 2127, Cowles Foundation for Research in Economics, Yale University.
    18. Xin Qiu & Donglin Zeng & Yuanjia Wang, 2018. "Estimation and evaluation of linear individualized treatment rules to guarantee performance," Biometrics, The International Biometric Society, vol. 74(2), pages 517-528, June.
    19. Ruoqing Zhu & Ying-Qi Zhao & Guanhua Chen & Shuangge Ma & Hongyu Zhao, 2017. "Greedy outcome weighted tree learning of optimal personalized treatment rules," Biometrics, The International Biometric Society, vol. 73(2), pages 391-400, June.
    20. Zeyu Bian & Erica E. M. Moodie & Susan M. Shortreed & Sahir Bhatnagar, 2023. "Variable selection in regression‐based estimation of dynamic treatment regimes," Biometrics, The International Biometric Society, vol. 79(2), pages 988-999, June.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:annopr:v:208:y:2013:i:1:p:383-416:10.1007/s10479-012-1248-5. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.