IDEAS home Printed from https://ideas.repec.org/a/spr/annopr/v208y2013i1p383-41610.1007-s10479-012-1248-5.html
   My bibliography  Save this article

Batch mode reinforcement learning based on the synthesis of artificial trajectories

Author

Listed:
  • Raphael Fonteneau
  • Susan Murphy
  • Louis Wehenkel
  • Damien Ernst

Abstract

In this paper, we consider the batch mode reinforcement learning setting, where the central problem is to learn from a sample of trajectories a policy that satisfies or optimizes a performance criterion. We focus on the continuous state space case for which usual resolution schemes rely on function approximators either to represent the underlying control problem or to represent its value function. As an alternative to the use of function approximators, we rely on the synthesis of “artificial trajectories” from the given sample of trajectories, and show that this idea opens new avenues for designing and analyzing algorithms for batch mode reinforcement learning. Copyright Springer Science+Business Media New York 2013

Suggested Citation

  • Raphael Fonteneau & Susan Murphy & Louis Wehenkel & Damien Ernst, 2013. "Batch mode reinforcement learning based on the synthesis of artificial trajectories," Annals of Operations Research, Springer, vol. 208(1), pages 383-416, September.
    • Handle: RePEc:spr:annopr:v:208:y:2013:i:1:p:383-416:10.1007/s10479-012-1248-5
    DOI: 10.1007/s10479-012-1248-5
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1007/s10479-012-1248-5
    Download Restriction: Access to full text is restricted to subscribers.
    File URL: https://libkey.io/10.1007/s10479-012-1248-5?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Murphy S.A. & van der Laan M.J. & Robins J.M., 2001. "Marginal Mean Models for Dynamic Regimes," Journal of the American Statistical Association, American Statistical Association, vol. 96, pages 1410-1423, December.
    2. S. A. Murphy, 2003. "Optimal dynamic treatment regimes," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 65(2), pages 331-355, May.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Ruitu Xu & Yifei Min & Tianhao Wang & Zhaoran Wang & Michael I. Jordan & Zhuoran Yang, 2023. "Finding Regularized Competitive Equilibria of Heterogeneous Agent Macroeconomic Models with Reinforcement Learning," Papers 2303.04833, arXiv.org.
    2. Shosei Sakaguchi, 2024. "Robust Learning for Optimal Dynamic Treatment Regimes with Observational Data," Papers 2404.00221, arXiv.org, revised Feb 2025.
    3. Stefano Bromuri, 2019. "Dynamic heuristic acceleration of linearly approximated SARSA( $$\lambda $$ λ ): using ant colony optimization to learn heuristics dynamically," Journal of Heuristics, Springer, vol. 25(6), pages 901-932, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jelena Bradic & Weijie Ji & Yuqian Zhang, 2021. "High-dimensional Inference for Dynamic Treatment Effects," Papers 2110.04924, arXiv.org, revised May 2023.
    2. Han, Sukjin, 2021. "Identification in nonparametric models for dynamic treatment effects," Journal of Econometrics, Elsevier, vol. 225(2), pages 132-147.
    3. Anders Bredahl Kock & Martin Thyrsgaard, 2017. "Optimal sequential treatment allocation," Papers 1705.09952, arXiv.org, revised Aug 2018.
    4. Yiwang Zhou & Peter X.K. Song & Haoda Fu, 2021. "Net benefit index: Assessing the influence of a biomarker for individualized treatment rules," Biometrics, The International Biometric Society, vol. 77(4), pages 1254-1264, December.
    5. Callaway, Brantly & Sant’Anna, Pedro H.C., 2021. "Difference-in-Differences with multiple time periods," Journal of Econometrics, Elsevier, vol. 225(2), pages 200-230.
    6. Alex Chin & Dean Eckles & Johan Ugander, 2022. "Evaluating Stochastic Seeding Strategies in Networks," Management Science, INFORMS, vol. 68(3), pages 1714-1736, March.
    7. Xinyu Tang & Abdus S. Wahed, 2011. "Comparison of treatment regimes with adjustment for auxiliary variables," Journal of Applied Statistics, Taylor & Francis Journals, vol. 38(12), pages 2925-2938, March.
    8. Hongming Pu & Bo Zhang, 2021. "Estimating optimal treatment rules with an instrumental variable: A partial identification learning approach," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 83(2), pages 318-345, April.
    9. Nina Zhou & Lu Wang & Daniel Almirall, 2023. "Estimating tree‐based dynamic treatment regimes using observational data with restricted treatment sequences," Biometrics, The International Biometric Society, vol. 79(3), pages 2260-2271, September.
    10. Yuqian Zhang & Weijie Ji & Jelena Bradic, 2021. "Dynamic treatment effects: high-dimensional inference under model misspecification," Papers 2111.06818, arXiv.org, revised Jan 2025.
    11. Ashesh Rambachan & Neil Shephard, 2019. "When do common time series estimands have nonparametric causal meaning?," Papers 1903.01637, arXiv.org, revised Jan 2025.
    12. Orellana Liliana & Rotnitzky Andrea & Robins James M., 2010. "Dynamic Regime Marginal Structural Mean Models for Estimation of Optimal Dynamic Treatment Regimes, Part I: Main Content," The International Journal of Biostatistics, De Gruyter, vol. 6(2), pages 1-49, March.
    13. Shuxiao Chen & Bo Zhang, 2021. "Estimating and Improving Dynamic Treatment Regimes With a Time-Varying Instrumental Variable," Papers 2104.07822, arXiv.org.
    14. Q. Clairon & R. Henderson & N. J. Young & E. D. Wilson & C. J. Taylor, 2021. "Adaptive treatment and robust control," Biometrics, The International Biometric Society, vol. 77(1), pages 223-236, March.
    15. Jin Wang & Donglin Zeng & D. Y. Lin, 2022. "Semiparametric single-index models for optimal treatment regimens with censored outcomes," Lifetime Data Analysis: An International Journal Devoted to Statistical Methods and Applications for Time-to-Event Data, Springer, vol. 28(4), pages 744-763, October.
    16. Shonosuke Sugasawa & Hisashi Noma, 2021. "Efficient screening of predictive biomarkers for individual treatment selection," Biometrics, The International Biometric Society, vol. 77(1), pages 249-257, March.
    17. Ji Liu, 2024. "Education legislations that equalize: a study of compulsory schooling law reforms in post-WWII United States," Palgrave Communications, Palgrave Macmillan, vol. 11(1), pages 1-12, December.
    18. Jingxiang Chen & Yufeng Liu & Donglin Zeng & Rui Song & Yingqi Zhao & Michael R. Kosorok, 2016. "Comment," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 111(515), pages 942-947, July.
    19. Durlauf, Steven N. & Navarro, Salvador & Rivers, David A., 2016. "Model uncertainty and the effect of shall-issue right-to-carry laws on crime," European Economic Review, Elsevier, vol. 81(C), pages 32-67.
    20. Kastoryano, Stephen, 2024. "Biological, Behavioural and Spurious Selection on the Kidney Transplant Waitlist," IZA Discussion Papers 16995, Institute of Labor Economics (IZA).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:annopr:v:208:y:2013:i:1:p:383-416:10.1007/s10479-012-1248-5. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.