IDEAS home Printed from https://ideas.repec.org/a/gam/jmathe/v10y2022i12p2073-d839482.html
   My bibliography  Save this article

Optimal Control with Partially Observed Regime Switching: Discounted and Average Payoffs

Author

Listed:
  • Beatris Adriana Escobedo-Trujillo

    (Facultad de Ingeniería Campus Coatzacoalcos, Universidad Veracruzana, Coatzacoalcos 96535, Veracruz, Mexico)

  • Javier Garrido-Meléndez

    (Facultad de Ingeniería Campus Coatzacoalcos, Universidad Veracruzana, Coatzacoalcos 96535, Veracruz, Mexico)

  • Gerardo Alcalá

    (Centro de Investigación en Recursos Energéticos y Sustentables, Universidad Veracruzana, Coatzacoalcos 96535, Veracruz, Mexico)

  • J. D. Revuelta-Acosta

    (Facultad de Ingeniería Campus Coatzacoalcos, Universidad Veracruzana, Coatzacoalcos 96535, Veracruz, Mexico)

Abstract

We consider an optimal control problem with the discounted and average payoff. The reward rate (or cost rate) can be unbounded from above and below, and a Markovian switching stochastic differential equation gives the state variable dynamic. Markovian switching is represented by a hidden continuous-time Markov chain that can only be observed in Gaussian white noise. Our general aim is to give conditions for the existence of optimal Markov stationary controls. This fact generalizes the conditions that ensure the existence of optimal control policies for optimal control problems completely observed. We use standard dynamic programming techniques and the method of hidden Markov model filtering to achieve our goals. As applications of our results, we study the discounted linear quadratic regulator (LQR) problem, the ergodic LQR problem for the modeled quarter-car suspension, the average LQR problem for the modeled quarter-car suspension with damp, and an explicit application for an optimal pollution control.

Suggested Citation

  • Beatris Adriana Escobedo-Trujillo & Javier Garrido-Meléndez & Gerardo Alcalá & J. D. Revuelta-Acosta, 2022. "Optimal Control with Partially Observed Regime Switching: Discounted and Average Payoffs," Mathematics, MDPI, vol. 10(12), pages 1-28, June.
    • Handle: RePEc:gam:jmathe:v:10:y:2022:i:12:p:2073-:d:839482
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2227-7390/10/12/2073/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2227-7390/10/12/2073/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Kawaguchi, Kazuhito & Morimoto, Hiroaki, 2007. "Long-run average welfare in a pollution accumulation model," Journal of Economic Dynamics and Control, Elsevier, vol. 31(2), pages 703-720, February.
    2. Xianggang Lu & G. Yin & Xianping Guo, 2017. "Infinite Horizon Controlled Diffusions with Randomly Varying and State-Dependent Discount Cost Rates," Journal of Optimization Theory and Applications, Springer, vol. 172(2), pages 535-553, February.
    3. Kunita, Hiroshi, 1971. "Asymptotic behavior of the nonlinear filtering errors of Markov processes," Journal of Multivariate Analysis, Elsevier, vol. 1(4), pages 365-393, December.
    4. Borkar, V. S., 2003. "Dynamic programming for ergodic control with partial observations," Stochastic Processes and their Applications, Elsevier, vol. 103(2), pages 293-310, February.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Francisco Germán Badía & María D. Berrade, 2023. "Special Issue “Probability Theory and Stochastic Modeling with Applications”," Mathematics, MDPI, vol. 11(14), pages 1-3, July.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ying Jiao, 2009. "Multiple defaults and contagion risks," Working Papers hal-00441500, HAL.
    2. Subhamay Saha, 2014. "Zero-Sum Stochastic Games with Partial Information and Average Payoff," Journal of Optimization Theory and Applications, Springer, vol. 160(1), pages 344-354, January.
    3. Moral, P. Del & Guionnet, A., 1998. "Large deviations for interacting particle systems: Applications to non-linear filtering," Stochastic Processes and their Applications, Elsevier, vol. 78(1), pages 69-95, October.
    4. Budhiraja, A., 2001. "Ergodic properties of the nonlinear filter," Stochastic Processes and their Applications, Elsevier, vol. 95(1), pages 1-24, September.
    5. Ying Jiao, 2009. "Multiple defaults and contagion risks," Papers 0912.3132, arXiv.org.
    6. Biswas, Anup & Budhiraja, Amarjit, 2011. "Exit time and invariant measure asymptotics for small noise constrained diffusions," Stochastic Processes and their Applications, Elsevier, vol. 121(5), pages 899-924.
    7. José López-Barrientos & Héctor Jasso-Fuentes & Beatris Escobedo-Trujillo, 2015. "Discounted robust control for Markov diffusion processes," TOP: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 23(1), pages 53-76, April.
    8. LeGland, François & Oudjane, Nadia, 2003. "A robustification approach to stability and to uniform particle approximation of nonlinear filters: the example of pseudo-mixing signals," Stochastic Processes and their Applications, Elsevier, vol. 106(2), pages 279-316, August.
    9. Borkar, V.S.Vivek S. & Budhiraja, Amarjit, 2004. "A further remark on dynamic programming for partially observed Markov processes," Stochastic Processes and their Applications, Elsevier, vol. 112(1), pages 79-93, July.
    10. Budhiraja, A. & Ocone, D., 1999. "Exponential stability in discrete-time filtering for non-ergodic signals," Stochastic Processes and their Applications, Elsevier, vol. 82(2), pages 245-257, August.
    11. Deck, T., 2006. "Asymptotic properties of Bayes estimators for Gaussian Itô-processes with noisy observations," Journal of Multivariate Analysis, Elsevier, vol. 97(2), pages 563-573, February.
    12. P. Del Moral & M. Ledoux, 2000. "Convergence of Empirical Processes for Interacting Particle Systems with Applications to Nonlinear Filtering," Journal of Theoretical Probability, Springer, vol. 13(1), pages 225-257, January.
    13. Anatoliy Swishchuk & Nikolaos Limnios, 2021. "Controlled Discrete-Time Semi-Markov Random Evolutions and Their Applications," Mathematics, MDPI, vol. 9(2), pages 1-26, January.
    14. Beatris Adriana Escobedo-Trujillo & José Daniel López-Barrientos & Carmen Geraldi Higuera-Chan & Francisco Alejandro Alaffita-Hernández, 2023. "Robust Statistic Estimation in Constrained Optimal Control Problems of Pollution Accumulation (Part II: Markovian Switchings)," Mathematics, MDPI, vol. 11(4), pages 1-22, February.
    15. Gerardo Hernandez-del-Valle & Carlos Pacheco-Gonzalez, 2009. "Optimal execution of Portfolio transactions with geometric price process," Papers 0908.1211, arXiv.org, revised Nov 2009.
    16. Papavasiliou, Anastasia, 2006. "Parameter estimation and asymptotic stability in stochastic filtering," Stochastic Processes and their Applications, Elsevier, vol. 116(7), pages 1048-1065, July.
    17. Beatris Adriana Escobedo-Trujillo & José Daniel López-Barrientos & Carmen Geraldi Higuera-Chan & Francisco Alejandro Alaffita-Hernández, 2023. "Robust Statistic Estimation of Constrained Optimal Control Problems of Pollution Accumulation (Part I)," Mathematics, MDPI, vol. 11(4), pages 1-19, February.
    18. Beatris Adriana Escobedo-Trujillo & José Daniel López-Barrientos & Javier Garrido-Meléndez, 2021. "A Constrained Markovian Diffusion Model for Controlling the Pollution Accumulation," Mathematics, MDPI, vol. 9(13), pages 1-29, June.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jmathe:v:10:y:2022:i:12:p:2073-:d:839482. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.