IDEAS home Printed from https://ideas.repec.org/p/cte/wsrepe/9865.html
   My bibliography  Save this paper

Pareto optimality in multiobjective Markov control processes

Author

Listed:
  • Hernández-Lerma, Onésimo
  • Romera, Rosario

Abstract

This paper studies discrete-time multiobjective Markov control processes (MCPs) on Borel spaces and with unbounded costs. Under mild assumptions, it shows the existence of Pareto optimal control policies, which are also characterized as optimal policies for a certain class of single-objective ( or "scalar") MCPs. A similar result is obtained for strong Pareto optimal policies, which are Pareto optimal policies whose cost vector is the closest, in the Euclidean norm, to the virtual minimum. To obtain these results, the basic idea is to transform the multiobjective MCP into an equivalent multiobjective measure problem (MMP). In addition, MMP is restated as a primal multiobjective linear program and it is shown that solving the scalarized MCPs is in fact the same as solving the dual of MMP. A multiobjective LQ example illustrates the main results.

Suggested Citation

  • Hernández-Lerma, Onésimo & Romera, Rosario, 2000. "Pareto optimality in multiobjective Markov control processes," DES - Working Papers. Statistics and Econometrics. WS 9865, Universidad Carlos III de Madrid. Departamento de Estadística.
    • Handle: RePEc:cte:wsrepe:9865
    as

    Download full text from publisher

    File URL: https://e-archivo.uc3m.es/rest/api/core/bitstreams/4363d5bd-48a9-4bbf-8a7a-37be0d4c3e0c/content
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Eugene A. Feinberg & Adam Shwartz, 1996. "Constrained Discounted Dynamic Programming," Mathematics of Operations Research, INFORMS, vol. 21(4), pages 922-945, November.
    2. Mordechai I. Henig, 1985. "The Principle of Optimality in Dynamic Programming with Returns in Partially Ordered Sets," Mathematics of Operations Research, INFORMS, vol. 10(3), pages 462-470, August.
    3. Dmitry Krass & Jerzy A. Filar & Sagnik S. Sinha, 1992. "A Weighted Markov Decision Process," Operations Research, INFORMS, vol. 40(6), pages 1180-1187, December.
    4. Wakuta, Kazuyoshi, 1992. "Optimal stationary policies in the vector-valued Markov decision process," Stochastic Processes and their Applications, Elsevier, vol. 42(1), pages 149-156, August.
    5. Onésimo Hernández-Lerma & Juan González-Hernández, 2000. "Constrained Markov control processes in Borel spaces: the discounted case," Mathematical Methods of Operations Research, Springer;Gesellschaft für Operations Research (GOR);Nederlands Genootschap voor Besliskunde (NGB), vol. 52(2), pages 271-285, November.
    6. Balbas, Alejandro & Heras, Antonio, 1993. "Duality theory for infinite-dimensional multiobjective linear programming," European Journal of Operational Research, Elsevier, vol. 68(3), pages 379-388, August.
    7. Wakuta, Kazuyoshi, 1995. "Vector-valued Markov decision processes and the systems of linear inequalities," Stochastic Processes and their Applications, Elsevier, vol. 56(1), pages 159-169, March.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Krishnamurthy Iyer & Nandyala Hemachandra, 2010. "Sensitivity analysis and optimal ultimately stationary deterministic policies in some constrained discounted cost models," Mathematical Methods of Operations Research, Springer;Gesellschaft für Operations Research (GOR);Nederlands Genootschap voor Besliskunde (NGB), vol. 71(3), pages 401-425, June.
    2. Eugene A. Feinberg & Uriel G. Rothblum, 2012. "Splitting Randomized Stationary Policies in Total-Reward Markov Decision Processes," Mathematics of Operations Research, INFORMS, vol. 37(1), pages 129-153, February.
    3. Truman F. Bewley, 1987. "Knightian Decision Theory, Part II. Intertemporal Problems," Cowles Foundation Discussion Papers 835, Cowles Foundation for Research in Economics, Yale University.
    4. Nandyala Hemachandra & Kamma Sri Naga Rajesh & Mohd. Abdul Qavi, 2016. "A model for equilibrium in some service-provider user-set interactions," Annals of Operations Research, Springer, vol. 243(1), pages 95-115, August.
    5. Vladimir Ejov & Jerzy A. Filar & Michael Haythorpe & Giang T. Nguyen, 2009. "Refined MDP-Based Branch-and-Fix Algorithm for the Hamiltonian Cycle Problem," Mathematics of Operations Research, INFORMS, vol. 34(3), pages 758-768, August.
    6. Ohlmann, Jeffrey W. & Bean, James C., 2009. "Resource-constrained management of heterogeneous assets with stochastic deterioration," European Journal of Operational Research, Elsevier, vol. 199(1), pages 198-208, November.
    7. Wakuta, Kazuyoshi, 1995. "Vector-valued Markov decision processes and the systems of linear inequalities," Stochastic Processes and their Applications, Elsevier, vol. 56(1), pages 159-169, March.
    8. Takashi Kamihigashi, 2008. "On the principle of optimality for nonstationary deterministic dynamic programming," International Journal of Economic Theory, The International Society for Economic Theory, vol. 4(4), pages 519-525, December.
    9. Kumar, Uday M & Bhat, Sanjay P. & Kavitha, Veeraruna & Hemachandra, Nandyala, 2023. "Approximate solutions to constrained risk-sensitive Markov decision processes," European Journal of Operational Research, Elsevier, vol. 310(1), pages 249-267.
    10. Eugene A. Feinberg, 2000. "Constrained Discounted Markov Decision Processes and Hamiltonian Cycles," Mathematics of Operations Research, INFORMS, vol. 25(1), pages 130-140, February.
    11. Maciej Nowak & Tadeusz Trzaskalik, 2013. "Interactive procedure for a multiobjective stochastic discrete dynamic problem," Journal of Global Optimization, Springer, vol. 57(2), pages 315-330, October.
    12. Jorge Alvarez-Mena & Onésimo Hernández-Lerma, 2006. "Existence of nash equilibria for constrained stochastic games," Mathematical Methods of Operations Research, Springer;Gesellschaft für Operations Research (GOR);Nederlands Genootschap voor Besliskunde (NGB), vol. 63(2), pages 261-285, May.
    13. N. Mahdavi-Amiri & F. Salehi Sadaghiani, 2017. "Strictly feasible solutions and strict complementarity in multiple objective linear optimization," 4OR, Springer, vol. 15(3), pages 303-326, September.
    14. Xianping Guo & Yi Zhang, 2016. "Optimality of Mixed Policies for Average Continuous-Time Markov Decision Processes with Constraints," Mathematics of Operations Research, INFORMS, vol. 41(4), pages 1276-1296, November.
    15. Juan González-Hernández & César Villarreal, 2011. "Optimal policies for constrained average-cost Markov decision processes," TOP: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 19(1), pages 107-120, July.
    16. Richard Chen & Eugene Feinberg, 2010. "Compactness of the space of non-randomized policies in countable-state sequential decision processes," Mathematical Methods of Operations Research, Springer;Gesellschaft für Operations Research (GOR);Nederlands Genootschap voor Besliskunde (NGB), vol. 71(2), pages 307-323, April.
    17. Trzaskalik, Tadeusz & Sitarz, Sebastian, 2007. "Discrete dynamic programming with outcomes in random variable structures," European Journal of Operational Research, Elsevier, vol. 177(3), pages 1535-1548, March.
    18. E. Galperin & P. Jimenez Guerra, 2001. "Duality of Nonscalarized Multiobjective Linear Programs: Dual Balance, Level Sets, and Dual Clusters of Optimal Vectors," Journal of Optimization Theory and Applications, Springer, vol. 108(1), pages 109-137, January.
    19. Luc, Dinh The, 2011. "On duality in multiple objective linear programming," European Journal of Operational Research, Elsevier, vol. 210(2), pages 158-168, April.
    20. Balbás, Alejandro & Balbás, Raquel & Mayoral, Silvia, 2009. "Portfolio choice and optimal hedging with general risk functions: A simplex-like algorithm," European Journal of Operational Research, Elsevier, vol. 192(2), pages 603-620, January.

    More about this item

    Keywords

    Pareto optimality;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:cte:wsrepe:9865. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Ana Poveda (email available below). General contact details of provider: http://portal.uc3m.es/portal/page/portal/dpto_estadistica .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.