IDEAS home Printed from https://ideas.repec.org/a/inm/oropre/v62y2014i6p1394-1415.html
   My bibliography  Save this article

Information Relaxations, Duality, and Convex Stochastic Dynamic Programs

Author

Listed:
  • David B. Brown

    (Fuqua School of Business, Duke University, Durham, North Carolina 27708)

  • James E. Smith

    (Fuqua School of Business, Duke University, Durham, North Carolina 27708)

Abstract

We consider the information relaxation approach for calculating performance bounds for stochastic dynamic programs (DPs). This approach generates performance bounds by solving problems with relaxed nonanticipativity constraints and a penalty that punishes violations of these nonanticipativity constraints. In this paper, we study DPs that have a convex structure and consider gradient penalties that are based on first-order linear approximations of approximate value functions. When used with perfect information relaxations, these penalties lead to subproblems that are deterministic convex optimization problems. We show that these gradient penalties can, in theory, provide tight bounds for convex DPs and can be used to improve on bounds provided by other relaxations, such as Lagrangian relaxation bounds. Finally, we apply these results in two example applications: first, a network revenue management problem that describes an airline trying to manage seat capacity on its flights; and second, an inventory management problem with lead times and lost sales. These are challenging problems of significant practical interest. In both examples, we compute performance bounds using information relaxations with gradient penalties and find that some relatively easy-to-compute heuristic policies are nearly optimal.

Suggested Citation

  • David B. Brown & James E. Smith, 2014. "Information Relaxations, Duality, and Convex Stochastic Dynamic Programs," Operations Research, INFORMS, vol. 62(6), pages 1394-1415, December.
  • Handle: RePEc:inm:oropre:v:62:y:2014:i:6:p:1394-1415
    DOI: 10.1287/opre.2014.1322
    as

    Download full text from publisher

    File URL: http://dx.doi.org/10.1287/opre.2014.1322
    Download Restriction: no

    File URL: https://libkey.io/10.1287/opre.2014.1322?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Leif Andersen & Mark Broadie, 2004. "Primal-Dual Simulation Algorithm for Pricing Multidimensional American Options," Management Science, INFORMS, vol. 50(9), pages 1222-1234, September.
    2. Huseyin Topaloglu, 2009. "Using Lagrangian Relaxation to Compute Capacity-Dependent Bid Prices in Network Revenue Management," Operations Research, INFORMS, vol. 57(3), pages 637-649, June.
    3. David B. Brown & James E. Smith, 2011. "Dynamic Portfolio Optimization with Transaction Costs: Heuristics and Dual Bounds," Management Science, INFORMS, vol. 57(10), pages 1752-1770, October.
    4. Thomas E. Morton, 1971. "The Near-Myopic Nature of the Lagged-Proportional-Cost Inventory Problem with Lost Sales," Operations Research, INFORMS, vol. 19(7), pages 1708-1716, December.
    5. Daniel Adelman & Adam J. Mersereau, 2008. "Relaxations of Weakly Coupled Stochastic Dynamic Programs," Operations Research, INFORMS, vol. 56(3), pages 712-727, June.
    6. David B. Brown & James E. Smith & Peng Sun, 2010. "Information Relaxations and Duality in Stochastic Dynamic Programs," Operations Research, INFORMS, vol. 58(4-part-1), pages 785-801, August.
    7. Sripad K. Devalkar & Ravi Anupindi & Amitabh Sinha, 2011. "Integrated Optimization of Procurement, Processing, and Trade of Commodities," Operations Research, INFORMS, vol. 59(6), pages 1369-1381, December.
    8. Martin B. Haugh & Leonid Kogan, 2004. "Pricing American Options: A Duality Approach," Operations Research, INFORMS, vol. 52(2), pages 258-270, April.
    9. Shane G. Henderson & Peter W. Glynn, 2002. "Approximating Martingales for Variance Reduction in Markov Process Simulation," Mathematics of Operations Research, INFORMS, vol. 27(2), pages 253-271, May.
    10. L. C. G. Rogers, 2002. "Monte Carlo valuation of American options," Mathematical Finance, Wiley Blackwell, vol. 12(3), pages 271-286, July.
    11. Guoming Lai & François Margot & Nicola Secomandi, 2010. "An Approximate Dynamic Programming Approach to Benchmark Practice-Based Heuristics for Natural Gas Storage Valuation," Operations Research, INFORMS, vol. 58(3), pages 564-582, June.
    12. Sumit Kunnumkal & Kalyan Talluri, 2011. "Equivalence of piecewise-linear approximation and Lagrangian relaxation for network revenue management," Economics Working Papers 1305, Department of Economics and Business, Universitat Pompeu Fabra, revised Nov 2012.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Nicola Secomandi, 2015. "Merchant Commodity Storage Practice Revisited," Operations Research, INFORMS, vol. 63(5), pages 1131-1143, October.
    2. Anna Maria Gambaro & Nicola Secomandi, 2021. "A Discussion of Non‐Gaussian Price Processes for Energy and Commodity Operations," Production and Operations Management, Production and Operations Management Society, vol. 30(1), pages 47-67, January.
    3. repec:cte:wsrepe:ws1521 is not listed on IDEAS
    4. Santiago R. Balseiro & David B. Brown, 2019. "Approximations to Stochastic Dynamic Programs via Information Relaxation Duality," Operations Research, INFORMS, vol. 67(2), pages 577-597, March.
    5. Alberto Vera & Siddhartha Banerjee, 2021. "The Bayesian Prophet: A Low-Regret Framework for Online Decision Making," Management Science, INFORMS, vol. 67(3), pages 1368-1391, March.
    6. Hossein Jahandideh & Julie Ward Drew & Filippo Balestrieri & Kevin McCardle, 2020. "Individualized Pricing for a Cloud Provider Hosting Interactive Applications," Service Science, INFORMS, vol. 12(4), pages 130-147, December.
    7. Mark Broadie & Weiwei Shen, 2017. "Numerical solutions to dynamic portfolio problems with upper bounds," Computational Management Science, Springer, vol. 14(2), pages 215-227, April.
    8. Černý, Aleš & Melicherčík, Igor, 2020. "Simple explicit formula for near-optimal stochastic lifestyling," European Journal of Operational Research, Elsevier, vol. 284(2), pages 769-778.
    9. David B. Brown & James E. Smith, 2020. "Index Policies and Performance Bounds for Dynamic Selection Problems," Management Science, INFORMS, vol. 66(7), pages 3029-3050, July.
    10. David B. Brown & Martin B. Haugh, 2017. "Information Relaxation Bounds for Infinite Horizon Markov Decision Processes," Operations Research, INFORMS, vol. 65(5), pages 1355-1379, October.
    11. Mei, Xiaoling & Nogales, Francisco J., 2018. "Portfolio selection with proportional transaction costs and predictability," Journal of Banking & Finance, Elsevier, vol. 94(C), pages 131-151.
    12. Alessio Trivella & Danial Mohseni-Taheri & Selvaprabu Nadarajah, 2023. "Meeting Corporate Renewable Power Targets," Management Science, INFORMS, vol. 69(1), pages 491-512, January.
    13. Qihang Lin & Selvaprabu Nadarajah & Negar Soheili, 2020. "Revisiting Approximate Linear Programming: Constraint-Violation Learning with Applications to Inventory Control and Energy Storage," Management Science, INFORMS, vol. 66(4), pages 1544-1562, April.
    14. Yuhang Ma & Paat Rusmevichientong & Mika Sumida & Huseyin Topaloglu, 2020. "An Approximation Algorithm for Network Revenue Management Under Nonstationary Arrivals," Operations Research, INFORMS, vol. 68(3), pages 834-855, May.
    15. David A. Goldberg & Martin I. Reiman & Qiong Wang, 2021. "A Survey of Recent Progress in the Asymptotic Analysis of Inventory Systems," Production and Operations Management, Production and Operations Management Society, vol. 30(6), pages 1718-1750, June.
    16. Martin Haugh & Garud Iyengar & Chun Wang, 2016. "Tax-Aware Dynamic Asset Allocation," Operations Research, INFORMS, vol. 64(4), pages 849-866, August.
    17. Steven Kou & Xianhua Peng & Xingbo Xu, 2016. "EM Algorithm and Stochastic Control in Economics," Papers 1611.01767, arXiv.org.
    18. Daniel R. Jiang & Lina Al-Kanj & Warren B. Powell, 2020. "Optimistic Monte Carlo Tree Search with Sampled Information Relaxation Dual Bounds," Operations Research, INFORMS, vol. 68(6), pages 1678-1697, November.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. David B. Brown & Martin B. Haugh, 2017. "Information Relaxation Bounds for Infinite Horizon Markov Decision Processes," Operations Research, INFORMS, vol. 65(5), pages 1355-1379, October.
    2. David B. Brown & James E. Smith, 2013. "Optimal Sequential Exploration: Bandits, Clairvoyants, and Wildcats," Operations Research, INFORMS, vol. 61(3), pages 644-665, June.
    3. Santiago R. Balseiro & David B. Brown, 2019. "Approximations to Stochastic Dynamic Programs via Information Relaxation Duality," Operations Research, INFORMS, vol. 67(2), pages 577-597, March.
    4. Vijay V. Desai & Vivek F. Farias & Ciamac C. Moallemi, 2012. "Pathwise Optimization for Optimal Stopping Problems," Management Science, INFORMS, vol. 58(12), pages 2292-2308, December.
    5. Mark Broadie & Weiwei Shen, 2016. "High-Dimensional Portfolio Optimization With Transaction Costs," International Journal of Theoretical and Applied Finance (IJTAF), World Scientific Publishing Co. Pte. Ltd., vol. 19(04), pages 1-49, June.
    6. Dragos Florin Ciocan & Velibor V. Mišić, 2022. "Interpretable Optimal Stopping," Management Science, INFORMS, vol. 68(3), pages 1616-1638, March.
    7. Daniel R. Jiang & Lina Al-Kanj & Warren B. Powell, 2020. "Optimistic Monte Carlo Tree Search with Sampled Information Relaxation Dual Bounds," Operations Research, INFORMS, vol. 68(6), pages 1678-1697, November.
    8. David B. Brown & James E. Smith & Peng Sun, 2010. "Information Relaxations and Duality in Stochastic Dynamic Programs," Operations Research, INFORMS, vol. 58(4-part-1), pages 785-801, August.
    9. Alessio Trivella & Danial Mohseni-Taheri & Selvaprabu Nadarajah, 2023. "Meeting Corporate Renewable Power Targets," Management Science, INFORMS, vol. 69(1), pages 491-512, January.
    10. Christian Bender & Christian Gärtner & Nikolaus Schweizer, 2018. "Pathwise Dynamic Programming," Mathematics of Operations Research, INFORMS, vol. 43(3), pages 965-965, August.
    11. Secomandi, Nicola & Seppi, Duane J., 2014. "Real Options and Merchant Operations of Energy and Other Commodities," Foundations and Trends(R) in Technology, Information and Operations Management, now publishers, vol. 6(3-4), pages 161-331, July.
    12. Helin Zhu & Fan Ye & Enlu Zhou, 2013. "Fast Estimation of True Bounds on Bermudan Option Prices under Jump-diffusion Processes," Papers 1305.4321, arXiv.org.
    13. Christian Bender & Nikolaus Schweizer & Jia Zhuo, 2013. "A primal-dual algorithm for BSDEs," Papers 1310.3694, arXiv.org, revised Sep 2014.
    14. Guoming Lai & François Margot & Nicola Secomandi, 2010. "An Approximate Dynamic Programming Approach to Benchmark Practice-Based Heuristics for Natural Gas Storage Valuation," Operations Research, INFORMS, vol. 58(3), pages 564-582, June.
    15. Indrajit Mitra & Leonid Kogan, 2014. "Accuracy Verification for Numerical Solutions of Equilibrium Models," 2014 Meeting Papers 423, Society for Economic Dynamics.
    16. Helin Zhu & Fan Ye & Enlu Zhou, 2015. "Fast estimation of true bounds on Bermudan option prices under jump-diffusion processes," Quantitative Finance, Taylor & Francis Journals, vol. 15(11), pages 1885-1900, November.
    17. Leonid Kogan & Indrajit Mitra, 2021. "Near-Rational Equilibria in Heterogeneous-Agent Models: A Verification Method," FRB Atlanta Working Paper 2021-16, Federal Reserve Bank of Atlanta.
    18. Nadarajah, Selvaprabu & Margot, François & Secomandi, Nicola, 2017. "Comparison of least squares Monte Carlo methods with applications to energy real options," European Journal of Operational Research, Elsevier, vol. 256(1), pages 196-204.
    19. Christian Bender & Christian Gaertner & Nikolaus Schweizer, 2016. "Pathwise Iteration for Backward SDEs," Papers 1605.07500, arXiv.org, revised Jun 2016.
    20. Alessio Trivella & Selvaprabu Nadarajah & Stein-Erik Fleten & Denis Mazieres & David Pisinger, 2021. "Managing Shutdown Decisions in Merchant Commodity and Energy Production: A Social Commerce Perspective," Manufacturing & Service Operations Management, INFORMS, vol. 23(2), pages 311-330, March.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:oropre:v:62:y:2014:i:6:p:1394-1415. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.