IDEAS home Printed from https://ideas.repec.org/a/eee/jomega/v47y2014icp116-126.html
   My bibliography  Save this article

Real-time dynamic pricing in a non-stationary environment using model-free reinforcement learning

Author

Listed:
  • Rana, Rupal
  • Oliveira, Fernando S.

Abstract

This paper examines the problem of establishing a pricing policy that maximizes the revenue for selling a given inventory by a fixed deadline. This problem is faced by a variety of industries, including airlines, hotels and fashion. Reinforcement learning algorithms are used to analyze how firms can both learn and optimize their pricing strategies while interacting with their customers. We show that by using reinforcement learning we can model the problem with inter-dependent demands. This type of model can be useful in producing a more accurate pricing scheme of services or products when important events affect consumer preferences. This paper proposes a methodology to optimize revenue in a model-free environment in which demand is learned and pricing decisions are updated in real-time. We compare the performance of the learning algorithms using Monte-Carlo simulation.

Suggested Citation

  • Rana, Rupal & Oliveira, Fernando S., 2014. "Real-time dynamic pricing in a non-stationary environment using model-free reinforcement learning," Omega, Elsevier, vol. 47(C), pages 116-126.
  • Handle: RePEc:eee:jomega:v:47:y:2014:i:c:p:116-126
    DOI: 10.1016/j.omega.2013.10.004
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S030504831300100X
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.omega.2013.10.004?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Chenavaz, Régis, 2012. "Dynamic pricing, product and process innovation," European Journal of Operational Research, Elsevier, vol. 222(3), pages 553-557.
    2. C. Raju & Y. Narahari & K. Ravikumar, 2006. "Learning dynamic prices in electronic retail markets with customer segmentation," Annals of Operations Research, Springer, vol. 143(1), pages 59-75, March.
    3. Andrew E. B. Lim & J. George Shanthikumar, 2007. "Relative Entropy, Exponential Utility, and Robust Dynamic Pricing," Operations Research, INFORMS, vol. 55(2), pages 198-214, April.
    4. Guillermo Gallego & Garrett van Ryzin, 1994. "Optimal Dynamic Pricing of Inventories with Stochastic Demand over Finite Horizons," Management Science, INFORMS, vol. 40(8), pages 999-1020, August.
    5. Anjos, Miguel F. & Cheng, Russell C. H. & Currie, Christine S. M., 2005. "Optimal pricing policies for perishable products," European Journal of Operational Research, Elsevier, vol. 166(1), pages 246-254, October.
    6. Dasu, Sriram & Tong, Chunyang, 2010. "Dynamic pricing when consumers are strategic: Analysis of posted and contingent pricing schemes," European Journal of Operational Research, Elsevier, vol. 204(3), pages 662-671, August.
    7. Banerjee, Pradeep K. & Turner, T. Rolf, 2012. "A flexible model for the pricing of perishable assets," Omega, Elsevier, vol. 40(5), pages 533-540.
    8. Gabriel Bitran & René Caldentey, 2003. "An Overview of Pricing Models for Revenue Management," Manufacturing & Service Operations Management, INFORMS, vol. 5(3), pages 203-229, August.
    9. Grigoriev, A. & Hiller, B. & Marban, S. & Vredeveld, T. & van der Zwaan, G.R.J., 2010. "Dynamic pricing problems with elastic demand," Research Memorandum 053, Maastricht University, Maastricht Research School of Economics of Technology and Organization (METEOR).
    10. M F Anjos & R C H Cheng & C S M Currie, 2004. "Maximizing revenue in the airline industry under one-way pricing," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 55(5), pages 535-541, May.
    11. Yossi Aviv & Amit Pazgal, 2005. "A Partially Observed Markov Decision Process for Dynamic Pricing," Management Science, INFORMS, vol. 51(9), pages 1400-1416, September.
    12. Berk, Emre & Gürler, Ülkü & YIldIrIm, Gonca, 2009. "On pricing of perishable assets with menu costs," International Journal of Production Economics, Elsevier, vol. 121(2), pages 678-699, October.
    13. Tsai, Wen-Hsien & Hung, Shih-Jieh, 2009. "Dynamic pricing and revenue management process in Internet retailing under uncertainty: An integrated real options approach," Omega, Elsevier, vol. 37(2), pages 471-481, April.
    14. Lin, Kyle Y., 2006. "Dynamic pricing with real-time demand learning," European Journal of Operational Research, Elsevier, vol. 174(1), pages 522-538, October.
    15. Ramasesh, Ranga V., 2010. "Lot-sizing decisions under limited-time price incentives: A review," Omega, Elsevier, vol. 38(3-4), pages 118-135, June.
    16. Pan, Kewen & Lai, K.K. & Liang, L. & Leung, Stephen C.H., 2009. "Two-period pricing and ordering policy for the dominant retailer in a two-echelon supply chain with demand uncertainty," Omega, Elsevier, vol. 37(4), pages 919-929, August.
    17. Zhao, Li & Tian, Peng & Xiangyong Li, 2012. "Dynamic pricing in the presence of consumer inertia," Omega, Elsevier, vol. 40(2), pages 137-148, April.
    18. Wedad Elmaghraby & P{i}nar Keskinocak, 2003. "Dynamic Pricing in the Presence of Inventory Considerations: Research Overview, Current Practices, and Future Directions," Management Science, INFORMS, vol. 49(10), pages 1287-1309, October.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Huijie Peng & Yan Cheng & Xingyuan Li, 2023. "Real-Time Pricing Method for Spot Cloud Services with Non-Stationary Excess Capacity," Sustainability, MDPI, vol. 15(4), pages 1-21, February.
    2. Alexander Kastius & Rainer Schlosser, 2022. "Dynamic pricing under competition using reinforcement learning," Journal of Revenue and Pricing Management, Palgrave Macmillan, vol. 21(1), pages 50-63, February.
    3. den Boer, Arnoud V., 2015. "Tracking the market: Dynamic pricing and learning in a changing environment," European Journal of Operational Research, Elsevier, vol. 247(3), pages 914-927.
    4. Jian Wang & Murtaza Das & Stephen Tappert, 2021. "Applying reinforcement learning to estimating apartment reference rents," Journal of Revenue and Pricing Management, Palgrave Macmillan, vol. 20(3), pages 330-343, June.
    5. Lu, Renzhi & Hong, Seung Ho & Zhang, Xiongfeng, 2018. "A Dynamic pricing demand response algorithm for smart grid: Reinforcement learning approach," Applied Energy, Elsevier, vol. 220(C), pages 220-230.
    6. Yi Zheng & Zehao Li & Peng Jiang & Yijie Peng, 2024. "Dual-Agent Deep Reinforcement Learning for Dynamic Pricing and Replenishment," Papers 2410.21109, arXiv.org.
    7. Bajwa, Naeem & Sox, Charles R. & Ishfaq, Rafay, 2016. "Coordinating pricing and production decisions for multiple products," Omega, Elsevier, vol. 64(C), pages 86-101.
    8. Yan, Yimo & Chow, Andy H.F. & Ho, Chin Pang & Kuo, Yong-Hong & Wu, Qihao & Ying, Chengshuo, 2022. "Reinforcement learning for logistics and supply chain management: Methodologies, state of the art, and future opportunities," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 162(C).
    9. Justin Dumouchelle & Emma Frejinger & Andrea Lodi, 2024. "Reinforcement learning for freight booking control problems," Journal of Revenue and Pricing Management, Palgrave Macmillan, vol. 23(4), pages 318-345, August.
    10. Chen, Jing & Dong, Ming & Rong, Ying & Yang, Liang, 2018. "Dynamic pricing for deteriorating products with menu cost," Omega, Elsevier, vol. 75(C), pages 13-26.
    11. Yilin Liang & Yuping Hu & Dongjun Luo & Qi Zhu & Qingxuan Chen & Chunmei Wang, 2023. "Distributed Dynamic Pricing Strategy Based on Deep Reinforcement Learning Approach in a Presale Mechanism," Sustainability, MDPI, vol. 15(13), pages 1-20, July.
    12. Yu Xia & Ali Arian & Sriram Narayanamoorthy & Joshua Mabry, 2023. "RetailSynth: Synthetic Data Generation for Retail AI Systems Evaluation," Papers 2312.14095, arXiv.org.
    13. Kazemi, Mohammad Sadegh & Fotopoulos, Stergios B. & Wang, Xinchang, 2023. "Minimizing online retailers’ revenue loss under a time-varying willingness-to-pay distribution," International Journal of Production Economics, Elsevier, vol. 257(C).
    14. Klein, Robert & Kolb, Johannes, 2015. "Maximizing customer equity subject to capacity constraints," Omega, Elsevier, vol. 55(C), pages 111-125.
    15. Raad Khraishi & Ramin Okhrati, 2022. "Offline Deep Reinforcement Learning for Dynamic Pricing of Consumer Credit," Papers 2203.03003, arXiv.org.
    16. Basu, Sumanta & Chakraborty, Soumyakanti & Sharma, Megha, 2015. "Pricing cloud services—the impact of broadband quality," Omega, Elsevier, vol. 50(C), pages 96-114.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. C S M Currie & R C H Cheng & H K Smith, 2008. "Dynamic pricing of airline tickets with competition," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 59(8), pages 1026-1037, August.
    2. Tatsiana Levina & Yuri Levin & Jeff McGill & Mikhail Nediak, 2009. "Dynamic Pricing with Online Learning and Strategic Consumers: An Application of the Aggregating Algorithm," Operations Research, INFORMS, vol. 57(2), pages 327-341, April.
    3. Ming Chen & Zhi-Long Chen, 2018. "Robust Dynamic Pricing with Two Substitutable Products," Manufacturing & Service Operations Management, INFORMS, vol. 20(2), pages 249-268, May.
    4. Athanassios N. Avramidis & Arnoud V. Boer, 2021. "Dynamic pricing with finite price sets: a non-parametric approach," Mathematical Methods of Operations Research, Springer;Gesellschaft für Operations Research (GOR);Nederlands Genootschap voor Besliskunde (NGB), vol. 94(1), pages 1-34, August.
    5. Hou, Lihua & Nie, Tengfei & Zhang, Jianghua, 2024. "Pricing and inventory strategies for perishable products in a competitive market considering strategic consumers," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 184(C).
    6. Omar Besbes & Assaf Zeevi, 2012. "Blind Network Revenue Management," Operations Research, INFORMS, vol. 60(6), pages 1537-1550, December.
    7. Kuo, Chia-Wei & Huang, Kwei-Long, 2012. "Dynamic pricing of limited inventories for multi-generation products," European Journal of Operational Research, Elsevier, vol. 217(2), pages 394-403.
    8. Wang, Xiaojun & Li, Dong, 2012. "A dynamic product quality evaluation based pricing model for perishable food supply chains," Omega, Elsevier, vol. 40(6), pages 906-917.
    9. Doan, Xuan Vinh & Lei, Xiao & Shen, Siqian, 2020. "Pricing of reusable resources under ambiguous distributions of demand and service time with emerging applications," European Journal of Operational Research, Elsevier, vol. 282(1), pages 235-251.
    10. Gökgür, Burak & Karabatı, Selçuk, 2019. "Dynamic and targeted bundle pricing of two independently valued products," European Journal of Operational Research, Elsevier, vol. 279(1), pages 184-198.
    11. Ayşe Kocabıyıkoğlu & Ioana Popescu & Catalina Stefanescu, 2014. "Pricing and Revenue Management: The Value of Coordination," Management Science, INFORMS, vol. 60(3), pages 730-752, March.
    12. Zizhuo Wang & Shiming Deng & Yinyu Ye, 2014. "Close the Gaps: A Learning-While-Doing Algorithm for Single-Product Revenue Management Problems," Operations Research, INFORMS, vol. 62(2), pages 318-331, April.
    13. Goker Aydin & Serhan Ziya, 2009. "Technical Note---Personalized Dynamic Pricing of Limited Inventories," Operations Research, INFORMS, vol. 57(6), pages 1523-1531, December.
    14. Yuri Levin & Jeff McGill & Mikhail Nediak, 2008. "Risk in Revenue Management and Dynamic Pricing," Operations Research, INFORMS, vol. 56(2), pages 326-343, April.
    15. Goker Aydin & Serhan Ziya, 2008. "Pricing Promotional Products Under Upselling," Manufacturing & Service Operations Management, INFORMS, vol. 10(3), pages 360-376, June.
    16. Omar Besbes & Assaf Zeevi, 2009. "Dynamic Pricing Without Knowing the Demand Function: Risk Bounds and Near-Optimal Algorithms," Operations Research, INFORMS, vol. 57(6), pages 1407-1420, December.
    17. René Caldentey & Ying Liu & Ilan Lobel, 2017. "Intertemporal Pricing Under Minimax Regret," Operations Research, INFORMS, vol. 65(1), pages 104-129, February.
    18. Serguei Netessine & Sergei Savin & Wenqiang Xiao, 2006. "Revenue Management Through Dynamic Cross Selling in E-Commerce Retailing," Operations Research, INFORMS, vol. 54(5), pages 893-913, October.
    19. Ibrahim, Michael Nawar & Atiya, Amir F., 2016. "Analytical solutions to the dynamic pricing problem for time-normalized revenue," European Journal of Operational Research, Elsevier, vol. 254(2), pages 632-643.
    20. Pırıl Tekin & Rızvan Erol, 2017. "A New Dynamic Pricing Model for the Effective Sustainability of Perishable Product Life Cycle," Sustainability, MDPI, vol. 9(8), pages 1-22, July.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:jomega:v:47:y:2014:i:c:p:116-126. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/375/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.