IDEAS home Printed from https://ideas.repec.org/a/eee/transe/v187y2024ics1366554524001674.html
   My bibliography  Save this article

A hybrid deep reinforcement learning approach for a proactive transshipment of fresh food in the online–offline channel system

Author

Listed:
  • Lee, Junhyeok
  • Shin, Youngchul
  • Moon, Ilkyeong

Abstract

To reduce the waste of fresh foods, one of the e-commerce companies in South Korea utilizes lateral transshipment in the network of online platforms and offline shops, which is called the online–offline channel system (OOCS). Even though the OOCS has achieved success in real practice, there is room for further study on this system with regard to deriving a transshipment policy. For this reason, this study aims to develop a solution approach that could derive a promising policy and analyze the impacts of transshipment in the OOCS. The main contributions are summarized as follows. First, we propose a model to deal with the proactive transshipment of perishable products in the OOCS. In particular, this is the first study that introduces the concept of the heterogeneous shelf life considering different properties of online and offline channels. Second, we develop the hybrid deep reinforcement learning (DRL) approach by combining the soft actor–critic algorithm with two novel acceleration methods. The developed method could obtain a promising policy without assumptions about demand distribution and mitigate computational burdens by reducing action spaces. On a set of experiments carried out on real-world demand data, the transshipment policy derived from the hybrid DRL approach could obtain the best profit compared to existing algorithms. Third, we examine the impacts of transshipment by differing types of demand and varying the unit transshipment cost parameter. We find that transshipment substantially reduces the outdating cost by allowing the offline channel to make good use of the old products that will be discarded in the online channel, which is new to the literature.

Suggested Citation

  • Lee, Junhyeok & Shin, Youngchul & Moon, Ilkyeong, 2024. "A hybrid deep reinforcement learning approach for a proactive transshipment of fresh food in the online–offline channel system," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 187(C).
  • Handle: RePEc:eee:transe:v:187:y:2024:i:c:s1366554524001674
    DOI: 10.1016/j.tre.2024.103576
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S1366554524001674
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.tre.2024.103576?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Li, Guiping & He, Xiuli & Zhou, Jing & Wu, Hao, 2019. "Pricing, replenishment and preservation technology investment decisions for non-instantaneous deteriorating items," Omega, Elsevier, vol. 84(C), pages 114-126.
    2. De Moor, Bram J. & Gijsbrechts, Joren & Boute, Robert N., 2022. "Reward shaping to improve the performance of deep reinforcement learning in perishable inventory management," European Journal of Operational Research, Elsevier, vol. 301(2), pages 535-545.
    3. Wang, Ke-Ming & Ma, Zu-Jun, 2015. "Age-based policy for blood transshipment during blood shortage," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 80(C), pages 166-183.
    4. Hossein Abouee-Mehrizi & Oded Berman & Shrutivandana Sharma, 2015. "Optimal Joint Replenishment and Transshipment Policies in a Multi-Period Inventory System with Lost Sales," Operations Research, INFORMS, vol. 63(2), pages 342-350, April.
    5. Dilupa Nakandala & Henry Lau & Paul K.C. Shum, 2017. "A lateral transshipment model for perishable inventory management," International Journal of Production Research, Taylor & Francis Journals, vol. 55(18), pages 5341-5354, September.
    6. Dehghani, Maryam & Abbasi, Babak & Oliveira, Fabricio, 2021. "Proactive transshipment in the blood supply chain: A stochastic programming approach," Omega, Elsevier, vol. 98(C).
    7. Abhijit Gosavi, 2009. "Reinforcement Learning: A Tutorial Survey and Recent Advances," INFORMS Journal on Computing, INFORMS, vol. 21(2), pages 178-192, May.
    8. Dehghani, Maryam & Abbasi, Babak, 2018. "An age-based lateral-transshipment policy for perishable items," International Journal of Production Economics, Elsevier, vol. 198(C), pages 93-103.
    9. Meissner, Joern & Senicheva, Olga V., 2018. "Approximate dynamic programming for lateral transshipment problems in multi-location inventory systems," European Journal of Operational Research, Elsevier, vol. 265(1), pages 49-64.
    10. Zhong-Zhong Jiang & Guangwen Kong & Yinghao Zhang, 2021. "Making the Most of Your Regret: Workers’ Relocation Decisions in On-Demand Platforms," Manufacturing & Service Operations Management, INFORMS, vol. 23(3), pages 695-713, May.
    11. Paterson, Colin & Kiesmüller, Gudrun & Teunter, Ruud & Glazebrook, Kevin, 2011. "Inventory models with lateral transshipments: A review," European Journal of Operational Research, Elsevier, vol. 210(2), pages 125-136, April.
    12. Kevin Glazebrook & Colin Paterson & Sandra Rauscher & Thomas Archibald, 2015. "Benefits of Hybrid Lateral Transshipments in Multi-Item Inventory Systems under Periodic Replenishment," Production and Operations Management, Production and Operations Management Society, vol. 24(2), pages 311-324, February.
    13. Haijema, René & Minner, Stefan, 2019. "Improved ordering of perishables: The value of stock-age information," International Journal of Production Economics, Elsevier, vol. 209(C), pages 316-324.
    14. Haijema, René & Minner, Stefan, 2016. "Stock-level dependent ordering of perishables: A comparison of hybrid base-stock and constant order policies," International Journal of Production Economics, Elsevier, vol. 181(PA), pages 215-225.
    15. Li, Ying & Liao, Yi & Hu, Xinxin & Shen, Wenjing, 2020. "Lateral transshipment with partial request and random switching," Omega, Elsevier, vol. 92(C).
    16. Firouz, Mohammad & Keskin, Burcu B. & Melouk, Sharif H., 2017. "An integrated supplier selection and inventory problem with multi-sourcing and lateral transshipments," Omega, Elsevier, vol. 70(C), pages 77-93.
    17. Ziteng Wang & Yue Dai & Shu‐Cherng Fang & Zhong‐Zhong Jiang & Yifan Xu, 2020. "Inventory transshipment game with limited supply: Trap or treat," Naval Research Logistics (NRL), John Wiley & Sons, vol. 67(6), pages 383-403, September.
    18. Sun, Wenjing & Zou, Yuan & Zhang, Xudong & Guo, Ningyuan & Zhang, Bin & Du, Guodong, 2022. "High robustness energy management strategy of hybrid electric vehicle based on improved soft actor-critic deep reinforcement learning," Energy, Elsevier, vol. 258(C).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Dehghani, Maryam & Abbasi, Babak & Oliveira, Fabricio, 2021. "Proactive transshipment in the blood supply chain: A stochastic programming approach," Omega, Elsevier, vol. 98(C).
    2. Dhahri, Akrem & Gharbi, Ali & Ouhimmou, Mustapha, 2022. "Integrated production-transshipment control policy for a two-location unreliable manufacturing system," International Journal of Production Economics, Elsevier, vol. 247(C).
    3. Naderi, Siamak & Kilic, Kemal & Dasci, Abdullah, 2020. "A deterministic model for the transshipment problem of a fast fashion retailer under capacity constraints," International Journal of Production Economics, Elsevier, vol. 227(C).
    4. Dijkstra, Arjan S. & Van der Heide, Gerlach & Roodbergen, Kees Jan, 2019. "Transshipments of cross-channel returned products," International Journal of Production Economics, Elsevier, vol. 209(C), pages 70-77.
    5. Kamyabniya, Afshin & Noormohammadzadeh, Zohre & Sauré, Antoine & Patrick, Jonathan, 2021. "A robust integrated logistics model for age-based multi-group platelets in disaster relief operations," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 152(C).
    6. Wan, Xiang & Gligor, David & Fan, Xuemei & Qi, Yinan & Britto, Rodrigo, 2024. "The value of timing, frequency, and quantity: The effects of transshipments on inventory turnover and order fulfillment," International Journal of Production Economics, Elsevier, vol. 274(C).
    7. Christian Weckenborg & Bastian Vorwerk & Thomas S. Spengler, 2024. "A proactive transshipment model for prototype parts logistics in the automotive industry," Journal of Business Economics, Springer, vol. 94(9), pages 1147-1168, November.
    8. LI, Li, 2019. "Cooperative purchasing and preactive inventory sharing – Channel balancing and performance improvement," European Journal of Operational Research, Elsevier, vol. 278(3), pages 738-751.
    9. Qingren He & Taiwei Shi & Botao Liu & Wanhua Qiu, 2022. "The Ordering Optimization Model for Bounded Rational Retailer with Inventory Transshipment," Mathematics, MDPI, vol. 10(7), pages 1-16, March.
    10. Bhatnagar, Rohit & Lin, Bing, 2019. "The joint transshipment and production control policies for multi-location production/inventory systems," European Journal of Operational Research, Elsevier, vol. 275(3), pages 957-970.
    11. Zhou, Zihan & Wang, Xinhui, 2023. "Replenishment and transshipment in periodic-review systems with a fixed order cost," European Journal of Operational Research, Elsevier, vol. 307(3), pages 1240-1247.
    12. Griffin, Emily C. & Keskin, Burcu B. & Allaway, Arthur W., 2023. "Clustering retail stores for inventory transshipment," European Journal of Operational Research, Elsevier, vol. 311(2), pages 690-707.
    13. Cavagnini, Rossana & Bertazzi, Luca & Maggioni, Francesca, 2022. "A rolling horizon approach for a multi-stage stochastic fixed-charge transportation problem with transshipment," European Journal of Operational Research, Elsevier, vol. 301(3), pages 912-922.
    14. Dehghani, Maryam & Abbasi, Babak, 2018. "An age-based lateral-transshipment policy for perishable items," International Journal of Production Economics, Elsevier, vol. 198(C), pages 93-103.
    15. Yongchang Wei & Fangyu Chen & Feng Xiong, 2018. "Dynamic Complexities in a Supply Chain System with Lateral Transshipments," Complexity, Hindawi, vol. 2018, pages 1-15, June.
    16. Vanvuchelen, Nathalie & De Boeck, Kim & Boute, Robert N., 2024. "Cluster-based lateral transshipments for the Zambian health supply chain," European Journal of Operational Research, Elsevier, vol. 313(1), pages 373-386.
    17. Gerrits, B. & Topan, E. & van der Heijden, M.C., 2022. "Operational planning in service control towers – heuristics and case study," European Journal of Operational Research, Elsevier, vol. 302(3), pages 983-998.
    18. van Wijk, A.C.C. & Adan, I.J.B.F. & van Houtum, G.J., 2019. "Optimal lateral transshipment policies for a two location inventory problem with multiple demand classes," European Journal of Operational Research, Elsevier, vol. 272(2), pages 481-495.
    19. Brandimarte, Paolo & Craparotta, Giuseppe & Marocco, Elena, 2024. "Inventory reallocation in a fashion retail network: A matheuristic approach," European Journal of Operational Research, Elsevier, vol. 317(2), pages 603-615.
    20. Zhang, Yi & Hua, Guowei & Cheng, T.C.E. & Zhang, Juliang & Fernandez, Vicenc, 2020. "Risk pooling through physical probabilistic selling," International Journal of Production Economics, Elsevier, vol. 219(C), pages 295-311.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:transe:v:187:y:2024:i:c:s1366554524001674. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/600244/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.