IDEAS home Printed from https://ideas.repec.org/a/gam/jmathe/v11y2023i2p327-d1028785.html
   My bibliography  Save this article

A Hybrid Reinforcement Learning Algorithm for 2D Irregular Packing Problems

Author

Listed:
  • Jie Fang

    (School of Mechanical Science and Engineering, Huazhong University of Science and Technology, Wuhan 430074, China
    State Key Laboratory of Digital Manufacturing Equipment and Technology, Huazhong University of Science and Technology, Wuhan 430074, China)

  • Yunqing Rao

    (School of Mechanical Science and Engineering, Huazhong University of Science and Technology, Wuhan 430074, China
    State Key Laboratory of Digital Manufacturing Equipment and Technology, Huazhong University of Science and Technology, Wuhan 430074, China)

  • Xusheng Zhao

    (School of Mechanical Science and Engineering, Huazhong University of Science and Technology, Wuhan 430074, China
    State Key Laboratory of Digital Manufacturing Equipment and Technology, Huazhong University of Science and Technology, Wuhan 430074, China)

  • Bing Du

    (School of Mechanical Science and Engineering, Huazhong University of Science and Technology, Wuhan 430074, China
    State Key Laboratory of Digital Manufacturing Equipment and Technology, Huazhong University of Science and Technology, Wuhan 430074, China)

Abstract

Packing problems, also known as nesting problems or bin packing problems, are classic and popular NP-hard problems with high computational complexity. Inspired by classic reinforcement learning (RL), we established a mathematical model for two-dimensional (2D) irregular-piece packing combined with characteristics of 2D irregular pieces. An RL algorithm based on Monte Carlo learning (MC), Q-learning, and Sarsa-learning is proposed in this paper to solve a 2D irregular-piece packing problem. Additionally, mechanisms of reward–return and strategy-update based on piece packing were designed. Finally, the standard test case of irregular pieces was used for experimental testing to analyze the optimization effect of the algorithm. The experimental results show that the proposed algorithm can successfully realize packing of 2D irregular pieces. A similar or better optimization effect can be obtained compared to some classical heuristic algorithms. The proposed algorithm is an early attempt to use machine learning to solve 2D irregular packing problems. On the one hand, our hybrid RL algorithm can provide a basis for subsequent deep reinforcement learning (DRL) to solve packing problems, which has far-reaching theoretical significance. On the other hand, it has practical significance for improving the utilization rate of raw materials and broadening the application field of machine learning.

Suggested Citation

  • Jie Fang & Yunqing Rao & Xusheng Zhao & Bing Du, 2023. "A Hybrid Reinforcement Learning Algorithm for 2D Irregular Packing Problems," Mathematics, MDPI, vol. 11(2), pages 1-17, January.
    • Handle: RePEc:gam:jmathe:v:11:y:2023:i:2:p:327-:d:1028785
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2227-7390/11/2/327/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2227-7390/11/2/327/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Cherri, Luiz H. & Mundim, Leandro R. & Andretta, Marina & Toledo, Franklina M.B. & Oliveira, José F. & Carravilla, Maria Antónia, 2016. "Robust mixed-integer linear programming models for the irregular strip packing problem," European Journal of Operational Research, Elsevier, vol. 253(3), pages 570-583.
    2. Leao, Aline A.S. & Toledo, Franklina M.B. & Oliveira, José Fernando & Carravilla, Maria Antónia & Alvarez-Valdés, Ramón, 2020. "Irregular packing problems: A review of mathematical models," European Journal of Operational Research, Elsevier, vol. 282(3), pages 803-822.
    3. Hopper, E. & Turton, B. C. H., 2001. "An empirical investigation of meta-heuristic and heuristic algorithms for a 2D packing problem," European Journal of Operational Research, Elsevier, vol. 128(1), pages 34-57, January.
    4. Liu, D.S. & Tan, K.C. & Huang, S.Y. & Goh, C.K. & Ho, W.K., 2008. "On solving multiobjective bin packing problems using evolutionary particle swarm optimization," European Journal of Operational Research, Elsevier, vol. 190(2), pages 357-382, October.
    5. Li, Zhenyu & Milenkovic, Victor, 1995. "Compaction and separation algorithms for non-convex polygons and their applications," European Journal of Operational Research, Elsevier, vol. 84(3), pages 539-561, August.
    6. J A Bennell & J F Oliveira, 2009. "A tutorial in irregular shape packing problems," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 60(1), pages 93-105, May.
    7. Gomes, A. Miguel & Oliveira, Jose F., 2006. "Solving Irregular Strip Packing problems by hybridising simulated annealing and linear programming," European Journal of Operational Research, Elsevier, vol. 171(3), pages 811-829, June.
    8. Yunqing Rao & Peng Wang & Qiang Luo, 2021. "Hybridizing Beam Search with Tabu Search for the Irregular Packing Problem," Mathematical Problems in Engineering, Hindawi, vol. 2021, pages 1-14, January.
    9. Elkeran, Ahmed, 2013. "A new approach for sheet nesting problem using guided cuckoo search and pairwise clustering," European Journal of Operational Research, Elsevier, vol. 231(3), pages 757-769.
    10. Burke, E.K. & Hellier, R.S.R. & Kendall, G. & Whitwell, G., 2007. "Complete and robust no-fit polygon generation for the irregular stock cutting problem," European Journal of Operational Research, Elsevier, vol. 179(1), pages 27-49, May.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Jie Fang & Yunqing Rao & Qiang Luo & Jiatai Xu, 2023. "Solving One-Dimensional Cutting Stock Problems with the Deep Reinforcement Learning," Mathematics, MDPI, vol. 11(4), pages 1-16, February.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Qiang Luo & Yunqing Rao, 2022. "Improved Sliding Algorithm for Generating No-Fit Polygon in the 2D Irregular Packing Problem," Mathematics, MDPI, vol. 10(16), pages 1-18, August.
    2. Sato, André Kubagawa & Martins, Thiago Castro & Gomes, Antonio Miguel & Tsuzuki, Marcos Sales Guerra, 2019. "Raster penetration map applied to the irregular packing problem," European Journal of Operational Research, Elsevier, vol. 279(2), pages 657-671.
    3. Umetani, Shunji & Murakami, Shohei, 2022. "Coordinate descent heuristics for the irregular strip packing problem of rasterized shapes," European Journal of Operational Research, Elsevier, vol. 303(3), pages 1009-1026.
    4. Leao, Aline A.S. & Toledo, Franklina M.B. & Oliveira, José Fernando & Carravilla, Maria Antónia & Alvarez-Valdés, Ramón, 2020. "Irregular packing problems: A review of mathematical models," European Journal of Operational Research, Elsevier, vol. 282(3), pages 803-822.
    5. Lastra-Díaz, Juan J. & Ortuño, M. Teresa, 2024. "Mixed-integer programming models for irregular strip packing based on vertical slices and feasibility cuts," European Journal of Operational Research, Elsevier, vol. 313(1), pages 69-91.
    6. Igor Kierkosz & Maciej Łuczak, 2019. "A one-pass heuristic for nesting problems," Operations Research and Decisions, Wroclaw University of Science and Technology, Faculty of Management, vol. 29(1), pages 37-60.
    7. Elkeran, Ahmed, 2013. "A new approach for sheet nesting problem using guided cuckoo search and pairwise clustering," European Journal of Operational Research, Elsevier, vol. 231(3), pages 757-769.
    8. Cherri, Luiz Henrique & Carravilla, Maria Antónia & Ribeiro, Cristina & Toledo, Franklina Maria Bragion, 2019. "Optimality in nesting problems: New constraint programming models and a new global constraint for non-overlap," Operations Research Perspectives, Elsevier, vol. 6(C).
    9. Gahm, Christian & Uzunoglu, Aykut & Wahl, Stefan & Ganschinietz, Chantal & Tuma, Axel, 2022. "Applying machine learning for the anticipation of complex nesting solutions in hierarchical production planning," European Journal of Operational Research, Elsevier, vol. 296(3), pages 819-836.
    10. Nascimento, Paulo Jorge & Silva, Cristóvão & Antunes, Carlos Henggeler & Moniz, Samuel, 2024. "Optimal decomposition approach for solving large nesting and scheduling problems of additive manufacturing systems," European Journal of Operational Research, Elsevier, vol. 317(1), pages 92-110.
    11. Edmund K. Burke & Graham Kendall & Glenn Whitwell, 2009. "A Simulated Annealing Enhancement of the Best-Fit Heuristic for the Orthogonal Stock-Cutting Problem," INFORMS Journal on Computing, INFORMS, vol. 21(3), pages 505-516, August.
    12. Miguel Santoro & Felipe Lemos, 2015. "Irregular packing: MILP model based on a polygonal enclosure," Annals of Operations Research, Springer, vol. 235(1), pages 693-707, December.
    13. Kimms, Alf & Király, Hédi, 2023. "An extended model formulation for the two-dimensional irregular strip packing problem considering general industry-relevant aspects," European Journal of Operational Research, Elsevier, vol. 306(3), pages 1202-1218.
    14. Toledo, Franklina M.B. & Carravilla, Maria Antónia & Ribeiro, Cristina & Oliveira, José F. & Gomes, A. Miguel, 2013. "The Dotted-Board Model: A new MIP model for nesting irregular shapes," International Journal of Production Economics, Elsevier, vol. 145(2), pages 478-487.
    15. Donald Jones, 2014. "A fully general, exact algorithm for nesting irregular shapes," Journal of Global Optimization, Springer, vol. 59(2), pages 367-404, July.
    16. Akang Wang & Christopher L. Hanselman & Chrysanthos E. Gounaris, 2018. "A customized branch-and-bound approach for irregular shape nesting," Journal of Global Optimization, Springer, vol. 71(4), pages 935-955, August.
    17. Hagspihl, Thomas & Kolisch, Rainer & Fontaine, Pirmin & Schiffels, Sebastian, 2024. "Apron layout planning–Optimal positioning of aircraft stands," Transportation Research Part B: Methodological, Elsevier, vol. 179(C).
    18. Bennell, J.A. & Cabo, M. & Martínez-Sykora, A., 2018. "A beam search approach to solve the convex irregular bin packing problem with guillotine guts," European Journal of Operational Research, Elsevier, vol. 270(1), pages 89-102.
    19. Iori, Manuel & de Lima, Vinícius L. & Martello, Silvano & Miyazawa, Flávio K. & Monaci, Michele, 2021. "Exact solution techniques for two-dimensional cutting and packing," European Journal of Operational Research, Elsevier, vol. 289(2), pages 399-415.
    20. Eunice López-Camacho & Gabriela Ochoa & Hugo Terashima-Marín & Edmund Burke, 2013. "An effective heuristic for the two-dimensional irregular bin packing problem," Annals of Operations Research, Springer, vol. 206(1), pages 241-264, July.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jmathe:v:11:y:2023:i:2:p:327-:d:1028785. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.