IDEAS home Printed from https://ideas.repec.org/a/spr/orspec/v44y2022i1d10.1007_s00291-021-00652-x.html
   My bibliography  Save this article

Stochastic mixed model sequencing with multiple stations using reinforcement learning and probability quantiles

Author

Listed:
  • Janis Brammer

    (Volkswagen AG)

  • Bernhard Lutz

    (University of Freiburg)

  • Dirk Neumann

    (University of Freiburg)

Abstract

In this study, we propose a reinforcement learning (RL) approach for minimizing the number of work overload situations in the mixed model sequencing (MMS) problem with stochastic processing times. The learning environment simulates stochastic processing times and penalizes work overloads with negative rewards. To account for the stochastic component of the problem, we implement a state representation that specifies whether work overloads will occur if the processing times are equal to their respective 25%, 50%, and 75% probability quantiles. Thereby, the RL agent is guided toward minimizing the number of overload situations while being provided with statistical information about how fluctuations in processing times affect the solution quality. To the best of our knowledge, this study is the first to consider the stochastic problem variation with a minimization of overload situations.

Suggested Citation

  • Janis Brammer & Bernhard Lutz & Dirk Neumann, 2022. "Stochastic mixed model sequencing with multiple stations using reinforcement learning and probability quantiles," OR Spectrum: Quantitative Approaches in Management, Springer;Gesellschaft für Operations Research e.V., vol. 44(1), pages 29-56, March.
    • Handle: RePEc:spr:orspec:v:44:y:2022:i:1:d:10.1007_s00291-021-00652-x
    DOI: 10.1007/s00291-021-00652-x
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s00291-021-00652-x
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s00291-021-00652-x?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Mosadegh, H. & Fatemi Ghomi, S.M.T. & Süer, G.A., 2020. "Stochastic mixed-model assembly line sequencing problem: Mathematical modeling and Q-learning based simulated annealing hyper-heuristics," European Journal of Operational Research, Elsevier, vol. 282(2), pages 530-544.
    2. Pâmela M.C. Cortez & Alysson M. Costa, 2015. "Sequencing mixed-model assembly lines operating with a heterogeneous workforce," International Journal of Production Research, Taylor & Francis Journals, vol. 53(11), pages 3419-3432, June.
    3. Golle, Uli & Rothlauf, Franz & Boysen, Nils, 2014. "Car sequencing versus mixed-model sequencing: A computational study," European Journal of Operational Research, Elsevier, vol. 237(1), pages 50-61.
    4. Boysen, Nils & Fliedner, Malte & Scholl, Armin, 2007. "A classification of assembly line balancing problems," European Journal of Operational Research, Elsevier, vol. 183(2), pages 674-693, December.
    5. Solnon, Christine & Cung, Van Dat & Nguyen, Alain & Artigues, Christian, 2008. "The car sequencing problem: Overview of state-of-the-art methods and industrial case-study of the ROADEF'2005 challenge problem," European Journal of Operational Research, Elsevier, vol. 191(3), pages 912-927, December.
    6. Gabrel, Virginie & Murat, Cécile & Thiele, Aurélie, 2014. "Recent advances in robust optimization: An overview," European Journal of Operational Research, Elsevier, vol. 235(3), pages 471-483.
    7. Bautista, Joaquín & Cano, Alberto & Alfaro, Rocío, 2012. "Models for MMSP-W considering workstation dependencies: A case study of Nissan’s Barcelona plant," European Journal of Operational Research, Elsevier, vol. 223(3), pages 669-679.
    8. Karim Aroui & Gülgün Alpan & Yannick Frein, 2017. "Minimising work overload in mixed-model assembly lines with different types of operators: a case study from the truck industry," International Journal of Production Research, Taylor & Francis Journals, vol. 55(21), pages 6305-6326, November.
    9. Boysen, Nils & Fliedner, Malte & Scholl, Armin, 2009. "Sequencing mixed-model assembly lines: Survey, classification and model critique," European Journal of Operational Research, Elsevier, vol. 192(2), pages 349-373, January.
    10. H. Mosadegh & S.M.T. Fatemi Ghomi & G.A. Süer, 2017. "Heuristic approaches for mixed-model sequencing problem with stochastic processing times," International Journal of Production Research, Taylor & Francis Journals, vol. 55(10), pages 2857-2880, May.
    11. Russell W. Bent & Pascal Van Hentenryck, 2004. "Scenario-Based Planning for Partially Dynamic Vehicle Routing with Stochastic Customers," Operations Research, INFORMS, vol. 52(6), pages 977-987, December.
    12. Michael Pinedo & Gideon Weiss, 1987. "The “Largest Variance First” Policy in Some Stochastic Scheduling Problems," Operations Research, INFORMS, vol. 35(6), pages 884-891, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Yilmazlar, I. Ozan & Kurz, Mary E. & Rahimian, Hamed, 2024. "Mixed-model sequencing with stochastic failures: A case study for automobile industry," European Journal of Operational Research, Elsevier, vol. 319(1), pages 206-221.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yilmazlar, I. Ozan & Kurz, Mary E. & Rahimian, Hamed, 2024. "Mixed-model sequencing with stochastic failures: A case study for automobile industry," European Journal of Operational Research, Elsevier, vol. 319(1), pages 206-221.
    2. Karim Aroui & Gülgün Alpan & Yannick Frein, 2017. "Minimising work overload in mixed-model assembly lines with different types of operators: a case study from the truck industry," International Journal of Production Research, Taylor & Francis Journals, vol. 55(21), pages 6305-6326, November.
    3. Boysen, Nils & Bock, Stefan, 2011. "Scheduling just-in-time part supply for mixed-model assembly lines," European Journal of Operational Research, Elsevier, vol. 211(1), pages 15-25, May.
    4. Masoud Rabbani & Mahdi Mokhtarzadeh & Neda Manavizadeh & Azadeh Farsi, 2021. "Solving a bi-objective mixed-model assembly-line sequencing using metaheuristic algorithms considering ergonomic factors, customer behavior, and periodic maintenance," OPSEARCH, Springer;Operational Research Society of India, vol. 58(3), pages 513-539, September.
    5. Asieh Varyani & Mohsen Salehi & Meysam Heydari Gharahcheshmeh, 2024. "Optimizing Mixed-Model Synchronous Assembly Lines with Bipartite Sequence-Dependent Setup Times in Advanced Manufacturing," Energies, MDPI, vol. 17(12), pages 1-19, June.
    6. Mosadegh, H. & Fatemi Ghomi, S.M.T. & Süer, G.A., 2020. "Stochastic mixed-model assembly line sequencing problem: Mathematical modeling and Q-learning based simulated annealing hyper-heuristics," European Journal of Operational Research, Elsevier, vol. 282(2), pages 530-544.
    7. Otto, Alena & Li, Xiyu, 2020. "Product sequencing in multiple-piece-flow assembly lines," Omega, Elsevier, vol. 91(C).
    8. Pontes, Lara & Neves, Carlos & Subramanian, Anand & Battarra, Maria, 2024. "The maximum length car sequencing problem," European Journal of Operational Research, Elsevier, vol. 316(2), pages 707-717.
    9. Bautista, Joaquín & Alfaro, Rocío & Batalla, Cristina, 2015. "Modeling and solving the mixed-model sequencing problem to improve productivity," International Journal of Production Economics, Elsevier, vol. 161(C), pages 83-95.
    10. Golle, Uli & Rothlauf, Franz & Boysen, Nils, 2014. "Car sequencing versus mixed-model sequencing: A computational study," European Journal of Operational Research, Elsevier, vol. 237(1), pages 50-61.
    11. Iwona Paprocka & Damian Krenczyk, 2023. "On Energy Consumption and Productivity in a Mixed-Model Assembly Line Sequencing Problem," Energies, MDPI, vol. 16(20), pages 1-19, October.
    12. Boysen, Nils & Scholl, Armin & Wopperer, Nico, 2012. "Resequencing of mixed-model assembly lines: Survey and research agenda," European Journal of Operational Research, Elsevier, vol. 216(3), pages 594-604.
    13. Sikora, Celso Gustavo Stall, 2024. "Balancing mixed-model assembly lines for random sequences," European Journal of Operational Research, Elsevier, vol. 314(2), pages 597-611.
    14. Battaïa, Olga & Dolgui, Alexandre, 2022. "Hybridizations in line balancing problems: A comprehensive review on new trends and formulations," International Journal of Production Economics, Elsevier, vol. 250(C).
    15. Staeblein, Thomas & Aoki, Katsuki, 2015. "Planning and scheduling in the automotive industry: A comparison of industrial practice at German and Japanese makers," International Journal of Production Economics, Elsevier, vol. 162(C), pages 258-272.
    16. Thorben Krueger & Achim Koberstein & Norbert Bittner, 2022. "Anticipating technical car sequencing rules in the master production scheduling of mixed-model assembly lines," Flexible Services and Manufacturing Journal, Springer, vol. 34(2), pages 351-407, June.
    17. Marcel Lehmann & Heinrich Kuhn, 2020. "Modeling and analyzing sequence stability in flexible automotive production systems," Flexible Services and Manufacturing Journal, Springer, vol. 32(2), pages 366-394, June.
    18. Florian Jaehn & Sergey Kovalev & Mikhail Y. Kovalyov & Erwin Pesch, 2014. "Multiproduct batching and scheduling with buffered rework: The case of a car paint shop," Naval Research Logistics (NRL), John Wiley & Sons, vol. 61(6), pages 458-471, September.
    19. Dominik Kress & Sebastian Meiswinkel & Erwin Pesch, 2018. "Mechanism design for machine scheduling problems: classification and literature overview," OR Spectrum: Quantitative Approaches in Management, Springer;Gesellschaft für Operations Research e.V., vol. 40(3), pages 583-611, July.
    20. Uli Golle & Franz Rothlauf & Nils Boysen, 2015. "Iterative beam search for car sequencing," Annals of Operations Research, Springer, vol. 226(1), pages 239-254, March.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:orspec:v:44:y:2022:i:1:d:10.1007_s00291-021-00652-x. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.