IDEAS home Printed from https://ideas.repec.org/a/eee/reensy/v215y2021ics0951832021001551.html
   My bibliography  Save this article

Systems Approach to Creating Test Scenarios for Automated Driving Systems

Author

Listed:
  • Khastgir, Siddartha
  • Brewerton, Simon
  • Thomas, John
  • Jennings, Paul

Abstract

Increased safety has been advocated as one of the major benefits of the introduction of Automated Driving Systems (ADSs). Incorporation of ADSs in vehicles means that associated software has safety critical application, thus requiring exhaustive testing. To prove ADSs are safer than human drivers, some work has suggested that they will need to be driven for over 11 billion miles. The number of test miles driven is not, by itself, a meaningful metric for judging the safety of ADSs. Rather, the types of scenarios encountered by the ADSs during testing are critically important.

Suggested Citation

  • Khastgir, Siddartha & Brewerton, Simon & Thomas, John & Jennings, Paul, 2021. "Systems Approach to Creating Test Scenarios for Automated Driving Systems," Reliability Engineering and System Safety, Elsevier, vol. 215(C).
    • Handle: RePEc:eee:reensy:v:215:y:2021:i:c:s0951832021001551
    DOI: 10.1016/j.ress.2021.107610
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0951832021001551
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.ress.2021.107610?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Dodd, Ian & Habli, Ibrahim, 2012. "Safety certification of airborne software: An empirical study," Reliability Engineering and System Safety, Elsevier, vol. 98(1), pages 7-23.
    2. Bolbot, Victor & Theotokatos, Gerasimos & Bujorianu, Luminita Manuela & Boulougouris, Evangelos & Vassalos, Dracos, 2019. "Vulnerabilities and safety assurance methods in Cyber-Physical Systems: A comprehensive review," Reliability Engineering and System Safety, Elsevier, vol. 182(C), pages 179-193.
    3. Mahajan, Haneet Singh & Bradley, Thomas & Pasricha, Sudeep, 2017. "Application of systems theoretic process analysis to a lane keeping assist system," Reliability Engineering and System Safety, Elsevier, vol. 167(C), pages 177-183.
    4. Vanslette, Kevin & Tohme, Tony & Youcef-Toumi, Kamal, 2020. "A general model validation and testing tool," Reliability Engineering and System Safety, Elsevier, vol. 195(C).
    5. Denney, Ewen & Pai, Ganesh & Whiteside, Iain, 2019. "The role of safety architectures in aviation safety cases," Reliability Engineering and System Safety, Elsevier, vol. 191(C).
    6. Read, G.J.M. & Naweed, A. & Salmon, P.M., 2019. "Complexity on the rails: A systems-based approach to understanding safety management in rail transport," Reliability Engineering and System Safety, Elsevier, vol. 188(C), pages 352-365.
    7. Wróbel, Krzysztof & Montewka, Jakub & Kujala, Pentti, 2018. "Towards the development of a system-theoretic model for safety assessment of autonomous merchant vessels," Reliability Engineering and System Safety, Elsevier, vol. 178(C), pages 209-224.
    8. Jensen, Anders & Aven, Terje, 2018. "A new definition of complexity in a risk analysis setting," Reliability Engineering and System Safety, Elsevier, vol. 171(C), pages 169-173.
    9. Leveson, Nancy, 2015. "A systems approach to risk management through leading safety indicators," Reliability Engineering and System Safety, Elsevier, vol. 136(C), pages 17-34.
    10. Levitin, Gregory & Xing, Liudong & Xiang, Yanping, 2020. "Cost minimization of real-time mission for software systems with rejuvenation," Reliability Engineering and System Safety, Elsevier, vol. 193(C).
    11. Stewart, Mark G. & Netherton, Michael D., 2019. "A probabilistic risk-acceptance model for assessing blast and fragmentation safety hazards," Reliability Engineering and System Safety, Elsevier, vol. 191(C).
    12. Bjerga, Torbjørn & Aven, Terje & Zio, Enrico, 2016. "Uncertainty treatment in risk analysis of complex systems: The cases of STAMP and FRAM," Reliability Engineering and System Safety, Elsevier, vol. 156(C), pages 203-209.
    13. Levitin, Gregory & Finkelstein, Maxim & Dai, Yuanshun, 2020. "Mission abort policy optimization for series systems with overlapping primary and rescue subsystems operating in a random environment," Reliability Engineering and System Safety, Elsevier, vol. 193(C).
    14. Zio, E., 2018. "The future of risk assessment," Reliability Engineering and System Safety, Elsevier, vol. 177(C), pages 176-190.
    15. Chen, Chao & Reniers, Genserik & Khakzad, Nima, 2019. "Integrating safety and security resources to protect chemical industrial parks from man-made domino effects: A dynamic graph approach," Reliability Engineering and System Safety, Elsevier, vol. 191(C).
    16. Rokseth, Børge & Utne, Ingrid Bouwer & Vinnem, Jan Erik, 2018. "Deriving verification objectives and scenarios for maritime systems using the systems-theoretic process analysis," Reliability Engineering and System Safety, Elsevier, vol. 169(C), pages 18-31.
    17. Cuer, Romain & Piétrac, Laurent & Niel, Eric & Diallo, Saidou & Minoiu-Enache, Nicoleta & Dang-Van-Nhan, Christophe, 2018. "A formal framework for the safe design of the Autonomous Driving supervision," Reliability Engineering and System Safety, Elsevier, vol. 174(C), pages 29-40.
    18. Kalra, Nidhi & Paddock, Susan M., 2016. "Driving to safety: How many miles of driving would it take to demonstrate autonomous vehicle reliability?," Transportation Research Part A: Policy and Practice, Elsevier, vol. 94(C), pages 182-193.
    19. Levitin, Gregory & Xing, Liudong & Luo, Liang, 2019. "Influence of failure propagation on mission abort policy in heterogeneous warm standby systems," Reliability Engineering and System Safety, Elsevier, vol. 183(C), pages 29-38.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Pauer, Gábor & Török, à rpád, 2022. "Introducing a novel safety assessment method through the example of a reduced complexity binary integer autonomous transport model," Reliability Engineering and System Safety, Elsevier, vol. 217(C).
    2. Antonello, Federico & Buongiorno, Jacopo & Zio, Enrico, 2022. "A methodology to perform dynamic risk assessment using system theory and modeling and simulation: Application to nuclear batteries," Reliability Engineering and System Safety, Elsevier, vol. 228(C).
    3. Chelouati, Mohammed & Boussif, Abderraouf & Beugin, Julie & El Koursi, El-Miloudi, 2023. "Graphical safety assurance case using Goal Structuring Notation (GSN) — challenges, opportunities and a framework for autonomous trains," Reliability Engineering and System Safety, Elsevier, vol. 230(C).
    4. Bensaci, Chaima & Zennir, Youcef & Pomorski, Denis & Innal, Fares & Lundteigen, Mary Ann, 2023. "Collision hazard modeling and analysis in a multi-mobile robots system transportation task with STPA and SPN," Reliability Engineering and System Safety, Elsevier, vol. 234(C).
    5. Shiokari, Megumi & Itoh, Hiroko & Yuzui, Tomohiro & Ishimura, Eiko & Miyake, Rina & Kudo, Junichi & Kawashima, Sonoko, 2024. "Structure model-based hazard identification method for autonomous ships," Reliability Engineering and System Safety, Elsevier, vol. 247(C).
    6. Tengilimoglu, Oguz & Carsten, Oliver & Wadud, Zia, 2023. "Implications of automated vehicles for physical road environment: A comprehensive review," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 169(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Chelouati, Mohammed & Boussif, Abderraouf & Beugin, Julie & El Koursi, El-Miloudi, 2023. "Graphical safety assurance case using Goal Structuring Notation (GSN) — challenges, opportunities and a framework for autonomous trains," Reliability Engineering and System Safety, Elsevier, vol. 230(C).
    2. Read, G.J.M. & Naweed, A. & Salmon, P.M., 2019. "Complexity on the rails: A systems-based approach to understanding safety management in rail transport," Reliability Engineering and System Safety, Elsevier, vol. 188(C), pages 352-365.
    3. Juntao Zhang & Hyungju Kim & Yiliu Liu & Mary Ann Lundteigen, 2019. "Combining system-theoretic process analysis and availability assessment: A subsea case study," Journal of Risk and Reliability, , vol. 233(4), pages 520-536, August.
    4. Langdalen, Henrik & Abrahamsen, Eirik Bjorheim & Abrahamsen, HÃ¥kon Bjorheim, 2020. "A New Framework To Idenitfy And Assess Hidden Assumptions In The Background Knowledge Of A Risk Assessment," Reliability Engineering and System Safety, Elsevier, vol. 200(C).
    5. Faiella, Giuliana & Parand, Anam & Franklin, Bryony Dean & Chana, Prem & Cesarelli, Mario & Stanton, Neville A. & Sevdalis, Nick, 2018. "Expanding healthcare failure mode and effect analysis: A composite proactive risk analysis approach," Reliability Engineering and System Safety, Elsevier, vol. 169(C), pages 117-126.
    6. Wróbel, Krzysztof & Montewka, Jakub & Kujala, Pentti, 2018. "Towards the development of a system-theoretic model for safety assessment of autonomous merchant vessels," Reliability Engineering and System Safety, Elsevier, vol. 178(C), pages 209-224.
    7. Shin, Sung-Min & Lee, Sang Hun & Shin, Seung Ki & Jang, Inseok & Park, Jinkyun, 2021. "STPA-Based Hazard and Importance Analysis on NPP Safety I&C Systems Focusing on Human–System Interactions," Reliability Engineering and System Safety, Elsevier, vol. 213(C).
    8. Utne, Ingrid Bouwer & Rokseth, Børge & Sørensen, Asgeir J. & Vinnem, Jan Erik, 2020. "Towards supervisory risk control of autonomous ships," Reliability Engineering and System Safety, Elsevier, vol. 196(C).
    9. Levitin, Gregory & Xing, Liudong & Dai, Yuanshun, 2023. "Optimal aborting policy for shock exposed missions with random rescue time," Reliability Engineering and System Safety, Elsevier, vol. 233(C).
    10. Levitin, Gregory & Xing, Liudong & Xiang, Yanping & Dai, Yuanshun, 2021. "Mixed failure-driven and shock-driven mission aborts in heterogeneous systems with arbitrary structure," Reliability Engineering and System Safety, Elsevier, vol. 212(C).
    11. Ke Chen & Xian Zhao & Qingan Qiu, 2022. "Optimal Task Abort and Maintenance Policies Considering Time Redundancy," Mathematics, MDPI, vol. 10(9), pages 1-16, April.
    12. Wu, Chao & Huang, Lang, 2019. "A new accident causation model based on information flow and its application in Tianjin Port fire and explosion accident," Reliability Engineering and System Safety, Elsevier, vol. 182(C), pages 73-85.
    13. Gregory Levitin & Liudong Xing & Yuanshun Dai, 2020. "Mission Abort Policy for Systems with Observable States of Standby Components," Risk Analysis, John Wiley & Sons, vol. 40(10), pages 1900-1912, October.
    14. Feng, Jian Rui & Yu, Guanghui & Zhao, Mengke & Zhang, Jiaqing & Lu, Shouxiang, 2022. "Dynamic risk assessment framework for industrial systems based on accidents chain theory: The case study of fire and explosion risk of UHV converter transformer," Reliability Engineering and System Safety, Elsevier, vol. 228(C).
    15. Levitin, Gregory & Xing, Liudong & Dai, Yuanshun, 2021. "Dynamic task distribution balancing primary mission work and damage reduction work in parallel systems exposed to shocks," Reliability Engineering and System Safety, Elsevier, vol. 215(C).
    16. Terje Aven, 2019. "The Call for a Shift from Risk to Resilience: What Does it Mean?," Risk Analysis, John Wiley & Sons, vol. 39(6), pages 1196-1203, June.
    17. Victor Bolbot & Gerasimos Theotokatos & LA Wennersberg & Jerome Faivre & Dracos Vassalos & Evangelos Boulougouris & Ørnulf Jan Rødseth & Pål Andersen & Ann-Sofie Pauwelyn & Antoon Van Coillie, 2023. "A novel risk assessment process: Application to an autonomous inland waterways ship," Journal of Risk and Reliability, , vol. 237(2), pages 436-458, April.
    18. Levitin, Gregory & Xing, Liudong & Dai, Yuanshun, 2023. "Optimal task sequencing and aborting in multi-attempt multi-task missions with a limited number of attempts," Reliability Engineering and System Safety, Elsevier, vol. 236(C).
    19. Varajão, João & Fernandes, Gabriela & Amaral, António & Gonçalves, A. Manuela, 2021. "Team Resilience Model: An Empirical Examination of Information Systems Projects," Reliability Engineering and System Safety, Elsevier, vol. 206(C).
    20. Qiu, Qingan & Kou, Meng & Chen, Ke & Deng, Qiao & Kang, Fengming & Lin, Cong, 2021. "Optimal stopping problems for mission oriented systems considering time redundancy," Reliability Engineering and System Safety, Elsevier, vol. 205(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:reensy:v:215:y:2021:i:c:s0951832021001551. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: https://www.journals.elsevier.com/reliability-engineering-and-system-safety .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.