IDEAS home Printed from https://ideas.repec.org/a/gam/jsusta/v15y2023i12p9661-d1172653.html
   My bibliography  Save this article

Towards Sustainable Safe Driving: A Multimodal Fusion Method for Risk Level Recognition in Distracted Driving Status

Author

Listed:
  • Huiqin Chen

    (College of Mechanical Engineering, Hangzhou Dianzi University, Hangzhou 310018, China)

  • Hao Liu

    (College of Mechanical Engineering, Hangzhou Dianzi University, Hangzhou 310018, China)

  • Hailong Chen

    (College of Mechanical Engineering, Hangzhou Dianzi University, Hangzhou 310018, China)

  • Jing Huang

    (College of Mechanical and Vehicle Engineering, Hunan University, Changsha 410082, China)

Abstract

Precise driving status recognition is a prerequisite for human–vehicle collaborative driving systems towards sustainable road safety. In this study, a simulated driving platform was built to capture multimodal information simultaneously, including vision-modal data representing driver behaviour and sensor-modal data representing vehicle motion. Multisource data are used to quantify the risk of distracted driving status from four levels, safe driving, slight risk, moderate risk, and severe risk, rather than detecting action categories. A multimodal fusion method called vision-sensor fusion transformer (V-SFT) was proposed to incorporate the vision-modal of driver behaviour and sensor-modal data of vehicle motion. Feature concatenation was employed to aggregate representations of different modalities. Then, successive internal interactions were performed to consider the spatiotemporal dependency. Finally, the representations were clipped and mapped into four risk level label spaces. The proposed approach was evaluated under different modality inputs on the collected datasets and compared with some baseline methods. The results showed that V-SFT achieved the best performance with an recognition accuracy of 92.0%. It also indicates that fusing multimodal information effectively improves driving status understanding, and V-SFT extensibility is conducive to integrating more modal data.

Suggested Citation

  • Huiqin Chen & Hao Liu & Hailong Chen & Jing Huang, 2023. "Towards Sustainable Safe Driving: A Multimodal Fusion Method for Risk Level Recognition in Distracted Driving Status," Sustainability, MDPI, vol. 15(12), pages 1-22, June.
    • Handle: RePEc:gam:jsusta:v:15:y:2023:i:12:p:9661-:d:1172653
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2071-1050/15/12/9661/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2071-1050/15/12/9661/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Huacai Xian & Yujia Hou & Yu Wang & Shunzhong Dong & Junying Kou & Zewen Li, 2022. "Influence of Risky Driving Behavior and Road Section Type on Urban Expressway Driving Safety," Sustainability, MDPI, vol. 15(1), pages 1-15, December.
    2. Neska Haouij & Jean-Michel Poggi & Raja Ghozi & Sylvie Sevestre-Ghalila & Mériem Jaïdane, 2019. "Random forest-based approach for physiological functional variable selection for driver’s stress level classification," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 28(1), pages 157-185, March.
    3. Muhammad Muhitur Rahman & Md Kamrul Islam & Ammar Al-Shayeb & Md Arifuzzaman, 2022. "Towards Sustainable Road Safety in Saudi Arabia: Exploring Traffic Accident Causes Associated with Driving Behavior Using a Bayesian Belief Network," Sustainability, MDPI, vol. 14(10), pages 1-22, May.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Shujaat Abbas & Hazrat Yousaf & Shabeer Khan & Mohd Ziaur Rehman & Dmitri Blueschke, 2023. "Analysis and Projection of Transport Sector Demand for Energy and Carbon Emission: An Application of the Grey Model in Pakistan," Mathematics, MDPI, vol. 11(6), pages 1-14, March.
    2. Afaq Khattak & Hamad Almujibah & Ahmed Elamary & Caroline Mongina Matara, 2022. "Interpretable Dynamic Ensemble Selection Approach for the Prediction of Road Traffic Injury Severity: A Case Study of Pakistan’s National Highway N-5," Sustainability, MDPI, vol. 14(19), pages 1-18, September.
    3. Fabrizio Maturo & Rosanna Verde, 2023. "Supervised classification of curves via a combined use of functional data analysis and tree-based methods," Computational Statistics, Springer, vol. 38(1), pages 419-459, March.
    4. Darcin Akin & Virginia P. Sisiopiku & Ali H. Alateah & Ali O. Almonbhi & Mohammed M. H. Al-Tholaia & Khaled A. Alawi Al-Sodani, 2022. "Identifying Causes of Traffic Crashes Associated with Driver Behavior Using Supervised Machine Learning Methods: Case of Highway 15 in Saudi Arabia," Sustainability, MDPI, vol. 14(24), pages 1-36, December.
    5. Antonis Kostopoulos & Thodoris Garefalakis & Eva Michelaraki & Christos Katrakazas & George Yannis, 2024. "Modeling and Sustainability Implications of Harsh Driving Events: A Predictive Machine Learning Approach," Sustainability, MDPI, vol. 16(14), pages 1-20, July.
    6. Mohd Anjum & Sana Shahab, 2023. "Improving Autonomous Vehicle Controls and Quality Using Natural Language Processing-Based Input Recognition Model," Sustainability, MDPI, vol. 15(7), pages 1-21, March.
    7. Susrutha Babu Sukhavasi & Suparshya Babu Sukhavasi & Khaled Elleithy & Ahmed El-Sayed & Abdelrahman Elleithy, 2022. "Deep Neural Network Approach for Pose, Illumination, and Occlusion Invariant Driver Emotion Detection," IJERPH, MDPI, vol. 19(4), pages 1-23, February.
    8. Mohd Anjum & Sana Shahab, 2023. "Emergency Vehicle Driving Assistance System Using Recurrent Neural Network with Navigational Data Processing Method," Sustainability, MDPI, vol. 15(4), pages 1-19, February.
    9. Nattawut Pumpugsri & Wanchai Rattanawong & Varin Vongmanee, 2023. "Development of a Safety Heavy-Duty Vehicle Model Considering Unsafe Acts, Unsafe Conditions and Near-Miss Events Using Structural Equation Model," Sustainability, MDPI, vol. 15(16), pages 1-20, August.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jsusta:v:15:y:2023:i:12:p:9661-:d:1172653. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.