IDEAS home Printed from https://ideas.repec.org/a/gam/jsusta/v13y2021i2p970-d482841.html
   My bibliography  Save this article

Modeling Two-Person Segmentation and Locomotion for Stereoscopic Action Identification: A Sustainable Video Surveillance System

Author

Listed:
  • Nida Khalid

    (Department of Computer Science, Air University, Islamabad 44000, Pakistan)

  • Munkhjargal Gochoo

    (Department of Computer Science and Software Engineering, United Arab Emirates University, Al Ain 15551, UAE)

  • Ahmad Jalal

    (Department of Computer Science, Air University, Islamabad 44000, Pakistan)

  • Kibum Kim

    (Department of Human-Computer Interaction, Hanyang University, Ansan 15588, Korea)

Abstract

Due to the constantly increasing demand for automatic tracking and recognition systems, there is a need for more proficient, intelligent and sustainable human activity tracking. The main purpose of this study is to develop an accurate and sustainable human action tracking system that is capable of error-free identification of human movements irrespective of the environment in which those actions are performed. Therefore, in this paper we propose a stereoscopic Human Action Recognition (HAR) system based on the fusion of RGB (red, green, blue) and depth sensors. These sensors give an extra depth of information which enables the three-dimensional (3D) tracking of each and every movement performed by humans. Human actions are tracked according to four features, namely, (1) geodesic distance; (2) 3D Cartesian-plane features; (3) joints Motion Capture (MOCAP) features and (4) way-points trajectory generation. In order to represent these features in an optimized form, Particle Swarm Optimization (PSO) is applied. After optimization, a neuro-fuzzy classifier is used for classification and recognition. Extensive experimentation is performed on three challenging datasets: A Nanyang Technological University (NTU) RGB+D dataset; a UoL (University of Lincoln) 3D social activity dataset and a Collective Activity Dataset (CAD). Evaluation experiments on the proposed system proved that a fusion of vision sensors along with our unique features is an efficient approach towards developing a robust HAR system, having achieved a mean accuracy of 93.5% with the NTU RGB+D dataset, 92.2% with the UoL dataset and 89.6% with the Collective Activity dataset. The developed system can play a significant role in many computer vision-based applications, such as intelligent homes, offices and hospitals, and surveillance systems.

Suggested Citation

  • Nida Khalid & Munkhjargal Gochoo & Ahmad Jalal & Kibum Kim, 2021. "Modeling Two-Person Segmentation and Locomotion for Stereoscopic Action Identification: A Sustainable Video Surveillance System," Sustainability, MDPI, vol. 13(2), pages 1-30, January.
  • Handle: RePEc:gam:jsusta:v:13:y:2021:i:2:p:970-:d:482841
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2071-1050/13/2/970/pdf
    Download Restriction: no

    File URL: https://www.mdpi.com/2071-1050/13/2/970/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Ahmad Jalal & Mouazma Batool & Kibum Kim, 2020. "Sustainable Wearable System: Human Behavior Modeling for Life-Logging Activities Using K-Ary Tree Hashing Classifier," Sustainability, MDPI, vol. 12(24), pages 1-21, December.
    2. Ahmad Jalal & Israr Akhtar & Kibum Kim, 2020. "Human Posture Estimation and Sustainable Events Classification via Pseudo-2D Stick Model and K-ary Tree Hashing," Sustainability, MDPI, vol. 12(23), pages 1-24, November.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Mahwish Pervaiz & Yazeed Yasin Ghadi & Munkhjargal Gochoo & Ahmad Jalal & Shaharyar Kamal & Dong-Seong Kim, 2021. "A Smart Surveillance System for People Counting and Tracking Using Particle Flow and Modified SOM," Sustainability, MDPI, vol. 13(10), pages 1-20, May.
    2. Naif Al Mudawi & Mahwish Pervaiz & Bayan Ibrahimm Alabduallah & Abdulwahab Alazeb & Abdullah Alshahrani & Saud S. Alotaibi & Ahmad Jalal, 2023. "Predictive Analytics for Sustainable E-Learning: Tracking Student Behaviors," Sustainability, MDPI, vol. 15(20), pages 1-18, October.
    3. Madiha Javeed & Munkhjargal Gochoo & Ahmad Jalal & Kibum Kim, 2021. "HF-SPHR: Hybrid Features for Sustainable Physical Healthcare Pattern Recognition Using Deep Belief Networks," Sustainability, MDPI, vol. 13(4), pages 1-28, February.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Mahwish Pervaiz & Yazeed Yasin Ghadi & Munkhjargal Gochoo & Ahmad Jalal & Shaharyar Kamal & Dong-Seong Kim, 2021. "A Smart Surveillance System for People Counting and Tracking Using Particle Flow and Modified SOM," Sustainability, MDPI, vol. 13(10), pages 1-20, May.
    2. Hira Ansar & Ahmad Jalal & Munkhjargal Gochoo & Kibum Kim, 2021. "Hand Gesture Recognition Based on Auto-Landmark Localization and Reweighted Genetic Algorithm for Healthcare Muscle Activities," Sustainability, MDPI, vol. 13(5), pages 1-26, March.
    3. Madiha Javeed & Munkhjargal Gochoo & Ahmad Jalal & Kibum Kim, 2021. "HF-SPHR: Hybrid Features for Sustainable Physical Healthcare Pattern Recognition Using Deep Belief Networks," Sustainability, MDPI, vol. 13(4), pages 1-28, February.
    4. Jiacheng Wu & Han Cui & Naim Dahnoun, 2023. "A Voxelization Algorithm for Reconstructing mmWave Radar Point Cloud and an Application on Posture Classification for Low Energy Consumption Platform," Sustainability, MDPI, vol. 15(4), pages 1-13, February.
    5. Agnieszka Dudziak & Monika Stoma & Emilia Osmólska, 2023. "Analysis of Consumer Behaviour in the Context of the Place of Purchasing Food Products with Particular Emphasis on Local Products," IJERPH, MDPI, vol. 20(3), pages 1-23, January.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jsusta:v:13:y:2021:i:2:p:970-:d:482841. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.