Microphone-Based Context Awareness and Coverage Planner for a Service Robot Using Deep Learning Techniques

Floor-cleaning robots are becoming popular and operating in public places to ensure the places are clean and tidy. These robots are often operated in a dynamic environment that is less safe and has a high probability of ending up in accidents. Sound event-based context detection is expected to overcome drawbacks in a robot’s visual sensing to avoid a hazardous environment, especially in improper illumination and occlusion situations. Even though numerous studies in the literature discuss the benefits of sound-based context detection, there is no work reported related to context avoidance for cleaning robots. To this end, we propose a novel context avoidance framework based on a deep-learning method that can detect and classify a specific sound and localize the source from a robot’s frame to avoid that environment. The proposed model receives the spectrogram from the array of microphones as the input and produces two parallel outputs. The first output provides information about the spectrum class after running the classification task. The second output contains the localization message of the identified sound source. With the identity of the location that needs to be avoided, the proposed module will generate an alternative trajectory. The proposed model is evaluated in two real-world scenarios, wherein the model is trained to detect the escalator sound in the robot’s surroundings and avoid its location. In all considered scenarios, the developed system accomplished a significantly higher success rate in detecting and avoiding the escalator.