Group-Level Human Affect Recognition with Multiple Graph Kernel Fusion

Research on group-level affect recognition has become emerging for predicting human behavior in a group. Due to the variability in group size, a critical issue should be addressed. That is, how to efficiently and effectively describe the affect similarity of two group-level images. To tackle this problem, this paper makes two-fold contributions: (1) similarity measurement of group affect based on graph kernel; (2) incorporation of multiple kernels and deep learning architecture to contribute to recognizing group-level affect. In this paper, we view a group as a graph. Thus, we first formulated the graph of the group, and then build the kernel between any two graphs. Its advantage is to efficiently use any kernel classifier. To further exploit the advantages of multiple kernels, we used two feature descriptors to extract the face features. Subsequently, we proposed straightforwardly using deep multiple kernel learning with three-layer. To resolve non-differential problem, we presented the graph kernel as the input of four kinds of base kernels and learned their corresponding weights. Intensive experiments are performed on a challenging group-level affective database. Performance comparisons considerably demonstrate the advantages of graph kernel and deep multiple kernel learning. Additionally, our proposed approach obtains promising performance for group-level affect recognition compared with the recent state-of-the-art methods.