A Novel Approach to Face Verification Based on Second-Order Face-Pair Representation

Face representation and matching are two essential issues in face verification task. Various approaches have been proposed focusing on these two issues. However, few of them addressed the joint optimal solutions of these two issues in a unified framework. In this paper, we present a second-order face representation method for face pair and a unified face verification framework, in which the feature extractors and the subsequent binary classification model design can be selected flexibly. Our contributions can be summarized in the following aspects. First, a novel face-pair representation method that employs the second-order statistical property of the face pairs is proposed, which retains more information compared to the existing methods. Second, a flexible binary classification model, which differs from the conventionally used metric learning, is constructed based on the new face-pair representation. Finally, we verify that our proposed face-pair representation can benefit from large training datasets. All the experiments are carried out on Labeled Face in the Wild (LFW) to verify the algorithm’s effectiveness against challenging uncontrolled conditions.