Facial Soft Biometrics For Recognition in the Wild: Recent Works Annotation and COTS Evaluation

The role of soft biometrics to enhance person recognition systems in unconstrained scenarios has not been extensively studied. We explore the utility of the following soft biometrics in unconstrained scenarios: gender, ethnicity, age, glasses, beard and moustache. We consider two assumptions: i) manual estimation of soft biometrics, and ii) automatic estimation from two COTS systems. All experiments are reported using the LFW database. First, we study the discrimination capabilities of soft biometrics standalone. Then, experiments are carried out fusing soft biometrics with two state-of-the-art face recognition systems based on deep learning, proving to what extent soft biometrics complement the evidence given by the face modality in unconstrained scenarios, with relative improvements in the verification performance up to 40% or 15% when using manual or automatic soft biometrics, respectively. Results are reproducible as we make public our manual annotations and COTS outputs of soft biometrics over LFW, as well as the face scores to which we incorporate the soft information.