親子關係確認在電腦視覺領域中是一個具有潛力的新興任務，但在這個任務中缺乏大型的資料集來訓練一個具有辨識能力和強健性的模型，進而限制了這個任務的發展。此外，人臉確認在膚色和種族方面存在偏見，這些問題在先前的研究中尚未被完全地解決。在這篇論文中我們提出一個多任務學習的模型架構同時加入注意力模塊來提升正確率，並超越了其他頂尖方法的表現。而且我們的公平性對比損失函數融合了去偏差變數和對抗性學習，進而大幅減少種族偏差、提升公平性。在實驗中，我們融合了數個親子資料集並為每個身分標上種族資訊，建立了一個大型種族親子資料集。詳盡的實驗結果證明了我們所提出的方法在標準差以及正確率都達到有效性和卓越的表現。

Kinship verification is an emerging task in computer vision with several potential applications. However, there is a lack of large kinship datasets to train a discriminative and robust model, which is a major limitation for this problem. Moreover, face verification is known to exhibit bias in skin colors and ethics, which has not been fully resolved by previous works. In this paper, we propose a multi-task learning model structure with attention module to improve the accuracy, which surpasses state-of-the-art performance. In addition, our fairness-aware contrastive loss function combined with a debias term and adversarial learning greatly mitigates racial bias, thus significantly improving the fairness. In the experiment, we build a large dataset by combining several existing kinship datasets. Exhaustive experimental evaluation demonstrates the effectiveness and superior performance of the proposed KFC in both standard deviation and accuracy at the same time.

[1] Ahonen, T., Hadid, A., and Pietikäinen, M. Face recognition with local binary patterns. vol. 3021, pp. 469–481.
[2] Bromley, J., Guyon, I., LeCun, Y., Säckinger, E., and Shah, R. Signature verification using a ”siamese” time delay neural network. In Proceedings of the 6th International Conference on Neural Information Processing Systems (San Francisco, CA, USA, 1993), NIPS’93, Morgan Kaufmann Publishers Inc., p. 737–744.
[3] Chergui, A., Ouchtati, S., Telli, H., Bougourzi, F., and Bekhouche, S. E. Lpq and ldp descriptors with ml representation for kinship verification.
[4] Chicco, D. Siamese Neural Networks: An Overview. Springer US, New York, NY, 2021, pp. 73–94.
[5] Chopra, S., Hadsell, R., and LeCun, Y. Learning a similarity metric discriminatively, with application to face verification. In 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05) (2005), vol. 1, pp. 539–546 vol. 1.
[6] Cui, L., and Ma, B. Adaptive feature selection for kinship verification. In 2017 IEEE International Conference on Multimedia and Expo (ICME) (2017), pp. 751–756.
[7] Deng, J., Guo, J., Yang, J., Xue, N., Kotsia, I., and Zafeiriou, S. ArcFace: Additive angular margin loss for deep face recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 44, 10 (oct 2022), 5962–5979.
[8] Fang, R., Gallagher, A. C., Chen, T., and Loui, A. Kinship classification by modeling facial feature heredity. In 2013 IEEE International Conference on Image Processing (2013), pp. 2983–2987.
[9] Fang, R., Tang, K. D., Snavely, N., and Chen, T. Towards computational models of kinship verification. In 2010 IEEE International Conference on Image Processing (2010), pp. 1577–1580.
[10] Gong, S., Liu, X., and Jain, A. K. Jointly de-biasing face recognition and demographic attribute estimation, 2020.
[11] Gong, S., Liu, X., and Jain, A. K. Mitigating face recognition bias via group adaptive classifier, 2020.
[12] Guardian, T. Amazon ditched ai recruiting tool that favored men for technical jobs, 2018. October 11, 2018.
[13] Guo, Y., Zhang, L., Hu, Y., He, X., and Gao, J. Ms-celeb-1m: A dataset and benchmark for large-scale face recognition, 2016.
[14] He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learning for image recognition, 2015.
[15] Jeff Larson, Surya Mattu, L. K., and Angwin, J. How we analyzed the compas recidivism algorithm, 2016. May 23, 2016.
[16] Khosla, P., Teterwak, P., Wang, C., Sarna, A., Tian, Y., Isola, P., Maschinot, A., Liu, C., and Krishnan, D. Supervised contrastive learning, 2021.
[17] Laiadi, O., Ouamane, A., Benakcha, ., taleb ahmed, A., and Hadid, A. A weighted exponential discriminant analysis through side-information for face and kinship verification using statistical binarized image features. International Journal of Machine Learning and Cybernetics 12 (01 2021).
[18] Lu, J., Hu, J., Zhou, X., Shang, Y., Tan, Y.-P., and Wang, G. Neighborhood repulsed metric learning for kinship verification. In 2012 IEEE Conference on Computer Vision and Pattern Recognition (2012), pp. 2594–2601.
[19] López, M. B., Boutellaa, E., and Hadid, A. Comments on the “kinship face in the wild”data sets. IEEE Transactions on Pattern Analysis and Machine Intelligence 38, 11 (2016), 2342–2344.
[20] M. Shao, S. X., and Fu, Y. Genealogical face recognition based on ub kinface database. In Proc. IEEE CVPR Workshop on Biometrics (BIOM) (2011).
[21] Mukherjee, M., and Meenpal, T. Binary cross coupled discriminant analysis for visual kinship verification. Signal Processing: Image Communication 108 (2022), 116829.
[22] Nandy, A., and Mondal, S. Kinship verification using deep siamese convolutional neural network. pp. 1–5.
[23] Park, S., Lee, J., Lee, P., Hwang, S., Kim, D., and Byun, H. Fair contrastive learning for facial attribute classification, 2022.
[24] Qin, X., Liu, D., and Wang, D. A novel factor analysis-based metric learning method for kinship verification. Multimedia Tools Appl. 81, 8 (mar 2022), 11049–11070.
[25] Raff, E., and Sylvester, J. Gradient reversal against discrimination, 2018.
[26] Robinson, J. P., Qin, C., Shao, M., Turk, M. A., Chellappa, R., and Fu, Y. The 5th recognizing families in the wild data challenge: Predicting kinship from faces, 2021.
[27] Robinson, J. P., Shao, M., Wu, Y., and Fu, Y. Families in the wild (fiw): Large-scale kinship image database and benchmarks. In Proceedings of the 2016 ACM on Multimedia Conference (2016), ACM, pp. 242–246.
[28] Robinson, J. P., Yin, Y., Khan, Z., Shao, M., Xia, S., Stopa, M., Timoner, S., Turk, M. A., Chellappa, R., and Fu, Y. Recognizing families in the wild (rfiw): The 4th edition. In 2020 15th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2020) (2020), pp. 857–862.
[29] S. Xia, M. Shao, J. L., and Fu, Y. Understanding kin relationships in a photo. IEEE Transactions on Multimedia 14, 4 (2012), 1046–1056
[30] S. Xia, M. S., and Fu, Y. Kinship verification through transfer learning. In Proc. International Joint Conferences on Artificial Intelligence (IJCAI) (2011), pp. 2539–2544.
[31] Shadrikov, A. Achieving better kinship recognition through better baseline. In 2020 15th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2020) (nov 2020), IEEE.
[32] Su, W.-T., Chen, M.-H., Wang, C.-Y., Lai, S.-H., and Chen, T. P.-C. Kinship representation learning with face componential relation, 2023.
[33] Taigman, Y., Yang, M., Ranzato, M., and Wolf, L. Deepface: Closing the gap to human-level performance in face verification. In 2014 IEEE Conference on Computer Vision and Pattern Recognition (2014), pp. 1701–1708.
[34] Vartan, S. Racial bias found in a major health care risk algorithm, 2019. October 24, 2019.
[35] Wang, F., and Liu, H. Understanding the behaviour of contrastive loss, 2021.
[36] Wang, F.-E., Wang, C.-Y., Sun, M., and Lai, S.-H. Mixfairface: Towards ultimate fairness via mixfair adapter in face recognition, 2022.
[37] Wang, H., Wang, Y., Zhou, Z., Ji, X., Gong, D., Zhou, J., Li, Z., and Liu, W. Cosface: Large margin cosine loss for deep face recognition, 2018.
[38] Wang, M., Deng, W., Hu, J., Tao, X., and Huang, Y. Racial faces in the wild: Reducing racial bias by information maximization adaptation network. In The IEEE International Conference on Computer Vision (ICCV) (October 2019).
[39] Wang, M., Zhang, Y., and Deng, W. Meta balanced network for fair face recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 44, 11 (2022), 8433–8448.
[40] Wang, Z., Dong, X., Xue, H., Zhang, Z., Chiu, W., Wei, T., and Ren, K. Fairness-aware adversarial perturbation towards bias mitigation for deployed deep models, 2022.
[41] Woo, S., Park, J., Lee, J.-Y., and Kweon, I. S. Cbam: Convolutional block attention module, 2018.
[42] Wu, X., Feng, X., Cao, X., Xu, X., Hu, D., López, M. B., and Liu, L. Facial kinship verification: A comprehensive review and outlook. Int. J. Comput. Vision 130, 6 (jun 2022), 1494–1525.
[43] Xu, X., Huang, Y., Shen, P., Li, S., Li, J., Huang, F., Li, Y., and Cui, Z. Consistent instance false positive improves fairness in face recognition, 2021.
[44] Yan, H., and Wang, S. Learning part-aware attention networks for kinship verification. Pattern Recognition Letters 128 (2019), 169–175.
[45] Yu, J., Li, M., Hao, X., and Xie, G. Deep fusion siamese network for automatic kinship verification, 2020.
[46] Zhang, K., Zhang, Z., Li, Z., and Qiao, Y. Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Signal Processing Letters 23, 10 (oct 2016), 1499–1503.
[47] Zhang, X., XU, M., Zhou, X., and Guo, G. Supervised contrastive learning for facial kinship recognition. In 2021 16th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2021) (2021), pp. 01–05.