簡易檢索 / 詳目顯示

研究生: 張楚翎
Chang, Chu-Ling
論文名稱: 通過使用三元組挖掘以及補丁增強的人臉反欺騙域泛化
Domain Generalization for Face Anti-spoofing via Patch-wise Augmentation and Triplet Mining
指導教授: 許秋婷
Hsu, Chiou-Ting
口試委員: 邵皓強
Shao, Hao-Chiang
邵皓強
Shao, Hao-Chiang
學位類別: 碩士
Master
系所名稱: 電機資訊學院 - 資訊系統與應用研究所
Institute of Information Systems and Applications
論文出版年: 2022
畢業學年度: 111
語文別: 英文
論文頁數: 30
中文關鍵詞: 人臉防偽域泛化補丁增強三源組挖掘
外文關鍵詞: face anti-spoofing, domain generalization, patch-wise augmentation, triplet mining
相關次數: 點閱:2下載:0
分享至:
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報
  • 現在已經有許多人臉反欺騙方法來應對各種偽冒攻擊,並取得了不錯的性能。
    儘管現有方法在數據集內場景上運行良好,但在處理看不見的場景時性能下降。
    因此,在本篇論文中,我們專注在人臉反欺騙問題的域泛化,並提出了一種新的
    圖片增強策略來增加數據多樣性。與幾種區域隨機遮擋的區域丟失策略不同,本
    篇所提出的增強策略增加了激活值較低的補丁區域以生成更具挑戰性的圖像。此
    外,我們將域對抗學習模塊與三元組挖掘相結合,以提取域不變的活性特徵,並
    為看不見的域學習更好的類邊界。此外,還進一步結合了像素級監督,以輔助人
    臉反欺騙模型通過估計像素因素來學習判別特徵。在幾個基準數據集上的廣泛實
    驗結果證明了我們方法的有效性,並顯示出對最先進的競爭對手的顯著改進。


    Many face anti-spoofing methods have been developed to counter diverse presentation
    attacks and achieved promising performance. Despite existing methods
    work well on the intra-dataset scenarios, the performance drops when dealing with
    the unseen scenarios. In this paper, we focus on domain generalization for face antispoofing
    (FAS) problem and propose a novel augmentation strategy to increase the
    data diversity. Unlike several regional dropout strategies, where the region are
    random occluded, the proposed augmentation strategy increase the patch area with
    lower activation value to generate more challenging images. Moreover, we combine
    Domain Adversarial Learning Module (DALM) with triplet mining to extract
    domain-invariant liveness features and learn a better class boundary for unseen domains.
    Additionally, the pixel-level supervision is further incorporated to guide the
    face anti-spoofing model to learn discriminative features by estimating pixel-wise
    factors. Extensive experimental results on several benchmark datasets prove that
    our method show significant improvement over the state-of-the-art competitors.

    摘要i Abstract ii Acknowledgements 1 Introduction 1 2 Related Work 4 2.1 Face Anti-spoofing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 2.1.1 Depth-based Methods . . . . . . . . . . . . . . . . . . . . . . . . . . 4 2.1.2 Temporal-based Methods . . . . . . . . . . . . . . . . . . . . . . . . . 5 2.2 Data Augmentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 2.3 Domain Generalization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 3 Method 7 3.1 Problem Definition and Notations . . . . . . . . . . . . . . . . . . . . . . . . 7 3.2 Data Augmentation Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 3.2.1 Obtain the Combined Face Image and Activation Map . . . . . . . . . 9 3.2.2 Crop and Paste . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 3.2.3 Augmented Images of Two Different Cases . . . . . . . . . . . . . . . 11 3.3 Domain Adversarial Learning Module . . . . . . . . . . . . . . . . . . . . . . 12 3.4 Triplet Mining . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 3.5 Pixel-level Supervision . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 3.6 Total Loss . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 4 Experiments 16 4.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 4.2 Dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 4.3 Evaluation metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 4.4 Implementation Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 4.5 Ablation Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 4.5.1 Experiment on Leave-one-out Setting . . . . . . . . . . . . . . . . . . 21 4.5.2 Experiment on Limited Source Domains . . . . . . . . . . . . . . . . . 22 4.5.3 Visualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 4.6 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 5 Conclusion 25

    [1] Y. Liu, A. Jourabloo, and X. Liu, “Learning deep models for face anti-spoofing: Binary
    or auxiliary supervision,” in Proceedings of the IEEE conference on computer vision and
    pattern recognition, pp. 389–398, 2018.
    [2] A. Jourabloo, Y. Liu, and X. Liu, “Face de-spoofing: Anti-spoofing via noise modeling,”
    in Proceedings of the European Conference on Computer Vision (ECCV), pp. 290–306,
    2018.
    [3] Y. Liu, J. Stehouwer, A. Jourabloo, and X. Liu, “Deep tree learning for zero-shot face antispoofing,”
    in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern
    Recognition, pp. 4680–4689, 2019.
    [4] X. Yang, W. Luo, L. Bao, Y. Gao, D. Gong, S. Zheng, Z. Li, and W. Liu, “Face antispoofing:
    Model matters, so does data,” in Proceedings of the IEEE/CVF Conference on
    Computer Vision and Pattern Recognition, pp. 3507–3516, 2019.
    [5] Z. Wang, Z. Yu, C. Zhao, X. Zhu, Y. Qin, Q. Zhou, F. Zhou, and Z. Lei, “Deep spatial gradient
    and temporal depth learning for face anti-spoofing,” in Proceedings of the IEEE/CVF
    Conference on Computer Vision and Pattern Recognition, pp. 5042–5051, 2020.
    [6] Z. Yu, X. Li, X. Niu, J. Shi, and G. Zhao, “Face anti-spoofing with human material perception,”
    in European Conference on Computer Vision, pp. 557–575, Springer, 2020.
    [7] K.-Y. Zhang, T. Yao, J. Zhang, Y. Tai, S. Ding, J. Li, F. Huang, H. Song, and L. Ma,
    “Face anti-spoofing via disentangled representation learning,” in European Conference
    on Computer Vision, pp. 641–657, Springer, 2020.
    [8] Y. Liu, J. Stehouwer, and X. Liu, “On disentangling spoof trace for generic face antispoofing,”
    in European Conference on Computer Vision, pp. 406–422, Springer, 2020.
    [9] H. Feng, Z. Hong, H. Yue, Y. Chen, K. Wang, J. Han, J. Liu, and E. Ding, “Learning
    generalized spoof cues for face anti-spoofing,” arXiv preprint arXiv:2005.03922, 2020.
    [10] K.-Y. Zhang, T. Yao, J. Zhang, S. Liu, B. Yin, S. Ding, and J. Li, “Structure destruction and
    content combination for face anti-spoofing,” in 2021 IEEE International Joint Conference
    on Biometrics (IJCB), pp. 1–6, IEEE, 2021.
    [11] Y.-C. Wang, C.-Y. Wang, and S.-H. Lai, “Disentangled representation with dual-stage feature
    learning for face anti-spoofing,” in Proceedings of the IEEE/CVF Winter Conference
    on Applications of Computer Vision, pp. 1955–1964, 2022.
    [12] S. Shankar, V. Piratla, S. Chakrabarti, S. Chaudhuri, P. Jyothi, and S. Sarawagi, “Generalizing
    across domains via cross-gradient training,” arXiv preprint arXiv:1804.10745,
    2018.
    [13] R. Volpi, H. Namkoong, O. Sener, J. C. Duchi, V. Murino, and S. Savarese, “Generalizing
    to unseen domains via adversarial data augmentation,” Advances in neural information
    processing systems, vol. 31, 2018.
    [14] M. Mancini, Z. Akata, E. Ricci, and B. Caputo, “Towards recognizing unseen categories
    in unseen domains,” in European Conference on Computer Vision, pp. 466–483, Springer,
    2020.
    [15] T. DeVries and G. W. Taylor, “Improved regularization of convolutional neural networks
    with cutout,” arXiv preprint arXiv:1708.04552, 2017.
    [16] G. Ghiasi, T.-Y. Lin, and Q. V. Le, “Dropblock: A regularization method for convolutional
    networks,” Advances in neural information processing systems, vol. 31, 2018.
    [17] S. Yun, D. Han, S. J. Oh, S. Chun, J. Choe, and Y. Yoo, “Cutmix: Regularization strategy
    to train strong classifiers with localizable features,” in Proceedings of the IEEE/CVF
    international conference on computer vision, pp. 6023–6032, 2019.
    [18] R. Shao, X. Lan, J. Li, and P. C. Yuen, “Multi-adversarial discriminative deep domain
    generalization for face presentation attack detection,” in Proceedings of the IEEE/CVF
    Conference on Computer Vision and Pattern Recognition, pp. 10023–10031, 2019.
    [19] G. Wang, H. Han, S. Shan, and X. Chen, “Cross-domain face presentation attack detection
    via multi-domain disentangled representation learning,” in Proceedings of the IEEE/CVF
    Conference on Computer Vision and Pattern Recognition, pp. 6678–6687, 2020.
    [20] Y. Jia, J. Zhang, S. Shan, and X. Chen, “Single-side domain generalization for face antispoofing,”
    in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern
    Recognition, pp. 8484–8493, 2020.
    [21] R. Shao, X. Lan, and P. C. Yuen, “Regularized fine-grained meta face anti-spoofing,” in
    Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 11974–11981,
    2020.
    [22] S. Saha, W. Xu, M. Kanakis, S. Georgoulis, Y. Chen, D. P. Paudel, and L. Van Gool,
    “Domain agnostic feature learning for image and video based face anti-spoofing,” in Proceedings
    of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops,
    pp. 802–803, 2020.
    [23] Z. Yu, J. Wan, Y. Qin, X. Li, S. Z. Li, and G. Zhao, “Nas-fas: Static-dynamic central
    difference network search for face anti-spoofing,” IEEE transactions on pattern analysis
    and machine intelligence, vol. 43, no. 9, pp. 3005–3023, 2020.
    [24] S. Liu, K.-Y. Zhang, T. Yao, M. Bi, S. Ding, J. Li, F. Huang, and L. Ma, “Adaptive normalized
    representation learning for generalizable face anti-spoofing,” in Proceedings of
    the 29th ACM International Conference on Multimedia, pp. 1469–1477, 2021.
    [25] J. Wang, J. Zhang, Y. Bian, Y. Cai, C. Wang, and S. Pu, “Self-domain adaptation for face
    anti-spoofing,” arXiv preprint arXiv:2102.12129, 2021.
    [26] Z. Chen, T. Yao, K. Sheng, S. Ding, Y. Tai, J. Li, F. Huang, and X. Jin, “Generalizable
    representation learning for mixture domain face anti-spoofing,” arXiv preprint
    arXiv:2105.02453, 2021.
    [27] A. Liu, Z. Tan, J. Wan, Y. Liang, Z. Lei, G. Guo, and S. Z. Li, “Face anti-spoofing via
    adversarial cross-modality translation,” IEEE Transactions on Information Forensics and
    Security, vol. 16, pp. 2759–2772, 2021.
    [28] Z. Huang, H. Wang, E. P. Xing, and D. Huang, “Self-challenging improves cross-domain
    generalization,” in European Conference on Computer Vision, pp. 124–140, Springer,
    2020.
    [29] Y. Atoum, Y. Liu, A. Jourabloo, and X. Liu, “Face anti-spoofing using patch and depthbased
    cnns,” in 2017 IEEE International Joint Conference on Biometrics (IJCB), pp. 319–
    328, 2017.
    [30] PeiKai-Huang, M.-C. Chin, and C.-T. Hsu, “Face anti-spoofing via robust auxiliary estimation
    and discriminative feature learning,” in Asian Conference on Pattern Recognition,
    Springer, 2021.
    [31] L. Sun, G. Pan, Z. Wu, and S. Lao, “Blinking-based live face detection using conditional
    random fields,” in International Conference on Biometrics, pp. 252–260, Springer, 2007.
    [32] K. Kollreider, H. Fronthaler, M. I. Faraj, and J. Bigun, “Real-time face detection and
    motion analysis with application in “liveness”assessment,” IEEE Transactions on Information
    Forensics and Security, vol. 2, no. 3, pp. 548–558, 2007.
    [33] S. Liu, P. C. Yuen, S. Zhang, and G. Zhao, “3d mask face anti-spoofing with remote photoplethysmography,”
    in European Conference on Computer Vision, pp. 85–100, Springer,
    2016.
    [34] X. Li, J. Komulainen, G. Zhao, P.-C. Yuen, and M. Pietikäinen, “Generalized face antispoofing
    by detecting pulse from face videos,” in 2016 23rd International Conference on
    Pattern Recognition (ICPR), pp. 4244–4249, IEEE, 2016.
    [35] S.-Q. Liu, X. Lan, and P. C. Yuen, “Remote photoplethysmography correspondence feature
    for 3d mask face presentation attack detection,” in Proceedings of the European Conference
    on Computer Vision (ECCV), pp. 558–573, 2018.
    [36] K. Zhou, Y. Yang, T. Hospedales, and T. Xiang, “Learning to generate novel domains
    for domain generalization,” in European conference on computer vision, pp. 561–578,
    Springer, 2020.
    [37] K. Zhou, Y. Yang, T. Hospedales, and T. Xiang, “Deep domain-adversarial image generation
    for domain generalisation,” in Proceedings of the AAAI Conference on Artificial
    Intelligence, vol. 34, pp. 13025–13032, 2020.
    [38] K. Kumar Singh and Y. Jae Lee, “Hide-and-seek: Forcing a network to be meticulous for
    weakly-supervised object and action localization,” in Proceedings of the IEEE International
    Conference on Computer Vision, pp. 3524–3533, 2017.
    [39] Z. Zhong, L. Zheng, G. Kang, S. Li, and Y. Yang, “Random erasing data augmentation,” in
    Proceedings of the AAAI conference on artificial intelligence, vol. 34, pp. 13001–13008,
    2020.
    [40] J. Choe and H. Shim, “Attention-based dropout layer for weakly supervised object localization,”
    in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern
    Recognition, pp. 2219–2228, 2019.
    [41] N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, “Dropout: a
    simple way to prevent neural networks from overfitting,” The journal of machine learning
    research, vol. 15, no. 1, pp. 1929–1958, 2014.
    [42] B. Zhou, A. Khosla, A. Lapedriza, A. Oliva, and A. Torralba, “Learning deep features for
    discriminative localization,” in Proceedings of the IEEE conference on computer vision
    and pattern recognition, pp. 2921–2929, 2016.
    [43] R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra, “Grad-cam:
    Visual explanations from deep networks via gradient-based localization,” in Proceedings
    of the IEEE international conference on computer vision, pp. 618–626, 2017.
    [44] Y. Ganin and V. Lempitsky, “Unsupervised domain adaptation by backpropagation,” in
    International conference on machine learning, pp. 1180–1189, PMLR, 2015.
    [45] Z. Boulkenafet, J. Komulainen, L. Li, X. Feng, and A. Hadid, “Oulu-npu: A mobile face
    presentation attack database with real-world variations,” in 2017 12th IEEE international
    conference on automatic face & gesture recognition (FG 2017), pp. 612–618, IEEE, 2017.
    [46] Z. Zhang, J. Yan, S. Liu, Z. Lei, D. Yi, and S. Z. Li, “A face antispoofing database with
    diverse attacks,” in 2012 5th IAPR international conference on Biometrics (ICB), pp. 26–
    31, IEEE, 2012.
    [47] D. Wen, H. Han, and A. K. Jain, “Face spoof detection with image distortion analysis,”
    IEEE Transactions on Information Forensics and Security, vol. 10, no. 4, pp. 746–761,
    2015.
    [48] I. Chingovska, A. Anjos, and S. Marcel, “On the effectiveness of local binary patterns
    in face anti-spoofing,” in 2012 BIOSIG-proceedings of the international conference of
    biometrics special interest group (BIOSIG), pp. 1–7, IEEE, 2012.
    [49] A. Anjos and S. Marcel, “Counter-measures to photo attacks in face recognition: a public
    database and a baseline,” in 2011 international joint conference on Biometrics (IJCB),
    pp. 1–7, IEEE, 2011.
    [50] J. Määttä, A. Hadid, and M. Pietikäinen, “Face spoofing detection from single images using
    micro-texture analysis,” in 2011 international joint conference on Biometrics (IJCB),
    pp. 1–7, IEEE, 2011.
    [51] T. d. Freitas Pereira, J. Komulainen, A. Anjos, J. M. De Martino, A. Hadid, M. Pietikäinen,
    and S. Marcel, “Face liveness detection using dynamic texture,” EURASIP Journal on
    Image and Video Processing, vol. 2014, no. 1, pp. 1–15, 2014.
    [52] Z. Boulkenafet, J. Komulainen, and A. Hadid, “Face spoofing detection using colour texture
    analysis,” IEEE Transactions on Information Forensics and Security, vol. 11, no. 8,
    pp. 1818–1830, 2016.
    [53] D. Li, Y. Yang, Y.-Z. Song, and T. M. Hospedales, “Learning to generalize: Meta-learning
    for domain generalization,” in Thirty-Second AAAI Conference on Artificial Intelligence,
    2018.
    [54] Y. Balaji, S. Sankaranarayanan, and R. Chellappa, “Metareg: Towards domain generalization
    using meta-regularization,” Advances in neural information processing systems,
    vol. 31, 2018.
    [55] H. Li, S. J. Pan, S. Wang, and A. C. Kot, “Domain generalization with adversarial feature
    learning,” in Proceedings of the IEEE conference on computer vision and pattern
    recognition, pp. 5400–5409, 2018.
    [56] Y. He, A. Carass, L. Zuo, B. E. Dewey, and J. L. Prince, “Self domain adapted network,”
    in International Conference on Medical Image Computing and Computer-Assisted Intervention,
    pp. 437–446, Springer, 2020.
    [57] Y. Qin, Z. Yu, L. Yan, Z. Wang, C. Zhao, and Z. Lei, “Meta-teacher for face anti-spoofing,”
    IEEE transactions on pattern analysis and machine intelligence, 2021.
    [58] Z. Wang, Z. Wang, Z. Yu, W. Deng, J. Li, T. Gao, and Z. Wang, “Domain generalization
    via shuffled style assembly for face anti-spoofing,” in Proceedings of the IEEE/CVF
    Conference on Computer Vision and Pattern Recognition, pp. 4123–4133, 2022.

    QR CODE