不同身份的人具有不同的面貌，對於相同表情的詮釋方式也隨之不同。此一受身份影響的特性，大大增加自動化表情辨識的困難度。在本文之中，我們提出雙子空間的非負圖形嵌入表示法，將臉部表情影像使用身份以及表情子空間加以描述。其中，身份子空間描述和身份有關的臉部變異；而表情子空間則描述不受身份影響的表情變異。透過本方法，我們可將輸入的臉部影像，透過對應的非負基底，分解為身份部份和表情部份。除此之外，對於相同表情的類別內變異，我們重新設計表情子空間的圖形限制以解決此一問題。我們透過CK+、JAFFE和TFEID三種人臉表情資料庫進行驗證；實驗結果顯示，我們的方法均優於現有的表情辨識方法。

Different persons usually exhibit various appearance changes when posing the same expression. This person-dependent behavior often complicates automatic facial expression recognition. In this thesis, to address the person-independent expression recognition problem, we propose a Dual Subspace Nonnegative Graph Embedding (DSNGE) to represent expressive images using two subspaces: identity and expression subspaces. The identity subspace characterizes person-dependent appearance variations; whereas the expression subspace characterizes person-independent expression variations. With DSNGE, we decompose each facial image into an identity part and an expression part represented by their corresponding nonnegative bases. We also address the intra-class variations issue in the expression recognition problem, and further devise a graph-embedding constraint on the expression subspace to tackle this problem. Our experimental results show that the proposed DSNGE outperforms other graph-based nonnegative factorization methods and existing expression recognition methods on CK+, JAFFE and TFEID databases.

[1] R. Valenti, A. Jaimes, and N. Sebe, “Sonify Your Face: Facial Expressions for Sound Generation,” ACM Int’l. Conf. Multimedia, 2010.
[2] S. Marcos, J. Gomez-Garcia-Bermejo, E. Zalama, and J. Lopez, “Nonverbal Communication with a Multimodal Agent via Facial Expression Recognition,” IEEE Int’l. Conf. Robotics and Automation, pp. 1199-1204, 2011.
[3] K. Mu, J. Tao, and M. Yang, “Animating a Chinese Interactive Virtual Character,” IEEE Int’l. Machine Learning for Signal Processing Workshop, 2011.
[4] P. N. Belhumeur, J. P. Hespanha, and D. J. Kriegman, “Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, pp. 711-721, 1997.
[5] J. Zou, Q. Ji, and G. Nagy, “A Comparative Study of Local Matching Approach for Face Recognition,” IEEE Trans. Image Processing, vol. 16, no. 10, pp. 2617-2628, 2007.
[6] G. Zhao, and M. Pietikainen, “Dynamic Texture Recognition Using Local Binary Patterns with an Application to Facial Expressions,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, pp. 915-928, 2007.
[7] C. Shan, S. Gong, and P. W. McOwan, “Facial Expression Recognition Based on Local Binary Patterns: A Comprehensive Study,” Image and Vision Computing, vol. 27, no. 6, pp. 803-816, 2009.
[8] T. Jabid, M. H. Kabir, and O. Chae,“Facial Expression Recognition Using Local Directional Pattern (LDP),” IEEE Int’l Conf. Image Processing, pp. 1605-1608, 2010.
[9] A. R. Rivera, R. Castillo, and O. Chae, “Local Directional Number Pattern for Face Analysis: Face and Expression Recognition, ” IEEE Trans. Image Processing, vol. 22, no. 5, pp. 1740-1752, 2013.
[10] T. F. Cootes, C. J. Taylor, D. H. Cooper, and J. Graham, “Active Shape Models - Their Training and Applications,” Computer Vision and Image Understanding, vol. 61, pp. 38-59, 1995.
[11] T. F. Cootes, G. J. Edwards, and C. J. Taylor, “Active Appearance Models,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 6, pp. 681-5, 2001.
[12] S. B. Gokturk, J.Y. Bouguet, C. Tomasi, and B. Girod, “Model-Based Face Tracking for View Independent Facial Expression Recognition,” Proc. IEEE Int’l. Conf. Face and Gesture Recognition, pp. 272-278, 2002.
[13] B. Schölkopf. Support Vector Learning. R. Oldenbourg Verlag, Munich, 1997.
[14] Y. L. Tian, T. Kanade, and J. F. Cohn, “Recognizing Action Units for Facial Expression Analysis,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 2, pp. 97-115, 2001.
[15] D. D. Lee, and H. S. Seung, “Learning the Parts of Objects by Non-negative Matrix Factorization,” Nature, vol. 401, pp.788–791, 1999.
[16] D. D. Lee, and H. S. Seung. “Algorithms for non-negative matrix factorization,” Advances in Neural Information Processing Systems, pages 556–562, 2001.
[17] D. Cai, X. He, X. Wu, and J. Han, “Non-negative Matrix Factorization on Manifold,” IEEE Int’l. Conf. Data Mining, pp. 63-72, 2008.
[18] R. Zhi, M. Flierl, Q. Ruan, and W. B. Kleijn, “Graph-Preserving Sparse Nonnegative Matrix Factorization with Application to Facial Expression Recognition,” IEEE Trans. SMC-Part B: Cybernetics, vol. 41, no. 1, pp. 38-52, 2011.
[19] J. Yang, S. Yan, Y. Fu, X. Li, and T. Huang, “Non-negative Graph Embedding,” IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[20] X. Liu, S. Yan, and H. Jin, “Projective Nonnegative Graph Embedding,” IEEE Trans. Image Processing, vol. 19, no. 5, pp. 1126-1137, 2010.
[21] H. Zhang, Z. J. Zha, S. Yan, M. Wang, and T. S. Chua, “Robust Non-negative Graph Embedding: Towards Noisy Data, Unreliable Graphs, and Noisy Labels,” IEEE Conf. Computer Vision and Pattern Recognition, 2012.
[22] Yi-Han Tu, and Chiou-Ting Hsu, “Dual subspace nonnegative matrix factorization for person-invariant facial expression recognition,” IEEE Int’l Conf. Pattern Recognition, pp. 2391-2394, 2012.
[23] S. Yan, D. Xu, B. Zhang, Q. Yang, H. Zhang, and S. Lin, “Graph Embedding and Extensions: A General Framework for Dimensionality Reduction,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, pp. 40-51, 2007.
[24] O. Rudovic, I. Patras, and M. Pantic, “Coupled Gaussian Process Regression for Pose-Invariant Facial Expression Recognition,” European Conference on Computer Vision, vol. 6312, pp. 350-363, 2010.
[25] G. Zhao and M. Pietikainen, “Dynamic Texture Recognition Using Local Binary Patterns with an Application to Facial Expressions,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, pp. 915-928, 2007.
[26] B. Fasel and J. Luttin, “Automatic Facial Expression Analysis: Survey,” Pattern Recognition, vol. 36, no. 1, pp. 259-275, 2003.
[27] P. Martins and J. Batista, “Identity and Expression Recognition on Low Dimensional Manifolds,” IEEE Int’l. Conf. Image Processing, 2009.
[28] Kai-Tai Song and Yi-Wen Chen, “A Design for Integrated Face and Facial Expression Recognition,” IEEE Conf. Industrial Electronics Society, 2012.
[29] G. Donato, M. S. Bartlett, J. C. Hager, P. Ekman, and T. J. Sejnowski, “Classifying Facial Actions,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 21, no. 10, pp. 974-989, 1999.
[30] S. Zafeiriou, and M. Petrou, “Sparse Representations For Facial Expressions Recognition via l1 optimization,” IEEE Conf. Computer Vision and Pattern Recognition, pp.32-39, 2010.
[31] P. Lucey, J. F. Cohn, T. Kanade, J. Saragih, Z. Ambadar, and I. Matthews, “The Extended Cohn-Kanade Dataset (CK+): A complete expression dataset for action unit and emotion-specified expression,” IEEE Conf. Computer Vision and Pattern Recognition Workshops, pp. 94-101, 2010.
[32] M. J. Lyons, S. Akamatsu, M. Kamachi, and J. Gyoba, “Coding Facial Expressions with Gabor Wavelets,” IEEE Int’l. Conf. Automatic Face and Gesture Recognition, pp. 200-205, 1998.
[33] The Taiwanese Facial Expression Image Database (TFEID), http://bml.ym.edu.tw/~download/html/news.htm
[34] E. Candés, and T. Tao, “Decoding by linear programming,” IEEE Trans. Information Theory, vol. 51, no. 12, pp.4203-4215, 2005.
[35] J. Wright, Y. Ma, J. Mairal, G. Sapiro, T. Huang, and S. Yan “Sparse Representation for Computer Vision and Pattern Recognition, ” Proceedings of IEEE, 2009.
[36] Y. Peng, A. Ganesh, J. Wright, W. Xu, and Y. Ma, “RASL: Robust Alignment by Sparse and Low-Rank Decomposition for Linearly Correlated Images,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 34, no. 11, pp. 2233-2246, 2012.
[37] S. Yang, and B. Bhanu, “Understanding Discrete Facial Expressions in Video Using an Emotion Avatar Image,” IEEE Trans. SMC-B, vol.42, no.4, pp.980-992, 2012.
[38] Y. Li, S. Wang, Y. Zhao, and Q. Ji, “Simultaneous Facial Feature Tracking and Facial Expression Recognition,” IEEE Trans. Image Processing, vol.22, no.7, pp. 2559-2573, 2013.
[39] M. W. Huang, Z. W. Wang, and Z. L. Ying , “A new method for facial expression recognition based on sparse representation plus LBP,” International Congress on Image and Signal Processing, pp.1750-1754, 2010.