不同身分的人有著不同的面貌，然而這項因素卻大幅增加了自動化表情辨識的難度。雖然我們可以藉由中性臉減少不同身分的人其面貌上之變異，在實際應用上，中性臉之取得並非那麼容易。為了能夠將與身分相關的面貌因素從表情影像中移除以擷取出只與表情有關之特徵，我們提出了一個創新的非負矩陣分解技術「雙子空間非負矩陣分解法」將人臉表情影像分解成兩個部份：身分與表情。身分部分用來描述人臉影像間與身分相關的變異；表情部分則用來描述表情特徵，且幾乎不包含與身分相關之資訊。表情辨識實驗中，我們提出的方法其辨識率優於大多數現有的方法，這也表示我們的雙子空間非負矩陣分解成功地將表情因素與身分因素自人臉影像中分離。另外，我們還實驗了在表情影響下之身分辨識，藉由雙子空間非負矩陣分解，我們大幅提高了辨識率。

Person-dependent appearance changes tend to increase difficulties in automatic facial expression recognition. Although one can use neutral face images to reduce the
personal variations, acquisition of neutral face images may not always be possible in real cases. In order to remove the person-dependent influence from expressive images,
we propose a novel nonnegative matrix factorization, called dual subspace nonnegative matrix factorization (DSNMF), to decompose facial images into two parts: identity and expression parts. The identity part should characterize
person-dependent variations, while the expression part should characterize person-invariant expression features. Our experimental results show that the proposed method significantly outperforms existing approaches on the CK+, JAFFE and TFEID expression databases. Furthermore, we also conducted DSNMF for face recognition across expression under single sample per person (SSPP) condition and the recognition rate is greatly improved by DSNMF.

[1]R. Valenti, A. Jaimes, and N. Sebe, “Sonify Your Face: Facial Expressions for Sound Generation,” ACM Intl. Conf. Multimedia, 2010.
[2]S. Marcos, J. Gómez-García-Bermejo, E. Zalama and J. López, “Nonverbal Communication with a Multimodal Agent via Facial Expression Recognition,” IEEE Intl. Conf. Robotics and Automation, pp. 1199-1204, 2011.
[3]K. Mu, J. Tao, and M. Yang, “Animating a Chinese Interactive Virtual Character,” IEEE Intl. Machine Learning for Signal Processing Workshop, 2011.
[4]O. Rudovic, I. Patras, and M. Pantic, “Coupled Gaussian Process Regression for Pose-Invariant Facial Expression Recognition,” European Conf. Computer Vision, vol. 6312, pp. 350-363, 2010.
[5]W. Zheng, H. Tang, Z. Lin, and T. S. Huang, “Emotion Recognition from Arbitrary View Facial Images,” European Conf. Computer Vision, vol. 6316, pp. 490-503, 2010.
[6]G. Zhao and M. Pietikainen, “Dynamic Texture Recognition Using Local Binary Patterns with an Application to Facial Expressions,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, pp. 915-928, 2007.
[7]C. Shan, S. Gong, and P. W. McOwan, “Facial Expression Recognition Based on Local Binary Patterns: A Comprehensive Study,” Image and Vision Computing, vol. 27, no. 6, pp. 803-816, 2009.
[8]B. Fasel and J. Luttin, “Automatic Facial Expression Analysis: Survey,” Pattern Recognition, vol. 36, no. 1, pp. 259-275, 2003.
[9]P. Martins and J. Batista, “Identity and Expression Recognition on Low Dimensional Manifolds,” IEEE Intl. Conf. Image Processing, pp. 3341-3344, 2009.
[10]Kai-Tai Song and Yi-Wen Chen, “A Design for Integrated Face and Facial Expression Recognition,” IEEE Conf. Industrial Electronics Society, pp.4306-4311, 2012.
[11]G. Donato, M. S. Bartlett, J. C. Hager, P. Ekman, and T. J. Sejnowski, “Classifying Facial Actions,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 21, no. 10, pp. 974-989, 1999.
[12]S. Zafeiriou, and M. Petrou, “Sparse Representations For Facial Expressions Recognition via L1 optimization,” IEEE Conf. Computer Vision and Pattern Recognition Workshops, 2010.
[13]Y. Tian, T. Kanade, J. Cohn, “Facial expression analysis,” Handbook of Face Recognition, Springer, 2005. (Chapter 11)
[14]C. Martin, U. Werner, and H. M. Gross, “A Real-Time Facial Expression Recognition System Based on Active Appearance Models Using Gray Images and Edge Images,” IEEE Conf. on Automatic Face and Gesture Recognition, 2008.
[15]S. Park, J. Shin, and D. Kim, “Facial Expression Analysis with Facial Expression Deformation,” Int. Conf. Pattern Recognition (ICPR), 2008.
[16]S.B. Gokturk, J.Y. Bouguet, C. Tomasi, and B. Girod, “Model-Based Face Tracking for View Independent Facial Expression Recognition,” Proc. IEEE Int. Conf. Face and Gesture Recognition, pp. 272-278, 2002.
[17]T. F. Cootes, G. J. Edwards, and C. J. Taylor, “Active Appearance Models,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 6, pp. 681-5, 2001.
[18]Z. Zhang, M. Lyons, M. Schuster, and S. Akamatsu, “Comparison Between Geometry-Based and Gabor-Wavelets-Based Facial Expression Recognition Using Multi-Layer Perceptron,” IEEE Proc. Automatic Face and Gesture Recognition, pp. 454-459, 1998.
[19]P. N. Belhumeur, J. P. Hespanha, and D. J. Kriegman, “Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, pp. 711-721, 1997.
[20]S. Dubuisson, F. Davoine, and M. Masson, “A solution for facial expression representation and recognition,” Signal Processing: Image Communication, vol. 17, pp. 657-673, 2002.
[21]J. Zou, Q. Ji, and G. Nagy, “A Comparative Study of Local Matching Approach for Face Recognition,” IEEE Trans. Image Processing, vol. 16, no. 10, pp. 2617-2628, 2007.
[22]B. Heisele, P. Ho, J. Wu, and T. Poggio, “Face Rrecognition: Component-based Versus Global Approaches,” Computer Vision and Image Understanding, vol. 91, pp. 6-12, 2003.
[23]D. D. Lee and H. S. Seung, “Learning the Parts of Objects by Non-negative Matrix Factorization,” Nature, vol. 401, pp.788–791, 1999.
[24]P. Lucey, J. F. Cohn, T. Kanade, J. Saragih, Z. Ambadar, and I. Matthews, “The Extended Cohn-Kanade Dataset (CK+): A complete expression dataset for action unit and emotion-specified expression,” IEEE Conf. Computer Vision and Pattern Recognition Workshops, pp. 94-101, 2010.
[25]M. J. Lyons, S. Akamatsu, M. Kamachi, and J. Gyoba, “Coding Facial Expressions with Gabor Wavelets,” IEEE Intl. Conf. Automatic Face and Gesture Recognition, pp. 200-205, 1998.
[26]The Taiwanese Facial Expression Image Database (TFEID), http://bml.ym.edu.tw/~download/html/news.htm
[27]D. D. Lee and H. S. Seung, “Algorithms for non-negative matrix factorization,” Advances in Neural Information Processing Systems, pp. 556–562, 2000.
[28]D. Cai, X. He, X. Wu, and J. Han, “Non-negative Matrix Factorization on Manifold,” IEEE Int. Conf. Data Mining, pp. 63-72, 2008.
[29]R. Zhi, M. Flierl, Q. Ruan, and W. B. Kleijn, “Graph-Preserving Sparse Nonnegative Matrix Factorization with Application to Facial Expression Recognition,” IEEE Trans. SMC-Part B: Cybernetics, vol. 41, no. 1, pp. 38-52, 2011.
[30]S. Yan, D. Xu, B. Zhang, Q. Yang, H. Zhang, and S. Lin, “Graph Embedding and Extensions: A General Framework for Dimensionality Reduction,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 29, pp. 40-51, 2007.
[31]N. Guan, D. Tao, Z. Luo, and B. Yuan, “Manifold Regularized Discriminative Nonnegative Matrix Factorization with Fast Gradient Descent,” IEEE Trans. Image Processing, vol. 20, no. 7, pp. 2030-2048, 2011.
[32]J. Yang, S. Yan, Y. Fu, X. Li, and T. Huang, “Non-negative Graph Embedding,” IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[33]X. Liu, S. Yan, and H. Jin, “Projective Nonnegative Graph Embedding,” IEEE Trans. Image Processing, vol. 19, no. 5, pp. 1126-1137, 2010.
[34]Y. Tian, T. Kanade, and J. Cohn, “Recognizing Action Units for Facial Expression Analysis,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, pp. 97-115, 2001.
[35]T. Cootes, C. Taylor, D. Cooper, and J. Graham, “Active Shape Models - Their Training and Applications,” Computer Vision and Image Understanding, vol. 61, pp. 38-59, 1995.
[36]C. Hu, Y. Chang, R. Feris, and M. Turk, “Manifold Based Analysis of Facial Expression,” IEEE Computer Vision and Pattern Recognition Workshop, 2004.
[37] J.B. Tenenbaum, V. de Silva, and J.C. Langford, “A Global Geometric Framework for Nonlinear-Dimensionality Reduction,” Science, vol. 290, no. 5500, pp. 2319-2323, 2000.
[38]S.T. Roweis and L.K. Saul, “Nonlinear-Dimensionality Reduction by Locally Linear Embedding,” Science, vol. 290, no. 5500, pp. 2323-2326, 2000.
[39]X. He and P. Niyogi, “Locality Preserving Projections,” Advances in Neural Information Processing Systems, vol.16, pp.153-160, 2003.
[40]C. Shan, S. Gong, and P. W. McOwan, “Appearance Manifold of Facial Expression,” Computer Vision in Human-Computer Interaction Workshop, 2005.
[41]Y. Chang, C. Hu, and M. Turk, “Manifold of Facial Expression,” IEEE Analysis and Modeling of Faces and Gestures Workshop, pp. 28-35, 2003.
[42]H. Li, J. M. Buenaposada, and L. Baumela, "An Empirical Comparison of Graph-based Dimensionality Reduction Algorithms on Facial Expression Recognition Tasks," IEEE Conf. Pattern Recognition, 2008.