In this thesis, we propose a novel system to estimate the intensity of different human facial expressions. Based on the modified optical flow computation, we model the motion of facial pixels instead of relying on complex muscle model and reduce the demand to detect many facial feature points.
According to the assumption that faces of the same expression are close in the expression space, this thesis also proposes a novel weighting scheme for the facial optical flow field by using a quadratic programming formulation. Experiments manifest the efficiency of the proposed system on the expression intensity estimation and expression recognition of face images.

在這篇論文中，我們提出一個可在不同人臉分析表情強度的分析系統。由於我們使用整張人臉的光流場向量作表情分析的特徵，所以並不需要依賴過多的人為控制即可做辨識。
首先，我們假設即使是不同個體在表現相同表情時，會有部分的臉部肌群是呈現類似的運動，因此我們設計一個新的特徵去描述該運動的方向性，並且 將整個問題轉換為一個有條件下的二次數學規劃問題( constrained quadratic programming )，透過統計找出對某種表情較具有代表性的區域，在做表情分析及表情分類時，提升該部分特徵的參考價值，進而做出較正確的判斷。

[1] A. Frome, Y. Singer, F. Sha, and J. Malik, ”Learning Globally-Consistent Local Distance Functions for Shape-Based Image Retrieval and Classification,” Proceedings of IEEE 11th International Conference on Computer Vision, pp.1-8, 2007.
[2] C.-H. Teng, S.-H. Lai, Y. S. Chen, and W.-H. Hsu, “Accurate optical flow computation under non-uniform brightness variations,” Computer Vision and Image Understanding, vol. 97, no. 3, pp. 315-346, 2005.
[3] P. Ekman and W.V. Friesen, Facial Action Coding System (FACS): Manual. Palo Alto: Consulting Psychologists Press, 1978.
[4] H. Chen, Z.Q. Liu, C. Rose, Y.Q. Xu, H.Y. Shum and D. Salesin, ”Example-Based Composite Sketching of Human Portraits,” Proceedings of 3rd International Symposium on Non-Photorealistic Animation and Rendering, pp.95-153, 2004.
[5] Cootes, T. F. and C. J. Taylor, Statistical Models of Appearance for Computer Vision, Technical report, Imaging Science and Biomedical Engineering, University of Manchester, Manchester, UK, 2004.
[6] Dong Guo and Terence Sim,“Digital face makeup by example,” Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp.1-8, 2009
[7] W. Fellenz, J. Taylor, N. Tsapatsoulis, S. Kollias, “Comparingtemplate-based, feature-based and supervised classification of facial expressions from static images,” Proceedings of Circuits, Systems, Communications and Computers, pp. 5331–5336, 1999
[8] Y. Kosaka and K. Kotani,”Facial expression analysis by kernel eigenspace method based on class features (KEMC) using nonlinear basis for separation of expression-classes,” Proceedings of IEEE International Conference on Computer Vision, pp. 1409- 1412, 2004.
[9] M. Black, Y. Yacoob,” Recognizing facial expressions in image sequences using local parameterized models of image motion,” International Journal of Computer Vision, vol.25, no.1, pp. 23-48, 1997.
[10] M. Pantic, L. Rothkrantz,”Expert system for automatic analysis of facial expression,” Image and Vision Computing Journal, vol.18, no.11, pp. 881-905, 2000.
[11] E. Chuang, H. Deshpande, and C. Bregler, “Facial Expression Space Learning,” Proceedings of 10th Pacific Graphics, pp.68-76, 2002
[12] C.-K. Hsieh, S.-H. Lai, and Y.-C. Chen," Expression-Invariant Face Recognition With Constrained Optical Flow Warping,” IEEE Transactions on Multimedia, vol. 11, no. 4, pp. 600-610 , 2009.
[13] B. K. P. Horn and B. G. Schunck, “Determine optical flow,” Artificial Intelligence, vol. 17, pp. 185–203, 1981.
[14] M. J. Lyons, S. Akamatsu, M. Kamachi, and J. Gyoba,”Coding facial expressions with Gabor wavelets,” Proceedings of 3rd IEEE International Conference Automatic Face and Gesture Recognition, pp. 200-205, 1999.
[15] G. Yang and T. S. Huang, “Human Face Detection in a Complex Background,” Pattern Recognition, vol. 27, no. 1, pp. 53-63, 1994.
[16] M.-H. Yang, D. J. Kriegman, and N. Ahuja,” Detecting Faces in Images: A Survey,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 24, no.1,pp.34-58, 2002.
[17] T.K. Leung, M.C. Burl, and P. Perona, “Finding Faces in Cluttered Scenes Using Random Labeled Graph Matching,” Proceedings of 5th IEEE International Conference on Computer Vision, pp. 637-644, 1995.
[18] S. McKenna, S. Gong, and Y. Raja, “Modelling Facial Colour and Identity with Gaussian Mixtures,” Pattern Recognition, vol. 31, no. 12, pp.1883-1892, 1998.
[19] M. Turk and A. Pentland, “Eigenfaces for Recognition,” Journal of Cognitive Neuroscience, vol. 3, no. 1, pp.71-86, 1991.
[20] H. Schneiderman and T. Kanade, “Probabilistic Modeling of Local Appearance and Spatial Relationships for Object Recognition,” Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp.45-51, 1998
[21] H. Rowley, S. Baluja, and T. Kanade, “Neural Network-Based Face Detection,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.20, no. 1, pp. 23-38, 1998.
[22] E. Osuna, R. Freund, and F. Girosi, “Training Support Vector Machines: An Application to Face Detection,” Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp.130-136, 1997.
[23] Paul Viola, Michael Jones, “Robust Real-Time Face Detection,” International Journal of Computer Vision,vol.57, no.2, pp.137-154, 2004
[24] K.-K. Sung and T. Poggio, " Example-based learning for view-based human face detection," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 20, no. 1, pp.39-51, 1998
[25] M. Black, Y. Yacoob,”Recognizing facial expressions in image sequences using local parameterized models of image motion,” International Journal of Computer Vision, vol.25, no.1 , pp. 23–48, 1997.
[26] M. Pantic, L. Rothkrantz, ”Expert system for automatic analysis of facial expression,” Image and Vision Computing , vol.18, no.11 ,pp. 881–905, 2000.
[27] S. Kimura and M. Yachida, "Facial Expression Recognition and its Degree Estimation," Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp.295- 300, 1997.
[28] C. Lisetti and D. Rumelhart, Facial Expression Recognition Using a Neural Network, Proceedings of the 11th International Conference Facial Expression Recognition, American Association for Artificial Intelligence Press, 1998.
[29] K. K. Lee and Y. Xu, “Real-time estimation of facial expression intensity,” Proceedings of IEEE International Conference on Robotics and Automation, pp. 2567- 2572, 2003.
[30] B. D. Lucas and T. Kanade, "An Iterative Image Registration Technique with an Application to Stereo Vision," Proceedings of the 7th International Joint Conference on Artificial Intelligence, pp.674-679, 1981.
[31] C. W. Hsu & C. J. Lin, ”A comparison of methods for multi-class SVMs,” IEEE Transactions on Neural Networks, vol.13, no.2, pp.415-425, 2002.
[32] Lijun Yin, Xiaozhou Wei, Yi Sun, Jun Wang, Matthew J. Rosato,”A 3D Facial Expression Database For Facial Behavior Research,” Proceedings of 7th International Conference on Automatic Face and Gesture Recognition, pp.211-216, 2006.