人臉辨識是近年來在電腦視覺和圖形識別領域受到最廣泛討論的研究項目之一，因為在人臉辨識有三個根本的問題：角度、光線和表情，也就是當人臉的取像角度太大、光線有不同的變化、或是有不同表情的情況下，人臉辨識的正確率會因此而降低。
雖然已經有許多研究試圖來解決這些挑戰，但是只有少數的人嘗試去在一個人只有一張訓練影像條件下，去辨識有不同表情的人臉影像。在這篇論文中，我們藉由導入給定臉部特徵點的相關性來改進規則化的光流（optical flow）演算法，以精確估計畫面上圖素的位移和灰階值的變化。再把目標函式整理成矩陣向量形式後，有條件限制的光流的計算便能使用修改過的ICPCG演算法有效率的完成。
一方面，我們可以用一張特定的中性無表情影像當作參考，計算輸入的影像和此參考影像間的的畫素移動資訊，進而將輸入影像上的表情藉由這些資訊作彈性化形變去除，以提高辨識效果。另一方面，光流的計算可以反過來執行，也就是以輸入的影像當作參考，計算資料庫中的中性影像到測試影像的形變。本論文並提出了一個整合系統，在一個機率的架構下，將不同表情間的光流資訊以及合成後的人臉影像作結合，可以有效的處理不同表情下的人臉辨識問題。在把我們所提出的方法套用到BU-3DFE資料庫的實驗中，顯示我們的方法可以有效的提高不同表情下的人臉辨識率。
此外，本論文也討論了一個處裡不同角度的人臉辨識解決方案。理想的方式是將輸入的影像重建出一個三維的模型，然後合成出正面的影像。但是這個方法對於一個即時的應用太複雜而不能被實行。我們將這個問題規劃成一個非線性的姿勢正規化問題，並提出結合核心函數(kernel function)和線性回歸法的演算法，讓這個解法更接進理想的解答。

Face recognition is one of the most intensively studied topics in computer vision and pattern recognition. There are three essential issues to be dealt with in the research of face recognition; namely, pose, illumination, and expression variations. The recognition rate will drop considerably when the head pose or illumination variation is too large, or when there is expression on the face.
Although many researches were focused on overcoming these challenges, few were focused on how to robustly recognize expressional faces with one single training sample per class. In this thesis, we modify the regularization-based optical flow algorithm by imposing constraints on some given point correspondences to compute precise pixel displacements and intensity variations. The constrained optical flow computation can be efficiently computed by applying the modified ICPCG algorithm.
By using the optical flow computed from the input expression-variant face image with respect to a reference neutral face image, we can remove the expression from the face image by elastic image warping to recognize the subject with facial expression. On the other hand, the optical flow can be computed in the opposite direction, which is from the neutral face image to the input expression-variant face image. By combining information from the computed intra-person optical flow and the synthesized face image in a probabilistic framework, an integrated face recognition system is proposed, which can be robust against facial expressions with a limited size of training database. Experimental validation on the Binghamton University 3D Face Expression (BU-3DFE) Database is given to show that the proposed expression normalization algorithm significantly improves the accuracy of face recognition on expression variant faces.
A possible solution for overcoming the pose variation problem in face recognition is also presented in this thesis. The ideal solution is to reconstruct a 3D model from the input images and synthesize the virtual image with the corresponding pose, which might be too complex to be implemented in a real-time application. By formulating this kind of solution as a nonlinear pose normalization problem, we propose an algorithm that integrates the nonlinear kernel function and the linear regression method, which makes the solution resemble to the ideal one. Some discussions and experiments on CMU PIE database are carried out, and the experimental results show that the proposed method is robust against pose variations.

[1] M.S. Bartlett, J.R. Movellan, and T.J. Sejnowski, “Face recognition by independent component analysis,” IEEE Trans. on Neural Networks, Vol. 13, No. 6, November 2002, pp. 1450-1464.
[2] H. Cevikalp, M. Neamtu, M. Wilkes, and A. Barkana, “Discriminative Common Vectors for Face Recognition,” IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 27, No. 1, 2005, pp. 4-13.
[3] X. Chai, S. Shan, X. Chen, and W. Gao , “Local linear regression (LLR) for pose invariant face recognition,” Proc. of International Conf. on Automatic Face and Gesture Recognition (FGR06).
[4] T. Cootes, C. Taylor, D. Cooper, and J. Graham, “Active shape models – their training and application,” Comput. Vision and Image Understanding, Vol. 61, pp. 18-23, 1995
[5] Cootes, G. J. Edwards, and C. Taylor. “Active appearance models,” IEEE Trans. Patt. Analy. Mach. Intell., Vol. 23, pp.681-685, 2001
[6] I.A. Essa and A. Pentland, “A vision system for observing and extracting facial action parameters,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 76-83, 1994.
[7] K. Fukunaga, Introduction to statistical pattern recognition (2nd edition), Academic Press, 1990.
[8] A.S. Georghiades, P.N. Belhumeur, and D.J. Kriegman, “From few to many: illumination cone models for face recognition under differing pose and lighting,” IEEE Tran. Pattern Analysis and Machine Intelligence, vol. 23, no. 6, pp. 643-660, June 2001.
[9] D. O. Gorodnichy, “Video-based framework for face recognition in video,” Proc. Canadian Conference on Computer and Robot Vision, pages 330–338, May. 2005.
[10] Y.H. He, L. Zhao, and C. R. Zou, “Kernel Discriminative Common Vectors for Face Recognition,” Proc. of International Conf. on Machine Learning and Cybernetics, Guangzhou, 18-21 August 2005, pp. 4605-4610.
[11] B. K. P. Horn and B.G. Schunck, “Determine optical flow,” Artif. Intell. 17 (1981) pp.185-203.
[12] C.K. Hsieh and Y.C. Chen, “Kernel-based pose invariant face recognition,” Proc. of International Conf. Multimedia and Expo., Beijing, China, July 2007.
[13] T.-K. Kim and J. Kittler, “Locally linear discriminant analysis for multimodally distributed classes for face recognition with a single model image,” IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 27, No. 3, March 2005, pp. 318-327.
[14] Y.-H. Kim, A. M. Martinez, and A. C. Kak, “Robust motion estimation under varying illumination,” Image and Vision Computing, vol. 23, pp. 365-375, 2005.
[15] S. Z. Li and A. K. Jain. Handbook of Face Recognition. Springer. New York, 2005.
[16] X. Li, G. Mori, and H. Zhang, “Expression-invariant face recognition with expression classification, “ Proc. of the 3rd Canadian Conf. on Computer and Robot Vision , June, 2006.
[17] S. C. Lin, M. J. Li, H. J. Zhang, and Q. S. Cheng, “Ranking prior likelihood distributions for Bayesian shape localization framework,” Proceedings of International Conf. Computer Vision, 2003.
[18] A.M. Martinez and A.C. Kak, “PCA versus LDA,” IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 23, No. 2, 2001, pp. 228-233.
[19] M. Martinez, “Recognizing imprecisely localized, partially occluded and expression variant faces from a single sample per class,” IEEE Trans. Pattern and Mach. Intell., vol. 24, no. 6, pp. 748-763, 2002.
[20] A.M. Martinez, “Matching expression variant faces,” Vision Research, vol. 43, pp. 1047-1060, 2003.
[21] M. Martinez, “Recognizing expression variant faces from a single sample image per class,” Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, June 2003.
[22] S. Negahdaripour, “Revised definition of optical flow: integration of radiometric and geometric cures for dynamic scene analysis,” IEEE Trans. on Pattern Anal. and Mach. Intell., Vol. 20, No. 9, pp. 961-979, Sept. 1998.
[23] U. Park, H. Chen, and A. Jain, “3D model-assisted face recognition in video,” Proc. Canadian Conference on Computer and Robot Vision, pp. 322–329, May. 2005.
[24] Rama and F.Tarres, “P2CA: A new face recognition scheme combining 2D and 3D information,” Proc. International Conf. Image Processing, Genoa, Italy, September 2005.
[25] Rama and F.Tarres, “Partial LDA VS partial PCA,” Proc. of International Conf. Multimedia and Expo., Ontario, Cananda, July 2006.
[26] M. Ramachandran, S. K. Zhou, D. Jhalani, and R. Chellappa, “A method for converting a smiling face to a neutral face with applications to face recognition,” Proc. ICASSP, Vol., pp. 18-23 March 2005.
[27] R. Ramamoorthi and P. Hanrahan, “On the relationship between radiance and irradiance: determining the illumination from images of a convex Lambertian object,” J. Optical Soc. Am., Vol. 18, no. 10, pp. 2448-2459, 2001.
[28] Shashua and T. Riklin-Raviv, “The quotient image: class-based re-rendering and recognition with varying illuminations,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 2, pp. 129-139.
[29] C.-H. Teng, S.-H. Lai, Y.-S. Chen, and W.-H. Hsu, “Accurate optical flow computation under non-uniform brightness variations,” Computer Vision and Image Understanding 97 (2005), pp. 315-346.
[30] P.-H. Tsai and T. Jan, “Expression-invariant face recognition system using subspace model analysis,“ Proc. IEEE Conference on Systems, Man and Cybernetics, Vol. 2, Oct. 2005, pp. 1712-1717.
[31] M.A. Turk and A.P. Pentland, “Face recognition using Eigenfaces,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, June 1991, Maui, Hawaii, USA, pp. 586-591
[32] Y. Yacoob and L.S. Davis, “Recognizing human facial expressions from long image sequences using optical flow,” IEEE Trans. on Pattern Analysis and Machine Intelligence, Volume 18, Issue 6, June 1996, pp. 636–642.
[33] M.-H. Yang, “Kernel Eigenfaces vs. Kernel Fisherfaces: face recognition using kernel methods,” Proc. International Conf. on Automatic Face and Gesture Recognition, May 2002, Washington D.C., USA, pp. 215-220.
[34] L. Yin, X. Wei, Y. Sun, J. Wang, and M. J. Rosato, “A 3D facial expression database for facial behavior research,” Proc. International Conf. Automatic Face and Gesture Recognition, April 2006 P:211 – 216.
[35] Y. Zhang, and A. M. Martinez, “A weighted probabilistic approach to face recognition from multiple images and video sequences,” Image and Vision Computing, vol. 24, pp. 626-638, 2006.
[36] W. Zhao, R. Chellappa, P. J. Phillips, and A. Rosenfeld. “Face recognition: A literature survey,” ACM Computing Surveys, 35(4):399–458, Dec. 2003.
[37] S. Milborrow and F. Nicolls. “Locating facial features with an extended active shape model,” Proc. European Conf. Computer Vision, October, 2008, Marseille, France.
[38] V. Blanz and T. Vetter, "Face recognition based on fitting a 3D morphable model," IEEE Trans. on Pattern Analysis and Machine Intelligence, VOL. 25, NO. 9, Sept. 2003.
[39] K. I. Chang, K. W. Bowyer, and P. J. Flynn, "An evaluation of multimodal 2D+3D face biometrics," IEEE Trans. on Pattern Analysis and Machine Intelligence, VOL. 27, NO. 4, Apr. 2005.
[40] X. Lu, A. K. Jain, and D. Colbry, "Matching 2.5D face scans to 3D models", IEEE Trans. on Pattern Analysis and Machine Intelligence, VOL. 28, NO. 1, Jan. 2006.
[41] K. Chang, K. Bowyer, and P. Flynn, “ Effects on facial expression in 3D face recognition,” Proc. of the SPIE Conf. Geometric Technology for Human Identification, 2005, pp. 132-143.
[42] K. Bowyer, K. Chang, and P. Flynn, “A survey of approaches and challenges in 3D and multi-model 3D+2D face recognition,” Comput. Vis. Image Understand., vol. 101, no. 1, pp1-15, 2006.
[43] K. Bowyer, K. Chang, and P. Flynn, “Multiple nose region matching for 3D face recognition under varying facial expression,” IEEE Tran. Pattern anal. Mach. Intell., vol. 28, no. 10, pp. 1695-1700, Oct. 2006.
[44] A.M. Bronstein, M.M. Bronstein, and R. Kimmel, “Expression-invariant representations of faces,” IEEE Trans. On Image Processing, Vol. 16, no. 1, pp.-188-197, Jan. 2007.
[45] S. C. Yan, M. J. Li, H. J. Zhang, Q. S. Cheng, “Ranking prior lilkelihood distributions for Bayesian shape localization framework”, Proceedings of the ICCV 2003.
[46] U. Park and A. K. Jain, “3D face recognition from stereo video”, Proceedings of the 3rd Canadian Conference on Computer and Robot Vision (CRV’06).
[47] J.J. Lien, A. Zlochower, J.F. Cohn, T. Kanade, “Automated Facial Expression Recognition.” Proceedings of the Third IEEE International Conference on Automatic Face and Gesture Recognition. Nara, Japan. April, 1998. 390-395.
[48] P. Kruizinga, N. Petkov, “Optical flow applied to person identification.” Proceeding of the 1994 EUROSIM Conference on Massively Parallel Processing Applications and Development, Delft, The Netherlands, 21-23 June 1994, Elsevier, Amsterdam, 871-878.
[49] X. Liu, T. Chen, and B. V. K. Vijaya Kumar, “Face authentication for multiple subjects using eigenflow,” Pattern Recognition, 36, 313–328, 2003.