在這篇論文中，我們提出了一個影像內插的演算法，從單張非正面的人臉照片合成出新的、不同角度的臉部影像。基於人臉對稱的假設，我們將原本的非正面人臉影像當作一張基底影像，之後影像對縱軸作水平的鏡射轉換，可得到另一張基底影像。這個系統使用了一個三維的人體頭部平均模型，並藉由使用者手動給定一組少量的臉部特徵點來估計原本影像和鏡射影像上人臉的姿勢。求密集對應關係一直是這類研究裡一個重要然而困難的問題，在本篇論文裡，我們藉由二維到三維的RBF(radial basis function)函數計算、三維到二維的透視圖法投影(perspective projection)，並且利用optical flow計算作微調以克服平均模型與目標影像之間的差異，再加上用relative gradient以克服光影變化太大的問題，即可求出兩張基底影像之間的密集點對應關係。本篇論文裡提出一個雙向的影像轉換演算法，求得正向和逆向的內插影像，以解決物體自身遮蔽所產生的點對應問題。最後，我們提出一個適當的加權函數來合併上述求得的兩張影像，以計算出最終的內插影像。實驗結果顯示，在這個系統裡，僅需要輸入單張非正面的臉部照片，可以內插出原本與鏡射影像間各個角度的內插影像，這些得到的內插影像可以模擬真實人臉在相同角度下的影像。實驗並提供真實人臉照片與合成後的結果影像之比較。

In this thesis, we propose a view interpolation algorithm to synthesize novel face images at different viewpoints from a single non-frontal face image. Based on the symmetry assumption of human faces, we mirror the original non-frontal face image to obtain another basis image. A 3D generic head model is used to estimate the face pose by manually labeling a small set of feature points as input to the system. The dense matching between two basis images is computed from 2D-3D RBF mapping, 3D-2D perspective projection and optical flow computation. To solve the self-occlusion problem in point correspondence, a bi-directional morphing algorithm is used to obtain the forward and the backward interpolated images. Finally, the final view-interpolated image is obtained by combining the above two interpolated images with an appropriate weighting function. Experimental results show satisfactory view interpolation results from a single face image.

[1] T. Beier and S. Neely, “Feature-Based Image Metamorphosis,” Proc. ACM SIGGRAPH, pp. 35-42, 1992.

[2] T. Poggio and R. Brunelli, “A Novel Approach to Graphics,” Technical Report A.I. Memo No. 1354, Artificial Intelligence Laboratory, Massachusetts Inst. of Technology, page 71, 1992.

[3] D. Beymer, A. Shashua, and T. Poggio, “Example Based Image Analysis and Synthesis,” Technical Report A.I. Memo No. 1431, Artificial Intelligence Laboratory, Massachusetts Inst. of Technology, 1993.

[4] S.E. Chen and L. Williams, “View Interpolation for Image Synthesis,” Proc. ACM SIGGRAPH, pp. 279-288, 1993.

[5] S.M. Seitz and C.R. Dyer, “Physically-Valid View Synthesis by Image Interpolation,” Proc. IEEE Workshop Representation of Visual Scenes, pp.18-25, 1995.

[6] S.M. Seitz and C.R. Dyer, “View Morphing,” Proc. ACM SIGGRAPH, pp. 21-30, 1996.

[7] Shai Avidan and Amnon Shashua, “Novel View Synthesis by Cascading Trilinear Tensors,” IEEE Trans. on Visualization and Computer Graphics, vol. 4, no. 4, pp. 293-306, 1998.

[8] H. Saito, S. Baba, and T. Kanade, “Appearance-Based Virtual View Generation from Multicamera Videos Captured in the 3-D Room,” IEEE Trans. on Multimedia, vol.5, no.3, pp.303-316, 2003.

[9] R.M. Haralick, H. Joo, C. Lee, X. Zhuang, V.G. Vaidya, and M.B. Kim, “Pose estimation from corresponding point data,” IEEE Trans. Systems, Man, and Cybernetics, vol.19, no. 6, pp. 1426-1446, 1989.

[10] Y. Liu, T.S. Huang, and O.D. Faugeras, “Determination of camera location from 2-d to 3-d line and point correspondences,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 12, no. 1, pp. 28-37, Jan. 1990.

[11] R. Franke and G. Nielson, “Scattered Data Interpolation and Applications: A Tutorial and Survey,” in Geometric Modeling: Methods and Their Application, H. Hagan and D. Roller, eds., Springer, Berlin, pp. 131-160, 1990.

[12] R. L. Hardy, “Multiquadric equation of topography and other irregular surfaces,” J. Geophysical Res. 76, pp. 1905-1915, 1971.

[13] Richard Hartley and Andrew Zisserman, “Camera Models,” in Multiple View Geometry in Computer Vision, Cambridge University Press, Cambridge, UK, pp.139-165, 2000.

[14] S.H. Lai and B.C. Vemuri, “Reliable and Efficient Computation of Optical Flow,” Intl. Journal of Computer Vision, vol. 29, no. 2, pp. 87-105, August-September 1998.