隨著3D顯示技術的成熟，裸視3D電視的可觀看視角越來越多，然而這也代表著對於影像壓縮的要求越來越嚴苛，各個不同視角影像的儲存和播放的問題也越來越難解決。傳統上，裸視3D電視的輸入使用一張影像配合一張深度圖來產生不同視角的影像來解決儲存的問題，然而隨之產生的是補洞或深度扭曲的問題，前者產生不自然的區域，後者則是會讓3D效果減損，無論哪個都是我們所不樂見的。
隨著新一代多媒體標準的制定，多視角多深度的輸入來源變的可能，這對於多視角顯示器是一個福音。然而，隨著多視角顯示器越來越多的視角和多視角3D內容的製作複雜度與視角數量呈正相關，多視角多深度的輸入內容可能無法盡數涵蓋每個視角，只能涵蓋主要角度的輸入。因此，對於那些無法涵蓋到的視角，如何使用既有的資訊來合成變成新的問題。
在本篇論文之中建立一套模型探討多視角顯示器與多個攝影機之間的相應關係與虛擬視野影像的合成，並且藉由多視角及其深度圖的虛擬視野合成方式避免了傳統利用一張影像一張深度的DIBR (Depth Image Based Rendering)所帶來的問題，以及如果在無深度輸入的情況之下，如何使用一套低複雜度的stereo matching求得合成虛擬視野影像所需的深度圖。最後，使用FPGA對於虛擬視野影像合成和求取深度這兩個部分的演算法分別作硬體驗証及實現。

Along with the development of 3D rendering technology, the view number of multi-view autostereoscopic TV has become much more than before. This means that the requirement of video compression is more serious than ever, and at the same time, the storage and rendering problems of multi-views are also getting harder. In traditional DIBR (Depth Image Based Rendering), multi-view TV can use a source that contains one image and one depth map to synthesize the other virtual views, so that the source need not store all views. However, it causes holes or depth distortion in virtual views, the former of which makes strange area on rendered image and the latter makes the depth perceived distorted.
As the newest video coding standard is to be established, multi-view video and its corresponding depth maps source becomes possibly available, thus benefits the utility of multi-view TV. But in reality, the multi-view source may not involve all views for multi-view TV. Because the number of views for multi-view TV is still increasing, and the complexity of creating a multi-view source for content producer is proportional to the view number. That means how to create the virtual views not contained in the original source will become a new problem.
This thesis creates a model to depict the relationships between multi-view TV, camera, and virtual views, and illustrates how to use the model to synthesize virtual views. The virtual view synthesis algorithm here by using multi images and depth maps as inputs, which are different from traditional DIBR. In this way, problems of traditional DIBR can be circumvented. In the meantime, our algorithm also proposes a stereo matching technique in low complexity which can be used when the multi-view source does not contain its own depth maps.

[1] D. Scharstein and R. Szeliski. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms.International Journal of Computer Vision, 47(1/2/3):7-42, April-June 2002.Microsoft Research Technical Report MSR-TR-2001-81, November 2001.

[2] ISO/IEC JTC2/SC29/WG11, “Report on experimental framework for 3D video coding,” N11631, Guangzhou, China, Oct. 2010.

[3] C.Fehn,“Depth-image-based rendering (DIBR), compression and transmission for a new approach on 3DTV,”in Proc. of SPIE Conference on Stereoscopic Displays and Virtual Reality Systems, vol. 5291, pp. 93-104, San José, CA, May 2004.

[4] J.Flack, P.Harman and S.Fox, “Low bandwidth stereoscopic image encoding and transmission,” in Proc. of SPIE Conference on Stereoscopic Displays and Virtual Reality Systems X, vol. 5006, pp. 206-214, CA, U.S.A., Jan. 2003.

[5] A.Redert, M.Op de Beeck, C.Fehn, W.IJsselsteijn, M.Pollefeys, L.Van Gool, E.Ofek, I.Sexton and P.Surman, “ATTEST—advanced three-dimensional television system techniques,” in Proc. 3DPVT’ 02, pp. 313-319, Padova, Italy, Jun. 2002.

[6] L.Zhang and W.J.Tam,“Stereoscopic image generation based on depth images for 3D TV,”IEEE Transactions On Broadcast, vol. 51, pp. 191-199, June 2005.

[7] W.J.Tam, G.Alain, L.Zhang, T.Martin and R. Renaud, “Smoothing depth maps for improved stereoscopic image quality,” in Proc. of SPIE Conference On Three-Dimensional TV, Video, and Display III, vol. 5599, pp. 162-172, Philadelphia, U.S.A., Oct. 2004.

[8] C. Tomasi and R. Manduchi, “ Bilateral filtering for gray and color images,“ in Proc. Of IEEE international Conference on Computer Vision, pp. 839 - 846, Bombay, India, Jan. 1998.

[9] Yuhua Zhu and Tong Zhen, “3D Multi-View Autostereoscopic Display and Its Key Technologie,” Asia-Pacific Conference on Information Processing, 2009
[10] U.S. application patent publication. Publication No. US20070018585A1

[11] Chia-Kai Liang, Chao-Chung Cheng, Yen-Chieh Lai, “Hardware-Efficient Belief Propagation”, Transactions on Circuits and Systems for Video Technology, VOL. 21, NO. 5, May 2011

[12] M. Gong and Y.-H. Yang. Near real-time reliable stereo matching using programmable graphics hardware. CVPR 2005.

Müller, K. Smolic, A. Dix, K. Kauff, P. Wiegand, T. “Reliability-based generation and view synthesis in layered depth video” International Workshop on Multimedia Signal Processing, Cairns, Queensland, Australia,2008.