環場影像接合技術的問世打破了原本影像視野範圍的限制，除了一般網頁瀏覽器可以 觀賞以外，虛擬實境眼鏡更拉近了觀眾與環場影像之間的距離，然而受限於拍攝者的位 置，觀眾無法在環場影像中自由的移動，或是僅能透過跳躍式的方式在點跟點之間進行移 動，這樣的移動方式常使人感到暈眩。為此，本論文研究如何在兩個相鄰點的環場影像之 間合成出新視野的環場影像。以影像為基礎的渲染(Image-based rendering)是一項新視 野合成的技術，透過環境點雲或是視差圖的輔助，將來源影像變形到新視野並接合而得到 新視野影像，然而大多數的研究都是針對同一物體或是小角度的環境改變的情況。本研究 中參考了 Chaurasia et al. [1] 的概念以及 Chang et al. [2] 的變形以及接合方法來進行周圍 環境新視野的合成。為了加速我們對後續合成研究的效率，我們先在虛擬的環境當中建立 點雲以及來源影像，並把來源影像切割為多個超像素 (Superpixel)，然後根據物體表面性 質進行合併。在合成時我們提出層次性的變形方法:單應性 (Homography) 的變形以及非 線性的變形，並將變形後的影像進行拼貼產生完整的一張新視野影像。我們使用不同方法 產生結果並進行比較，我們發現合併後的超像素比未合併的超像素使用層次性變形的方法 時還要的完整，而且針對大面積且同平面的超像素使用單應性變形比非線性變形在速度上 還要快。這些結果凸顯了我們的方法可以透過超像素的合併而有良好的合成效果以及利用 層次性變形來節省變形時間。

Panorama stitching has broken the limit of the eld of view in traditional image. In addition to watching from the web browser, Watching form VR (Virtual reality) headset make the distance between the panorama and user closer. However, the user can not move freely in the panorama and can only jump betwween the camera sets, which is limited by the positions of shooting cameras. Image-baesd rendering is a kind of technique for novel view synthesis that use the point cloud or disparty map as the aid to warp the source image to novel view and stitch them together. Their works can only handle the scenerios with small camera angle di erence or foucus on speci c kinds of objects. In this thesis, we adopt the similar idea of Chaurasia et al. [1] and Chang et al. [2] propose a system to synthesize any novel views. In order to design the algorithm of novel view synthesis directly, we simulate the source images and point cloud in virtual environment, oversegment the source image based on color in many superpixels, and merge these superpixel based on the object surface attributes. In the synthesis stage, we use a hierarchical warping strategy, which consists of homography wapring and non-linear warping, to warp the source images to the novel view. Finally these warped images are stitched together to render the novel view. We conducted an experiment to validate the e ectiveness of hierarchical warping and superpixel merging. The results show that the hierarchical warping can greatly reduce computation time and generates better results with merged superpixels.

[1] Gaurav Chaurasia, Sylvain Duchene, Olga Sorkine-Hornung, and George Drettakis. Depth synthesis and local warps for plausible image-based navigation. ACM Trans. Graph., 32 (3):30:1–30:12, 2013.
[2] Chia-Sheng Chang, Hung-Kuo Chu, and Niloy J. Mitra. Interactive videos: Plausible video editing using sparse structure points. Computer Graphics Forum (Proc. Eurographics), 35, 2016.
[3] Shenchang Eric Chen and Lance Williams. View interpolation for image synthesis. In Pro- ceedings of the 20th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH ’93, pages 279–288. ACM, 1993.
[4] Jiangjian Xiao and Mubarak Shah. Tri-view morphing. Comput. Vis. Image Underst., 96 (3):345–366, 2004.
[5] Noah Snavely, Steven M. Seitz, and Richard Szeliski. Photo tourism: Exploring photo collections in 3d. In SIGGRAPH Conference Proceedings, pages 835–846. ACM Press, 2006.
[6] Michael Goesele, Jens Ackermann, Simon Fuhrmann, Carsten Haubold, Ronny Klowsky, Drew Steedly, and Richard Szeliski. Ambient point clouds for view interpolation. ACM Trans. Graph., 29(4):95:1–95:6, 2010.
[7] Wojciech Matusik, Chris Buehler, Ramesh Raskar, Steven J. Gortler, and Leonard McMil- lan. Image-based visual hulls. In Proceedings of the 27th Annual Conference on Com- puter Graphics and Interactive Techniques, SIGGRAPH ’00, pages 369–374. ACM Press/ Addison-Wesley Publishing Co., 2000.
[8] Joel Carranza, Christian Theobalt, Marcus A. Magnor, and Hans-Peter Seidel. Free- viewpoint video of human actors. ACM Trans. Graph., 22(3):569–577, 2003.
[9] Sundar Vedula, Simon Baker, and Takeo Kanade. Image-based spatio-temporal modeling and view interpolation of dynamic events. ACM Trans. Graph., 24(2):240–261, 2005.
[10] Christian Lipski, Christian Linz, Kai Berger, and Marcus Magnor. Virtual video camera: Image-based viewpoint navigation through space and time. In SIGGRAPH ’09: Posters, SIGGRAPH ’09, pages 93:1–93:1. ACM, 2009.
[11] C. Lawrence Zitnick, Sing Bing Kang, Matthew Uyttendaele, Simon Winder, and Richard Szeliski. High-quality video view interpolation using a layered representation. ACM Trans. Graph., 23(3):600–608, 2004.
[12] Luca Ballan, Gabriel J. Brostow, Jens Puwein, and Marc Pollefeys. Unstructured video- based rendering: Interactive exploration of casually captured videos. ACM Trans. Graph., 29(4):87:1–87:11, 2010.
[13] Naoki Kawai, Cédric Audras, Sou Tabata, and Takahiro Matsubara. Panorama image interpolation for real-time walkthrough. In ACM SIGGRAPH 2016 Posters, SIGGRAPH ’16, pages 33:1–33:2. ACM, 2016.
[14] Radhakrishna Achanta, Appu Shaji, Kevin Smith, Aurelien Lucchi, Pascal Fua, and Sabine Susstrunk. SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Trans. Pattern Anal. Mach. Intell., 34(11):2274–2282, 2012.
[15] Feng Liu, Michael Gleicher, Hailin Jin, and Aseem Agarwala. Content-preserving warps for 3d video stabilization. ACM Trans. Graph., 28(3):44:1–44:9, 2009.
[16] R. I. Hartley and A. Zisserman. Multiple View Geometry in Computer Vision. Cambridge University Press, ISBN: 0521540518, second edition, 2004.
[17] Yanqing Chen, Timothy A. Davis, William W. Hager, and Sivasankaran Rajamanickam. Algorithm 887: Cholmod, supernodal sparse cholesky factorization and update/downdate. ACM Trans. Math. Softw., 35(3):22:1–22:14, 2008.
[18] Aseem Agarwala, Mira Dontcheva, Maneesh Agrawala, Steven Drucker, Alex Colburn, Brian Curless, David Salesin, and Michael Cohen. Interactive digital photomontage. ACM Trans. Graph., 23(3):294–302, 2004.
[19] Yuri Boykov, Olga Veksler, and Ramin Zabih. Fast approximate energy minimization via graph cuts. IEEE Trans. Pattern Anal. Mach. Intell., 23(11):1222–1239, 2001.