隨著虛擬實境的盛行，愈來愈多廠商也跟著推出360相機提供使用者方便瀏覽全景圖像。有了3D場景重建的輔助，使用者可以在特殊裝置下切換至想要觀賞的畫面；這種切換視角的方式讓我們更有身歷其境的感覺。
本篇論文提出一種球型全景影像的視圖內插，我們利用兩張360全景影像可以在共線的位置上內插出中間任一影像。360全景影像一般用等距離長方圓柱投影的方式表現，該影像於南北極中存在著很大的扭曲。傳統的立體視覺影像推估深度圖的方式無法應用於的該影像。我們首先對360影像偵測特徵點並用共軛限制去除錯誤的對應。利用有效的對應點求出最佳的旋轉矩陣以校正360影像使得左圖右圖影像上的特徵點共用相同的x軸。此時便可以利用傳統的立體視覺影像計算出正確的深度圖。內插時對於遮蔽區域我們會利用修改過後的TV_L1處理，對於無結構的地方例如地板我們則利用區域性的補貼。在實驗的過程中我們嘗試內插出位於左右全景中間的球型全景影像。並且我們也產生出一系列位於左圖與右圖之間的連續影像。

Spherical panorama images with 360-degree view provide users with immersive experiences for the real world under an appropriate display environment. With the aid of advanced stereo and 3D reconstruction techniques, people can freely navigate the environment through 360-degree spherical images at different viewpoints.
In this thesis, we propose a framework that can interpolate a plausible spherical panorama on the baseline between two spherical cameras. Traditional stereo reconstruction methods could not perform well on generating a spherical image from a novel viewpoint since the equirectangular format contains large image distortion around poles. The proposed algorithm first applies feature detection and epipolar constraint in the spherical coordinate to extract reliable feature correspondences. Then, we rectify the spherical images to align the corresponding feature points on the same x-coordinate with optimized rotation matrices. Next, we use the modified TV_L1 method to compute the disparity maps for the rectified stereo spherical images. Finally, we apply the pixel-wise image warping with the estimated disparity maps and the mesh-based interpolation to complete missing regions to obtain the spherical image at an intermediate location between those of the two input spherical images. Our experiments demonstrate the proposed algorithm can generate intermediate spherical images from two spherical images and the results are evaluated with objective image quality.

[1] Kawai, N., Audras, C., Tabata, S., Matsubara, T. “Panorama image interpolation for real-time walkthrough.” ACM SIGGRAPH 2016 Posters, p. 33, July 2016.
[2] Kim, H., Hilton, A. “3d scene reconstruction from multiple spherical stereo pairs.” International journal of computer vision, vol. 104, pp. 94-116, 2013
[3] Bodington, D., Thatte, J., Hu, M. “Rendering of stereoscopic 360° views from spherical image pairs.” 2015.
[4] Zhao, Q., Wan, L., Feng, W., Zhang, J., & Wong, T. T. “Cube2video: Navigate between cubic panoramas in real-time.” IEEE Transactions on Multimedia, vol. 15, no. 8 , pp. 1745-1754, 2013.
[5] Li, S., Fukumori, K. “Spherical stereo for the construction of immersive VR environment.” Virtual Reality, 2005. Proceedings. VR 2005. IEEE, pp. 217-222, 2005.
[6] Pagani, A., Gava, C. C., Cui, Y., Krolla, B., Hengen, J. M., & Stricker, D. “Dense 3D Point Cloud Generation from Multiple High-resolution Spherical Images.” In VAST, pp. 17-24, 2011.
[7] Fujiki, J. “Three types of reprojection error on spherical epipolar geometry.” The 8th Workshop on Omnidirectional Vision, Camera Networks and Non-classical Cameras-OMNIVIS, 2008.
[8] Pagani, A., Stricker, D. “Structure from motion using full spherical panoramic cameras.” ICCV Workshops, 2011 IEEE International Conference on IEEE, pp. 375-382, 2011.
[9] Zhao, Q., Feng, W., Wan, L., Zhang, J. “SPHORB: A fast and robust binary feature on the sphere.” International Journal of Computer Vision, vol. 113, no. 2, pp. 143-159, 2015.
[10] Taira, H., Inoue, Y., Torii, A., Okutomi, M. “Robust feature matching for distorted projection by spherical cameras.” IPSJ Transactions on Computer Vision and Applications, vol. 7, pp. 84-88, 2015.
[11] Aly, M., Bouguet, J. Y. “Street view goes indoors: Automatic pose estimation from uncalibrated unordered spherical panoramas.” Applications of Computer Vision (WACV), 2012 IEEE Workshop on IEEE, pp. 1-8, 2012.
[12] Chaurasia, G., Duchene, S., Sorkine-Hornung, O., Drettakis, G. “Depth synthesis and local warps for plausible image-based navigation.” ACM Transactions on Graphics (TOG), vol. 32, no. 3, p. 30, 2013.
[13] Cheng, C. M., Lin, S. J., Lai, S. H. “Spatio-temporally consistent novel view synthesis algorithm from video-plus-depth sequences for autostereoscopic displays.” IEEE Transactions on Broadcasting, vol. 57, no. 2, pp. 523-532, 2011.
[14] Im, S., Ha, H., Rameau, F., Jeon, H. G., Choe, G., Kweon, I. S. “All-Around Depth from Small Motion with a Spherical Panoramic Camera.” European Conference on Computer Vision. Springer International Publishing, pp. 156-172, 2016.
[15] Schönbein, M., Geiger, A. “Omnidirectional 3d reconstruction in augmented manhattan worlds.” Intelligent Robots and Systems (IROS 2014), 2014 IEEE/RSJ International Conference on IEEE, pp. 716-723, 2014.
[16] Li, S. “Binocular spherical stereo.” IEEE Transactions on Intelligent Transportation Systems, vol. 9, no. 4, pp. 589-600, 2008.
[17] Chen, S. E., Williams, L. “View interpolation for image synthesis.” Proceedings of the 20th annual conference on Computer graphics and interactive techniques, ACM, pp. 279-288, 1993.
[18] Zhang, C., Li, Z., Cheng, Y., Cai, R., Chao, H., Rui, Y. “Meshstereo: A global stereo model with mesh alignment regularization for view interpolation.” Proceedings of the IEEE International Conference on Computer Vision, pp. 2057-2065, 2015.
[19] Gurdan, T., Oswald, M. R., Gurdan, D., Cremers, D. “Spatial and temporal interpolation of multi-view image sequences.” German Conference on Pattern Recognition, Springer International Publishing, pp. 305-316, 2014.
[20] Liu, S., Chen, J., Sun, M., Zhao, L. “The Derivation and Analysis of Spherical Panorama Epipolar Line Based on Essential Matrix.” Journal of Computer-Aided Design & Computer Graphics, vol. 4, no. 28, pp. 785-794, 2016.
[21] Zhang, G., He, Y., Chen, W., Jia, J., Bao, H. “Multi-viewpoint panorama construction with wide-baseline images.” IEEE Transactions on Image Processing, vol. 25, no. 7, pp. 3099-3111, 2016.
[22] Chang, C. H., Sato, Y., Chuang, Y. Y. “Shape-preserving half-projective warps for image stitching.” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3254-3261, 2014.
[23] Zach, C., Pock, T., Bischof, H. “A duality based approach for realtime TV-L1 optical flow.” Pattern Recognition, pp. 214-223, 2007.
[24] Lan W. L., Yao S. H., Lai S.-H. “Edge-preserving disparity map estimation from stereo videos for bokeh synthesis.” In IEEE International Conference on Multimedia and Expo, 2017.
[25] Hosni, A., Rhemann, C., Bleyer, M., Rother, C., Gelautz, M. “Fast cost-volume filtering for visual correspondence and beyond.” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no. 2, pp. 504-511, 2013.
[26] Shen, L. C., "360 panorama video stabilization" Master's thesis, National Taiwan University, Taiwan, 2016.