在本篇論文中，我們提出一套針對在具有多平面的場景中可估測近點光源之演算法。藉由偵測平面標記，我們可以估測場景的三維空間配置。透過計算，我們可轉換影像像素從影像坐標系至世界坐標系以便於估測光源位置，最後輸出的光照參數可藉由最小化誤差函數得到。在實驗的部分，我們分別使用合成的影像以及現實的影像進行評估，在實驗中，我們使用合成的影像以及現實的數據集進行評估，並於實驗結果顯示出我們所提出的演算法優於目前較先進的光照估測方法。此外我們同時也發展擴增實境系統，在應用光照效果在繪製虛擬物件的部分，運用到本論文方法所估測出之光照參數，給予逼真的視覺效果。

In this thesis, we present a novel lighting estimation algorithm for the scene containing two or more planes. This thesis focuses on near point light source estimation. We detect planar markers to estimate the poses of the 3D planes in the scene. Then we transform the image pixels from image coordinates to world coordinates for the estimation of light position. The output of the proposed method is the lighting parameters estimated from minimizing the error function. In the experiments, we test on synthetic data and real dataset. Our experimental results show the proposed algorithm outperforms the state-of-the-art lighting estimation method. Moreover, we develop an augmented reality system that includes lighting estimation by using the proposed algorithm.

[1] IKEA Catalog https://itunes.apple.com/us/app/ikea-catalog/id386592716?mt=8.
[2] Anatomy 4D https://itunes.apple.com/tw/app/anatomy-4d/id555741707?mt=8.
[3] G. Oxholm and K. Nishino. Multiview shape and reflectance from natural illumination. In Computer Vision and Pattern Recognition (CVPR), 2014 IEEE Conference on, pages 2163–2170. IEEE, 2014.
[4] M. K. Johnson and E. H. Adelson. Shape estimation in natural illumination. In Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on, pages 2553–2560. IEEE, 2011.
[5] J. F. Blinn. Models of light reflection for computer synthesized pictures. In ACM SIGGRAPH Computer Graphics, volume 11, pages 192–198. ACM, 1977.
[6] J. Ackermann, S. Fuhrmann, and M. Goesele. Geometric point light source calibration. In VMV, pages 161–168, 2013.
[7] P. Debevec. Rendering synthetic objects into real scenes: Bridging traditional and image-based graphics with global illumination and high dynamic range photography. In ACM SIGGRAPH 2008 classes, page 32. ACM, 2008.
[8] M.W. Powell, S. Sarkar, and D. Goldgof. A simple strategy for calibrating the geometry of light sources. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 23(9):1022–1027, 2001.
[9] T. Takai, A. Maki, K. Niinuma, and T. Matsuyama. Difference sphere: an approach to near light source estimation. Computer Vision and Image Understanding, 113(9):966–978, 2009.
[10] K. Kato, F. Sakaue, and J. Sato. Extended multiple view geometry for lights and cameras from photometric and geometric constraints. In Pattern Recognition (ICPR), 2010 20th International Conference on, pages 2110–2113. IEEE, 2010.
[11] K.-Y. K. Wong, D. Schnieders, and S. Li. Recovering light directions and camera poses from a single sphere. In Computer Vision–ECCV 2008, pages 631–642. Springer, 2008.
[12] L.-F. Yu, S.-K. Yeung, Y.-W. Tai, D. Terzopoulos, and T. F. Chan. Outdoor photometric stereo. In Computational Photography (ICCP), 2013 IEEE International Conference on, pages 1–8. IEEE, 2013.
[13] D. Schnieders and K.-Y. K.Wong. Camera and light calibration from reflections on a sphere. Computer Vision and Image Understanding, 117(10):1536–1547, 2013.
[14] M. Haller, S. Drab, and W. Hartmann. A real-time shadow approach for an augmented reality application using shadow volumes. In Proceedings of the ACM symposium on Virtual reality software and technology, pages 56–65. ACM, 2003.
[15] Y. Wang and D. Samaras. Estimation of multiple directional light sources for synthesis of augmented reality images. Graphical Models, 65(4):185–205, 2003.
[16] J.-F. Lalonde and I. Matthews. Lighting estimation in outdoor image collections. In 3D Vision (3DV), 2014 2nd International Conference on, volume 1, pages 131–138. IEEE, 2014.
[17] T.-P.Wu and C.-K. Tang. Dense photometric stereo using a mirror sphere and graph cut. In Computer Vision and Pattern Recognition, 2005. CVPR 2005. IEEE Computer Society Conference on, volume 1, pages 140–147. IEEE, 2005.
[18] J. Pilet, A. Geiger, P. Lagger, V. Lepetit, and P. Fua. An all-in-one solution to geometric and photometric calibration. In Mixed and Augmented Reality, 2006. ISMAR 2006. IEEE/ACM International Symposium on, pages 69–78. IEEE, 2006.
[19] X. Chen, K. Wang, and X. Jin. Single image based illumination estimation for lighting virtual object in real scene. In Computer-Aided Design and Computer Graphics (CAD/Graphics), 2011 12th International Conference on, pages 450–455. IEEE, 2011.
[20] S. Lombardi and K. Nishino. Reflectance and natural illumination from a single image. In Computer Vision– ECCV 2012, pages 582–595. Springer, 2012.
[21] L. Gruber, T. Richter-Trummer, and D. Schmalstieg. Real-time photometric registration from arbitrary geometry. In Mixed and Augmented Reality (ISMAR), 2012 IEEE International Symposium on, pages 119–128. IEEE, 2012.
[22] J. Park, S. N. Sinha, Y. Matsushita, Y.-W. Tai, and I. S. Kweon. Calibrating a non-isotropic near point light source using a plane. In Computer Vision and Pattern Recognition (CVPR), 2014 IEEE Conference on, pages 2267–2274. IEEE, 2014.
[23] M. Meilland, C. Barat, and A. Comport. 3d high dynamic range dense visual slam and its application to real-time object re-lighting. In Mixed and Augmented Reality (ISMAR), 2013 IEEE International Symposium on, pages 143–152. IEEE, 2013.
[24] NLopt http://ab-initio.mit.edu/wiki/index.php/Main_Page
[25] H. Kato and M. Billinghurst. Marker tracking and hmd calibration for a video-based augmented reality conferencing system. In Augmented Reality, 1999.(IWAR’99) Proceedings. 2nd IEEE and ACM International Workshop on, pages 85–94. IEEE, 1999.
[26] H. Bay, T. Tuytelaars, and L. Van Gool. Surf: Speeded up robust features. In Computer vision–ECCV 2006, pages 404–417. Springer, 2006.
[27] M. J. Powell. A direct search optimization method that models the objective and constraint functions by linear interpolation. In Advances in optimization and numerical analysis, pages 51–67. Springer, 1994.