結合人臉檢測與人物分割之臉部影像廣角鏡頭失真校正

簡易檢索 / 詳目顯示

回結果列表

研究生：	許斯涵 Hsu, Ssu-Han
論文名稱：	結合人臉檢測與人物分割之臉部影像廣角鏡頭失真校正 Face-based Lens Distortion Correction Combined with Face Detection and Human Segmentation
指導教授：	林嘉文 Lin, Chia-Wen
口試委員:	蔡文錦施皇嘉胡敏君
學位類別：	碩士 Master
系所名稱：	電機資訊學院 - 電機工程學系 Department of Electrical Engineering
論文出版年：	2020
畢業學年度：	109
語文別：	英文
論文頁數：	31
中文關鍵詞：	廣角鏡頭校正、⼈物分割遮罩、⼈臉邊界框、影像扭曲、透視投影校正、電腦視覺
外文關鍵詞：	Fisheye Distortion Calibration, Human Segmentation Mask, Face Bounding Box, Content-Aware Image Warping, Perspective Correction, Computer Vision
相關次數：	點閱：3 下載：0
分享至:	分享至facebook 分享至twitter

查詢本校圖書館目錄查詢臺灣博碩士論文知識加值系統勘誤回報

⽣活中，我們常常使用魚眼鏡頭以取得更廣角視野的影像，然⽽在享受寬廣
視野這個優點時，因為光學投射的設計，在鏡頭內的物體會被扭曲變形，包括⼈
的臉型扭曲、筆直的樑柱被彎曲等，使得物體在影像中的呈現與實際上⼤相徑庭，
讓⼈看得不舒服。因此，失真校正在電腦視覺領域中是⼀項重要的議題。
以前的校正⽅法⼤多注重在直線的拉伸或⼿動標注關鍵物體作為基準來還
原影像，⽽在此我們提出⼀個以⼈臉為主要目標物的校正⽅式，將背景與⼈臉套
用不同的光學投影，配合深度學習的⼈臉偵測和⼈物分割技術，能自動偵測並校
正影像，達到更⾼度還原影像實際狀況的效果。我們使用⼀些損失函數來增強不
同投影網格之間的流暢度，能同時保持臉部形狀⽽不扭曲背景。此外，我們提出
⼀個自定義的權重加⼊損失函數，能增加收斂效率，我們希望這個⽅法能降低失
真造成的不適並還原真實臉型。

Fisheyes camera can take wide-angle photos which have expanding views.
However, a wider FOV introduces strongly distortions. These distortions lead to stretched face shape and bending straight lines, which is extremely different from real life, causing human eyes suffering. Therefore, face-based lens distortion correction is an important issue for computer vision applications.
In contrast to conventional approaches, which focus on detecting straight lines or extracting hand-crafted features from input images, here we propose a content-aware warping mesh, which adapts to the stereographic projection on facial regions and the perspective projection on the background, with automatic human face detecting and
masking. Some energy functions are used to encourage smooth transitions between the two conflicting projections at face boundary, also keep the face a natural look without twisting background. Additionally, to improve efficiency, we apply self-designed weight to loss functions, which can reduce iteration and running time. This work hopes to lower the distortion and make human faces a nature look.

摘要................................................................................................................................. i
Abstract .......................................................................................................................... ii
Content ........................................................................................................................... 1
Chapter 1 Introduction ................................................................................................ 3
Chapter 2 Related Work .............................................................................................. 6
2.1 Line-aware Rectification ............................................................................ 6
2.2 High-level Semantic Information .............................................................. 7
2.3 Content-aware warping mesh .................................................................... 8
2.4 Face Segmentation Mask ........................................................................... 8
Chapter 3 Proposed Method ...................................................................................... 10
3.1 Face Mask Segmentation ......................................................................... 11
3.1.1 Human Segmentation Mask ................................................................ 11
3.1.2 Face Box Detection ............................................................................. 12
3.2 Perspective Projection Mesh .................................................................... 15
3.3 Stereographic Projection Mesh ................................................................ 15
3.4 Loss Function Regression ........................................................................ 16
3.4.1 Face Objective Term ........................................................................... 16
3.4.2 Line-Bending Term ............................................................................. 16
3.4.3 Smoothing Regularization Term ......................................................... 17
3.4.4 Mesh Boundary Extension term .......................................................... 17
3.5 Self-Designed Weight .............................................................................. 18
3.6 Detail Process after Loss .......................................................................... 19
Chapter 4 Experiments .............................................................................................. 20
2
4.1 Implement and Experimental Setup ......................................................... 20
4.2 Image Presentation ................................................................................... 21
4.3 Ablation study .......................................................................................... 23
4.3.1 Weight for loss term ............................................................................ 23
4.3.2 Weight for other masks ....................................................................... 26
Chapter 5 Conclusion ................................................................................................ 27
Reference ..................................................................................................................... 28
                                

[1] Zhucun Xue, Nan Xue, Gui-Song Xia, and Weiming Shen (2019). “Learning to
calibrate straight lines for fisheye image rectification.” In Proceedings of the IEEE
Conference on Computer Vision and Pattern Recognition (pp. 1643-1651).
[2] Manuel Lopez, Roger Mari, Pau Gargallo, Yubin Kuang, Javier Gonzalez-Jimenez,
and Gloria Haro. “Deep Single Image Camera Calibration with Radial Distortion.”
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern
Recognition (CVPR), 2019, pp. 11817-11825
[3] Jin Wei, Chen-Feng Li, Shi-Min Hu, Ralph R. Martin, and Chiew-Lan Tai.
“Fisheye video correction.” IEEE Transactions on Visualization and Computer
Graphics 18.10 (2011): 1771-1783.
[4] Xiaoqing Yin, Xinchao Wang, Jun Yu, Maojun Zhang, Pascal Fua, and Dacheng
Tao. “FishEyeRecNet: A multi-context collaborative deep network for fisheye
image rectification.” In Proceedings of the European Conference on Computer
Vision (ECCV), 2018, pp. 469-484.
[5] Xiang Mei, Sen Yang, Jiangpeng Rong, Xianghua Ying, Shiyao Huang, and
Hongbin Zha. “Radial lens distortion correction using cascaded one-parameter
division model.” In IEEE International Conference on Image Processing (ICIP),
2015, pp. 3615-3619.
[6] Faisal Bukhari and Matthew N Dailey, “Automatic radial distortion estimation
from a single image,” Journal of mathematical imaging and vision, vol. 45, no. 1,
pp. 31–45, 2013.
[7] Mi Zhang, Jian Yao, Menghan Xia, Kai Li, Yi Zhang, and Yaping Liu. “Line-based
multi-label energy optimization for fisheye image rectification and calibration.” In
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,pages 4137–4145, 2015.
[8] Rui Melo, Michel Antunes, João Pedro Barreto, Gabriel Falcao, and Nuno
Gonçalves. “Unsupervised Intrinsic Calibration from a Single Frame Using a
"Plumb-Line" Approach.” In Proc. of Internatoinal Conference on Computer
Vision, pages 537–544, 2013.
[9] YiChang Shih, Wei-Sheng Lai, and Chia-Kai Liang. “Distortion-Free Wide-Angle
Portraits on Camera Phones.” ACM Transactions on Graphics 38, no. 4 (December
2019): 1–12.
[10] Chun-Hao Chao, Pin-Lun Hsu, Hung-Yi Lee, and Yu-Chiang Frank Wang. “Self-
Supervised Deep Learning for Fisheye Image Rectification.” In 2020 IEEE
International Conference on Acoustics, Speech and Signal Processing
(ICASSP), pp. 2248-2252. IEEE.
[11] Zhu-Cun Xue, Nan Xue, and Gui-Song Xia. “Fisheye Distortion Rectification
from Deep Straight Lines.” arXiv preprint arXiv:2003.11386 (2020).
[12] Hyunjoon Lee, Eli Shechtman, Jue Wang, and Seungyong Lee, “Automatic upright
adjustment of photographs.” In 2012 IEEE Conference on Computer Vision and
Pattern Recognition (pp. 877-884). IEEE.
[13] Quan Chen, Tiezheng Ge, Yanyu Xu, Zhiqiang Zhang, Xinxin Yang, and Kun Gai.
“Semantic human matting.” In Proceedings of the 26th ACM international
conference on Multimedia (pp. 618-626), 2018.
[14] Xuebin Qin, Zichen Zhang, Chenyang Huang, Chao Gao, Masood Dehghan, and
Martin Jagersand. “Basnet: Boundary-aware salient object detection,” in
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,
2019, pp. 7479–7489.
[15] Joseph Redmon, & Ali Farhadi. Yolov3: An incremental improvement. arXiv
preprint arXiv:1804.02767, 2018.
[16] Qibin Hou, Ming-Ming Cheng, Xiaowei Hu, Ali Borji, Zhuowen Tu, and Philip
H. S. Torr. “Deeply supervised salient object detection with short connections.”
In Proceedings of the IEEE Conference on Computer Vision and Pattern
Recognition (pp. 3203-3212), 2017.
[17] Chang, Che-Han, Chia-Kai Liang, and Yung-Yu Chuang. “Content-aware display
adaptation and interactive editing for stereoscopic images.” IEEE Transactions on
Multimedia, vol. 13, no. 4, pp. 589–601, Aug. 2011.
[18] Yajie Zhao, Zeng Huang, Tianye Li, Weikai Chen, Chloe LeGendre, Xinglei Ren,
Ari Shapiro, and Hao Li. “Learning perspective undistortion of portraits.”
In Proceedings of the IEEE International Conference on Computer Vision (ICCV),
pp. 7849-7859, 2019.
[19] Vikas Gupta, & Shanmuganathan Raman. “Automatic trimap generation for image
matting.” In 2016 International Conference on Signal and Information Processing
(IConSIP), pp. 1-5, IEEE.
[20] Shaofan Cai, Xiaoshuai Zhang, Haoqiang Fan, Haibin Huang, Jiangyu Liu,
Jiaming Liu, Jiaying Liu, Jue Wang, and Jian Sun. “Disentangled image matting.”
In Proceedings of the IEEE International Conference on Computer Vision (ICCV),
pp. 8819-8828, 2019.
[21] Saining Xie, &Zhuowen Tu. “Holistically-nested edge detection.” In Proceedings
of the IEEE international conference on computer vision (ICCV), pp. 1395-1403,
2015.
[22] Kevis-Kokitsi Maninis, Jordi Pont-Tuset, Pablo Arbeláez, and Luc Van Gool.
“Convolutional oriented boundaries: From image segmentation to high-leveltasks.” IEEE transactions on pattern analysis and machine intelligence, vol. 40,
no. 4, pp. 819–833, Apr. 2018.
[23] Yun Liu, Ming-Ming Cheng, Xiao-wei Hu, Kai Wang, and Xiang Bai. “Richer
convolutional features for edge detection.” In Proceedings of the IEEE Conference
on Computer Vision and Pattern Recognition (CVPR), pp. 3000-3009, 2017.
[24] Kun Huang, Yifan Wang, Zihan Zhou, Tianjiao Ding, Shenghua Gao, and Yi Ma.
“Learning to Parse Wireframes in Images of Man-Made Environments.” In
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
(CVPR), 2018, pp. 626-635.
[25] Nan Xue, Song Bai, Fudong Wang, Gui-Song Xia, Tianfu Wu, Liangpei Zhang,
and Philip H.S. Torr. “Learning regional attraction for line segment detection.”
IEEE Trans. Pattern Analysis and Machine Intelligence (TPAMI), 2019.
[26] Nan Xue, Tianfu Wu, Song Bai, Fudong Wang, Gui-Song Xia, Liangpei Zhang,
and Philip H.S. Torr. “Holistically-attracted wireframe parsing,” in Proceedings
of the IEEE/CVF Conference on Computer Vision and Pattern Recognition
(CVPR), 2020, pp. 2788-2797.
[27] Xiyang Luo, Ruohan Zhan, Huiwen Chang, Feng Yang, and Peyman Milanfar.
“Distortion Agnostic Deep Watermarking.” In Proceedings of the IEEE/CVF
Conference on Computer Vision and Pattern Recognition (CVPR), 2020, pp.
13548-13557.
[28] Lijun Wang, Huchuan Lu, Yifan Wang, Mengyang Feng, Dong Wang, Baocai Yin,
and Xiang Ruan. “Learning to detect salient objects with image-level supervision.”
In Proceedings of the IEEE Conference on Computer Vision and Pattern
Recognition (CVPR), 2017, pp. 136-145.

簡易檢索 / 詳目顯示

相關論文