研究生: |
黃柏翰 Huang, Po-Han |
---|---|
論文名稱: |
基於模糊捕捉模塊在深度多補丁網絡上的單圖像去模糊 A Deep Multi-patch Network with Blur Capture Module for Single Image Deblurring |
指導教授: |
張隆紋
Chang, Long-Wen |
口試委員: |
陳朝欽
Chen, Chaur-Chin 陳永昌 Chen, Yung-Chang |
學位類別: |
碩士 Master |
系所名稱: |
電機資訊學院 - 資訊工程學系 Computer Science |
論文出版年: | 2021 |
畢業學年度: | 109 |
語文別: | 英文 |
論文頁數: | 35 |
中文關鍵詞: | 去模糊 、恢復模糊 、深度學習 |
外文關鍵詞: | Deblurring, Reblurring, Deep Learning |
相關次數: | 點閱:2 下載:0 |
分享至: |
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報 |
運動模糊是計算機視覺中的一個基本的問題,因為它可能會破壞圖像的內容並阻礙模型推理。隨著卷積網絡 (CNN) 的快速發展,已經提出了大量基於 CNN 的方法來解決該問題。然而,運動模糊往往是不均勻和空間變化的。現有方法使用大內核大小或注意力模塊來解決這個問題,但這會帶來大量參數和緩慢的推理時間。我們提出了兩個模塊,一個模糊擷取模塊 (blur capture module) 和再模糊模塊 (reblur module)。本文提出的損失函數配合前面的模塊有助於我們的模型更好地理解運動模糊是如何發生的。我們還提供特徵圖作為證據來證明我們提出的模塊的有效性。實驗結果說明,我們提出的網絡優於現有的去模糊工作。我們的模型在增加 5% 的參數下,在 PSNR 中達到了 30.734,在 SSIM 中達到了 0.94。定量結果表明,與其他最先進的方法相比,所提出的網絡和其他應用我們模塊的模型可以恢復清晰的圖像,內部模糊更少。
Motion blur is an essential issue in computer vision as it may destroy the image's content and hinder inference. With the rapid development in convolutional networks (CNN), plenty of CNN-based methods have been proposed to solve the problem. However, motion blur tends to be non-uniform and spatial variant. Existing approaches use a large kernel-size or attention module to resolve this problem, but this brings significant parameters and slow inference time. This thesis proposed two modules, a blur capture module and a reblur module for single image deblurring. The loss in this thesis combined with our proposed module helps our model to understand how motion blur occurs.
Experiments show that our proposed network outperforms existing deblurring works. We achieve 30.734 in PSNR and 0.94 in SSIM with an additional 5 percent of parameters. The quantitative results show that the proposed network and other models applying our modules can restore a sharp image with less blur inside than other state-of-the-art methods.
[1] R. Fergus, B. Singh, A. Hertzmann, S. T. Roweis, and W. T. Freeman, “Removing camera shake from a single photograph,” In ACM Transactions on Graphics (TOG), ACM, pages 787 – 794, 2006.
[2] S. Nah, T. Kim, and K.M Lee. Deep multi-scale convolutional neural network for dynamic scene deblurring. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 257 – 265, 2017.
[3] X. Tao, H. Gao, X. Shen, J. Wang, and J. Jia. Scale-recurrent network for deep image deblurring. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 8174–8182, 2018.
[4] S. Xingjian, Z. Chen, H. Wang, D.-Y. Yeung, W.-K. Wong, and W.-c. Woo. Convolutional lstm network: A machine learning approach for precipitation nowcasting. In NIPS, pages 802–810, 2015.
[5] H. Zhang, Y. Dai, H. Li, and P. Koniusz. Deep stacked hierarchical multi-patch network for image deblurring. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 5978– 5986, 2019.
[6] J. Sun, W. Cao, Z. Xu, and J. Ponce. Learning a convolutional neural network for nonuniform motion blur removal. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 769–777, 2015.
[7] J. Zhang, J. Pan, J. Ren, Y. Song, L. Bao, R. Lau, and M. Yang. Dynamic scene deblurring using spatially variant recurrent neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2521–2529, 2018.
[8] H. Gao, X. Tao, X. Shen, and J. Jia. Dynamic scene deblurring with parameter selective sharing and nested skip connections. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3848–3856, 2018.
[9] Huaijin Chen, Jinwei Gu, Orazio Gallo, Ming-Yu Liu, Ashok Veeraraghavan, and Jan Kautz. Reblur2deblur: Deblurring videos via self-supervised learning. In 2018 IEEE International Conference on Computational Photography (ICCP), pages 1–9. IEEE, 2018.
[10] Kaihao Zhang, Wenhan Luo, Yiran Zhong, Lin Ma, Bjorn Stenger, Wei Liu, and Hongdong Li. Deblurring by realistic blurring. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2737– 2746, 2020.
[11] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 770–778, 2016.
[12] Maitreya Suin, Kuldeep Purohit, and A. N. Rajagopalan. Spatially-attentive patch-hierarchical network for adaptive motion deblurring, in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 3606–3615, 2020.
[13] Kuldeep Purohit and AN Rajagopalan. Region-adaptive dense network for efficient motion deblurring. In AAAI, pages 11882–11889, 2020.
[14] Woo, J. Park, J. Lee, and I. Kweon. Cbam: Convolutional block attention module. In Proceedings of the European Conference on Computer Vision (ECCV), pages 3–19, 2018.
[15] Qibin Hou, Li Zhang, Ming-Ming Cheng, and Jiashi Feng. Strip pooling: Rethinking spatial pooling for scene parsing. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4003–4012, 2020.
[16] J.-Y. Zhu, T. Park, P. Isola, and A. A. Efros. Unpaired imageto-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE International Conference on Computer Vision (ICCV) , pages 1-8, 2017.
[17] D.P Kingma and J. Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
[18] S. W. Zamir et al., Multi-stage progressive image restoration, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1–11, 2021.
[19] Orest Kupyn, Tetiana Martyniuk, Junru Wu, and Zhangyang Wang. DeblurGAN-v2: Deblurring (orders-ofmagnitude) faster and better. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), pages 1–8, 2019.
[20] Y. Yuan, W. Su, and D. Ma. Efficient dynamic scene deblurring using spatially variant deconvolution network with optical flow guided training. In Proc. Conf. Computer Vision and Pattern Recognition, pages 3555 – 3562, 2020.