近幾年，在深度學習應用於影像處理中，單張超解析度成像(single image super-resolution)是一個很重要的問題。在超解析度成像領域中基於卷積神經網路會有很好的效果。然而，現在很多解析度成像的網路層數越來越深，越深的網路往往輕易地增加參數量和高計算複雜度。此外，這些網路也常常忽略充分利用網路每一層的資訊。因此，為了解決這些問題我們提出了一個殘差特徵融合網路(residual feature fusion network，RFFN)。我們藉由殘差學習中增加全局殘差連接與區域殘差連接來降低訓練困難度。另外，我們使用全局特徵融合與區域特徵融合來充分利用每一個殘差模塊，這些方法都可以有效地使效能優於其他網路。在標準的測量單位(benchmark metric)與資料集(datasets)，我們提出的RFFN效能比其他先進的模型來的出色。

Recently, application in deep learning of image processing, single image super-resolution (SISR) is a vital problem. The model based convolutional neural networks have shown great results in SR. However, now most networks have become deeper and wider. With the network’s depth growing, the number of parameters will increase easily and with high computation complexity. Also, lots of modern networks neglect to fully utilize the information of each layer. Therefore, to address these problems, we proposed a residual feature fusion network (RFFN). We use residual learning adding global and local skip connections to overcome training difficulty. In addition, we also fully utilize every residual module (RM) feature with global and local features fusion which can efficiently make the performance better than the other methods. Experiments show the comparison between our RFFN and state-of-the-art SISR networks with the benchmark metrics and datasets.

[1] Dong, C., Loy, C.C., He, K., & Tang, X. (2016). Image Super-Resolution Using Deep Convolutional Networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38, 295-307.
[2] Dong, C., Loy, C.C., & Tang, X. (2016). Accelerating the Super-Resolution Convolutional Neural Network. ECCV.
[3] Kim, J., Lee, J.K., & Lee, K.M. (2016). Accurate Image Super-Resolution Using Very Deep Convolutional Networks. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 1646-1654.
[4] Kim, J., Lee, J.K., & Lee, K.M. (2016). Deeply-Recursive Convolutional Network for Image Super-Resolution. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 1637-1645.
[5] Shi, W., Caballero, J., Huszár, F., Totz, J., Aitken, A.P., Bishop, R., Rueckert, D., & Wang, Z. (2016). Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 1874-1883.
[6] He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep Residual Learning for Image Recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 770-778.
[7] Lim, B., Son, S., Kim, H., Nah, S., & Lee, K.M. (2017). Enhanced Deep Residual Networks for Single Image Super-Resolution. 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 1132-1140.
[8] Zhang, Y., Tian, Y., Kong, Y., Zhong, B., & Fu, Y.R. (2018). Residual Dense Network for Image Super-Resolution. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2472-2481.
[9] Tong, T., Li, G., Liu, X., & Gao, Q. (2017). Image Super-Resolution Using Dense Skip Connections. 2017 IEEE International Conference on Computer Vision (ICCV), 4809-4817.
[10] Wang, F., Jiang, M., Qian, C., Yang, S., Li, C., Zhang, H., Wang, X., & Tang, X. (2017). Residual Attention Network for Image Classification. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 6450-6458.
[11] Hu, J., Shen, L., Albanie, S., Sun, G., & Wu, E. (2020). Squeeze-and-Excitation Networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42, 2011-2023.
[12] Zhang, Y., Li, K., Li, K., Wang, L., Zhong, B., & Fu, Y.R. (2018). Image Super-Resolution Using Very Deep Residual Channel Attention Networks. ECCV.
[13] Liu, J., Zhang, W., Tang, Y., Tang, J., & Wu, G. (2020). Residual Feature Aggregation Network for Image Super-Resolution. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2356-2365.
[14] Zhang, L., & Wu, X. (2006). An edge-guided image interpolation algorithm via directional filtering and data fusion. IEEE Transactions on Image Processing, 15, 2226-2238.
[15] Yang, J., Wright, J., Huang, T.S., & Ma, Y. (2008). Image super-resolution as sparse representation of raw image patches. 2008 IEEE Conference on Computer Vision and Pattern Recognition, 1-8.
[16] Yang, J., Wright, J., Huang, T.S., & Ma, Y. (2010). Image Super-Resolution Via Sparse Representation. IEEE Transactions on Image Processing, 19, 2861-2873.
[17] Agarap, A.F. (2018). Deep Learning using Rectified Linear Units (ReLU). ArXiv, abs/1803.08375.
[18] E. Agustsson, R. Timofte, “Ntire 2017 challenge on single image super-resolution: Dataset and study,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2017.
[19] M. Bevilacqua, A. Roumy, C. Guillemot, M.L. Alberi-Morel, “Low-complexity single-image super-resolution based on nonnegative neighbor embedding,” in British Machine Vision Conference,2012.
[20] R. Zeyde, M. Elad, M. Protter, “On single image scale-up using sparse-representations,” in Curves and Surfaces, 2010.
[21] P. Arbelaez, M. Maire, C. Fowlkes, J. Malik, “Contour detection and hierarchical image segmentation,” in Proceedings of the IEEE Transactions on Pattern Analysis and Machine Intelligence,2011.
[22] J. B. Huang, A. Singh, N. Ahuja, “Single image super-resolution from transformed self-exemplars,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015.
[23] Wang, Z., Bovik, A.C., Sheikh, H.R., & Simoncelli, E.P. (2004). Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing, 13, 600-612.
[24] D. Kingma, J. Ba, “Adam: A method for stochastic optimization,” in Proceedings of the International Conference on Learning Representations, 2014.
[25] Dai, T., Cai, J., Zhang, Y., Xia, S., & Zhang, L. (2019). Second-Order Attention Network for Single Image Super-Resolution. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 11057-11066.
[26] Y. Tai, J. Yang, X. Liu, C. Xu, “Memnet: A persistent memory network for image restoration” in Proceedings of the IEEE International Conference on Computer Vision ,2017.
[27] Muhammad Haris, Gregory Shakhnarovich, and Norimichi Ukita. Deep back-projection networks for super-resolution. In CVPR, pages 1664–1673. IEEE Computer Society, 2018.
[28] W.S. Lai, J.B. Huang, N. Ahuja, M.H. Yang, “ Deep laplacian pyramid networks for fast and accurate super-resolution,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition ,2017.