單張影像超解析度在影像處理是一個常見的問題。隨著深度學習快速發展，出現許多基於深度學習的方法來解決這個問題。然而，這些方法有兩個主要的缺點。第一，大部分超解析度的網路只是把資訊直接傳下去得到最後的結果，網路淺層無法利用從網路深層提取的高階特徵的資訊，造成資訊不流通。第二，很多網路為了提高效果而不斷加深網路深度，這可能導致參數量大幅上升與過度學習的問題。因此，超解析度最近引入回饋結構。回饋結構能將高階特徵連接回網路淺層來精煉低階特徵。此外，回饋結構因為他遞歸結構的關係所以也可以視為一種共享參數的網路，所以這種結構也能省下很多參數。受到這些想法的啟發，我們發展出了一種新的架構稱作二階回饋網路 (second order feedback network, SOFN)。我們在一個網路裡面用了兩個回饋結構，也就是說，我們將資訊的流動分成外迴圈與內迴圈。內迴圈注重於加深子網路的網路深度而外迴圈注重於回饋高解析度圖片的資訊給低階特徵。實驗顯示這種結構能保持它的效果與其他先進網路一樣好，但節省了大量的參數。

Single image super-resolution (SR) is a common issue in image processing. As rapid development in deep learning (DL), plenty of DL based methods have been proposed to solve the problem. However, there are two major shortcomings in these methods. First, most SR networks only straightly forward information to the end to get the final output. Shallow layers could not exploit the information of high-level features extracted from deeper layers, which results in the non-circulation of information. Second, many works aim to deepen structure to get better performance, which could lead to a large amount of parameters and overfitting problem. So, a feedback mechanism is introduced in SR recently. The feedback structure could connect the high-level features back to shallow layers and thus refine the low-level features. Moreover, it can be viewed as a weight sharing network due to its recurrent structure, so it could also save lots of parameters. Inspired by these thoughts, we develop a novel structure named second order feedback network (SOFN). We adopt two feedback structures in a single network. That is, we divide the information flows in two loops: an outer loop and an inner loop. The inner loop focuses on deepening the sub-network depth and the outer loop focuses on feedback high-resolution image information to low-level features. Experiments show that such structure could retain competitive results with other state-of-the-art networks but save a great deal of parameters.

[1] C. Dong, C. Loy, K. He, X. Tang, “Image Super-Resolution Using Deep Convolutional Networks,” in Proceedings of the European Conference on Computer Vision, 2014.
[2] Y. Zhang, K. Li, K. Li, L. Wang, B. Zhong, and Y. Fu, “Image Super-Resolution Using Very Deep Residual Channel Attention Networks,” in Proceedings of the European Conference on Computer Vision, 2018.
[3] T. Dai, J. Cai, Y. Zhang, S. Xia, L. Zhang, “Second-order Attention Network for Single Image Super-Resolution,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019.
[4] Z. Li, J. Yang, Z. Liu, X. Yang, G. Jeon, W. Wu, “Feedback Network for Image Super-Resolution,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019.
[5] Q. Li, Z. Li, L. Lu, G. Jeon, K. Liu, X. Yang, “Gated Multiple Feedback Network for Image Super-Resolution,” in British Machine Vision Conference, 2019.
[6] Y. Tai, J. Yang, and X. Liu, “Image Super-Resolution via Deep Recursive Residual Network,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017.
[7] J. Kim, J. Lee and K. Lee, “Deeply-Recursive Convolutional Network for Image Super-Resolution,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016.
[8] Y. Zhang, Y. Tian, Y. Kong, B. Zhong, Y. Fu1, “Residual Dense Network for Image Super-Resolution,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018.
[9] C. Ledig, L. Theis, F. Huszar, J. Caballero, A. Cunningham, “Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017.
[10] J. Kim, J. Lee and K. Lee, “Accurate Image Super-Resolution Using Very Deep Convolutional Networks,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016.
[11] M. Haris, G. Shakhnarovich, and N. Ukita, “Deep Back-Projection Networks For Super-Resolution,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018.
[12] T. Tong, G. Li, X. Liu, Q. Gao, “Image Super-Resolution Using Dense Skip Connections,” in Proceedings of the IEEE International Conference on Computer Vision, 2017.
[13] B. Lim, S. Son, H. Kim, S. Nah, K. Lee, “Enhanced Deep Residual Networks for Single Image Super-Resolution,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2017.
[14] G. Huang, Z. Liu, L. Maaten, “Densely Connected Convolutional Networks,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017.
[15] M. Bevilacqua, A. Roumy, C. Guillemot, and M. L. A.-Morel, “Low-complexity single-image super-resolution based on nonnegative neighbor embedding,” in British Machine Vision Conference, 2012.
[16] R. Zeyde, M. Elad, and M. Protter, “On single image scale-up using sparse-representations,” in Curves and Surfaces, 2010.
[17] D. R. Martin, C. C. Fowlkes, D. Tal, and J. Malik, “A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics,” in Proceedings of the IEEE International Conference on Computer Vision, 2001.
[18] J. B. Huang, A. Singh, and N. Ahuja, “Single image super-resolution from transformed self-exemplars,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015
[19] Y. Matsui, K. Ito, Y. Aramaki, A. Fujimoto, T. Ogawa, T. Yamasaki, and K. Aizawa, “Sketch-based manga retrieval using manga109 dataset,” Multimedia Tools and Applications, 2017.
[20] E. Agustsson and R. Timofte, “Ntire 2017 challenge on single image super-resolution: Dataset and study,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2017.
[21] C. Dong, C. Loy, and X. Tang, “Accelerating the Super-Resolution Convolutional Neural Network,” in Proceedings of the European Conference on Computer Vision, 2016.
[22] R. Timofte, R. Rothe, and L. Gool, “Seven ways to improve example-based single image super resolution,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017.
[23] K. He, X. Zhang, S. Ren, and J. Sun, “Delving deep into rectifiers: Surpassing human-level performance on imagenet classification,” in Proceedings of the IEEE International Conference on Computer Vision, 2015.
[24] K. He, X. Zhang, S. Ren, J. Sun, “Deep Residual Learning for Image Recognition,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015.
[25] A. R. Zamir, T. L. Wu, L. Sun, W. B. Shen, B. E. Shi, J. Malik, S. Savarese, “Feedback Networks,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017.
[26] D. B. Sam and R. V. Babu, “Top-down feedback for crowd counting convolutional neural network,” in Thirty-Second AAAI Conference on Artificial Intelligence, 2018.
[27] J. Carreira, P. Agrawal, K. Fragkiadaki, and J. Malik, “Human pose estimation with iterative error feedback,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015.
[28] X. Zhang, T. Wang, J. Qi, H. Lu, and G. Wang, “Progressive attention guided recurrent network for salient object detection,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018.