簡易檢索 / 詳目顯示

回結果列表
研究生： 呂保明
Lu, Pao-Ming
論文名稱： 利用深度學習中的轉移學習來處理錯誤標記的數據
Utilizing mislabeled data by transfer learning in deep neural networks
指導教授： 吳金典
Wu, Chin-Tien
高淑蓉
Kao, Shu-Jung
口試委員: 朱家杰
Chu, Chia-Chieh
王夏聲
Wang, Shiah-Sen
學位類別： 碩士
Master
系所名稱： 理學院 - 數學系
Department of Mathematics
論文出版年： 2018
畢業學年度： 106
語文別： 英文
論文頁數： 48
中文關鍵詞： 深度學習 、影像分割 、標記錯誤資料
外文關鍵詞： deep learning, Image segmentation, mislabeled data
相關次數： 點閱：1  下載：0 
分享至:
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報

    • 深度學習對於設備與資料有著非常高的要求[1]，其中資料更是模型好壞的核心。然而在實際問題上，我們要獲取標記好的資料是不容易的。在我們的論文中就是希望能解決這類型的問題，儘管手上的資料不一定非常的好，但還是能透過一些方法，得到一個結果不差的模型。
    我們的實驗主要是在處理無人機影像與衛星影像的地物分析。這些資料數量不多，且不是這麼容易進行標記。我們透過GoogleAPI尋找類似我們實驗的資料集來進行訓練，然而我們收集的資料也不是這麼理想，有20%-30%的比例是標記錯誤的。
    為了解決這個問題，我們藉由[2],[3],[4],[5],[6]..等類神經網路來設計我們的模型架構，並參考了[7], [8] ,[9]等非監督式模型。透過[10]可以先假定我們原始資料所訓練的模型是不差的，並結合blockwise的model來輔助我們修改標記，來解決標記錯誤的問題。為了取得更好的效果，我們將這個方法進行疊代，使得這筆GoogleAPI的資料集提升了3%~5%的準確度，並找到了我們想要的模型。
    最後，當我們拿到一筆新的未標記資料時，可以在不使用其他資料集的前提下，只需進行一些簡單的標記，就可以利用我們的模型架構和演算法直接訓練出一個適合的模型，依此來解決難以取得有標記資料的問題。

    Deep learning has very high requirements for equipment and materials [1], and the data is the core of the model. However, in practical matters, it is not easy for us to obtain the marked information. In our paper, we hope to solve this type of problem. Although the information we have is not good enough, we can still get a model with good results through some methods.
    Our experiments are mainly in image segmentation of unmanned aerial vehicles and satellite imagery. The amount of this information is less and difficult to label, so we use GoogleAPI to find a data set similar to our experiment for training. However, the data we collected is not so ideal, and 20%-30% is labeled incorrectly.
    In order to solve this problem, we design our model architecture by [2],[3],[4],[5],[6] and other neural networks, and reference to [7], [8], [9] and other unsupervised models. By [10], we can assume that the model we trained in the original data is not bad, then combine with the block-wise model to assist us in modifying the markup and solve the problem of markup errors. For better results, we use this method to iterate. Then we improved the 3%~5% accuracy in the GoogleAPI dataset and found the model we wanted.
    Finally, when we get a new unlabeled dataset, we can use our model architecture and algorithms to directly train a suitable model without using other data sets. In order to solve the problem of difficult to obtain labeled data.

    CONTENTS 口試委員會審定書 # 中文摘要 2 ABSTRACT 3 CONTENTS 4 LIST OF FIGURES 6 Chapter 1 Introduction 9 Chapter 2 Background 10 2.1 Introduction to Artificial Neural Networks 10 2.2 Backpropagation 12 2.3 Loss function and Overfitting 14 2.4 Convolution Neural Networks 16 Chapter 3 Neural Network Architecture 17 3.1 VGG 18 3.2 Residual Net 18 3.3 Fully convolution neural network 20 3.4 U-Net 20 3.5 Refine Net 21 Chapter 4 Algorithms and Implementation 22 4.1 Data and data processing 22 4.1.1 Motivation and Problem 22 4.2 Model structure 24 4.3 Processing data problems 25 4.3.1 Distribution of data 25 4.3.2 Fine-tuning 26 4.3.3 Self-transfer Learning 27 4.3.4 Transfer Learning with Modify Part of Data 28 4.3.5 Transfer Learning with Noise 30 4.3.6 Transfer Learning with Block-wise Prediction 33 Chapter 5 Experimental result 38 5.1 Results of methods with data selection 38 5.2 Results of methods without data selection 40 5.2.1 Transfer Learning with Noise on Mnist 41 5.2.2 Integrate the application of our method 44 Chapter 6 Experimental result 46 Reference 47 LIST OF FIGURES Fig. 2.1 Comparing Perceptron and Logistic Regression 11 Fig. 2.2 An example of fully connect neural networks 11 Fig. 2.3 A simple example of a neural 12 Fig. 2.4 A picture that assists us to let us know backpropagation 13 Fig. 2.5 Backward Calculation Algorithm 13 Fig. 2.6 Rectified linear unit 14 Fig. 2.7 Dropout 15 Fig. 2.8 Convolution on Neural Networks 16 Fig. 2.9 Max pooling 17 Fig. 3.1 The experiment of very deep network model 19 Fig. 3.2 The principle of Residual Net 19 Fig. 3.3 The neural network architecture for FCNN 20 Fig. 3.4 The neural network architecture for U-net 21 Fig. 3.5 The neural network architecture for Refine net 21 Fig. 4.1 Pixel-wise labeling image in GoogleAPI 22 Fig. 4.2 The image with labeling error 23 Fig. 4.3 Our model structure 24 Fig. 4.4 Definition of our data set 25 Fig. 4.5 The flow chart of fine-tuning 26 Fig. 4.6 New label data of self-transfer learning 27 Fig. 4.7 The flow chart of fine-tuning 28 Fig. 4.8 New label data by transfer learning with modify part of data 29 Fig. 4.9 The flow chart of transfer learning with modify part of data 29 Fig. 4.10 Use original model to predict the results of general satellite imagery. 30 Fig. 4.11 New label data of transfer learning with noise 31 Fig. 4.12 The flow chart of transfer learning with noise 32 Fig. 4.13 Iterate the method of Transfer Learning with Noise 32 Fig. 4.14 Results obtained after iterate of Fig. 4.11 32 Fig. 4.15 Block-wise labeling data 33 Fig. 4.16 CNN with block-wise data 33 Fig. 4.17 Block-wise prediction 34 Fig. 4.18 The result of block-wise predicts 34 Fig. 4.19 Cut data into four blocks 35 Fig. 4.20 New label data of transfer learning with block-wise prediction 36 Fig. 4.21 The flow chart of transfer learning with block-wise prediction 36 Fig. 5.1 Each prediction result of the first category 38 Fig. 5.2 Do iteration on Transfer Learning with Modify Part of Data 39 Fig. 5.3 Each prediction accuracy of the first category 39 Fig. 5.4 Each prediction result of the second category 40 Fig. 5.5 Each prediction accuracy of the second category 40 Fig. 5.6 Do iteration on transfer learning with noise 41 Fig. 5.7 The amount of data changed to the correct label on each time 42 Fig. 5.8 Results of training the same data with many times 43 Fig. 5.9 Change block-wise label to the pixel-wise label 44 Fig. 5.10 Accuracy in the process of iteration 44 Fig. 5.11 The results we predicted on the test set after our algorithm 45

    Reference
    1. Chen, X.-W. and X. Lin, Big data deep learning: challenges and perspectives. IEEE access, 2014. 2: p. 514-525.
    2. Simonyan, K. and A. Zisserman, Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014.
    3. He, K., et al. Deep residual learning for image recognition. in Proceedings of the IEEE conference on computer vision and pattern recognition. 2016.
    4. Long, J., E. Shelhamer, and T. Darrell. Fully convolutional networks for semantic segmentation. in Proceedings of the IEEE conference on computer vision and pattern recognition. 2015.
    5. Ronneberger, O., P. Fischer, and T. Brox. U-net: Convolutional networks for biomedical image segmentation. in International Conference on Medical image computing and computer-assisted intervention. 2015. Springer.
    6. Lin, G., et al. RefineNet: Multi-path Refinement Networks for High-Resolution Semantic Segmentation. in Cvpr. 2017.
    7. Pan, S.J. and Q. Yang, A survey on transfer learning. IEEE Transactions on knowledge and data engineering, 2010. 22(10): p. 1345-1359.
    8. Raina, R., et al. Self-taught learning: transfer learning from unlabeled data. in Proceedings of the 24th international conference on Machine learning. 2007. ACM.
    9. Dai, W., et al. Eigentransfer: a unified framework for transfer learning. in Proceedings of the 26th Annual International Conference on Machine Learning. 2009. ACM.
    10. Rolnick, D., et al., Deep learning is robust to massive label noise. arXiv preprint arXiv:1705.10694, 2017.
    11. Parkhi, O.M., A. Vedaldi, and A. Zisserman. Deep face recognition. in BMVC. 2015.
    12. Baldi, P. and Y. Chauvin, Neural networks for fingerprint recognition. Neural Computation, 1993. 5(3): p. 402-418.
    13. Jean, N., et al., Combining satellite imagery and machine learning to predict poverty. Science, 2016. 353(6301): p. 790-794.
    14. Bengio, Y. Deep learning of representations for unsupervised and transfer learning. in Proceedings of ICML Workshop on Unsupervised and Transfer Learning. 2012.
    15. Lee, H., et al. Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations. in Proceedings of the 26th annual international conference on machine learning. 2009. ACM.
    16. Hastie, T., R. Tibshirani, and J. Friedman, Unsupervised learning, in The elements of statistical learning. 2009, Springer. p. 485-585.
    17. LeCun, Y., Y. Bengio, and G. Hinton, Deep learning. nature, 2015. 521(7553): p. 436.
    18. Goodfellow, I., et al., Deep learning. Vol. 1. 2016: MIT press Cambridge.
    19. Zhang, S., A.E. Choromanska, and Y. LeCun. Deep learning with elastic averaging SGD. in Advances in Neural Information Processing Systems. 2015.
    20. Girosi, F., M. Jones, and T. Poggio, Regularization theory and neural networks architectures. Neural computation, 1995. 7(2): p. 219-269.
    21. Simard, P.Y., D. Steinkraus, and J.C. Platt. Best practices for convolutional neural networks applied to visual document analysis. in null. 2003. IEEE.
    22. Krizhevsky, A., I. Sutskever, and G.E. Hinton. Imagenet classification with deep convolutional neural networks. in Advances in neural information processing systems. 2012.

    QR CODE