無監督式領域適應 (UDA) 是一種有用的技術，可以從標記良好的來源領域中為未標記的目標領域訓練模型。然而，缺乏目標數據的標籤使得傳統的模型壓縮方法難以同時達到良好的模型精度和壓縮率。在本論文中，我們提出了一種新的UDA通道剪枝方法，稱為TICUDA（以實例轉換壓縮無監督式領域適配模型），它利用變換後的實例來評估通道重要性。轉換後的實例是從來源領域中的數據生成的，以模擬目標領域中的數據。該生成基於兩個數據集的領域差距，由領域對抗訓練神經網路(DANN)測量。我們還提出了一種自適應演算法來動態確定轉換實例的適用性。在ImageCLEF-DA和Office31數據集上使用VGG16和ResNet50框架進行了實驗，以評估提出方法的性能。我們將TICUDA與利用全來源資訊的方法和使用正則化的剪枝方法進行了比較。結果表明，在模型精度、模型大小和計算成本方面，TICUDA 在稱為縮減因子的組合分數方面優於它們。

Unsupervised domain adaptation (UDA) is a useful technique to train models for an unlabeled target domain from a well-labeled source domain. However, the lack of labels for target data makes conventional model compression methods hard to achieve good model accuracy and compression ratio simultaneously. In this thesis, we propose a new channel pruning method for UDA, called TICUDA (Transformed Instances for Compressing Unsupervised Domain Adaptation), which utilizes the transformed instances to evaluate the channel importance. The transformed instances are generated from the data in the source domain to mimic the data in the target domain. The generation is based on the domain gap for two data-sets, measured by the Domain-Adversarial Training of Neural Networks (DANN). An adaptive algorithm is also proposed to dynamically determine the applicability of transformed instances.
Experiments that use VGG16 and ResNet50 frameworks on ImageCLEF-DA and Office31 datasets were conducted to evaluate the performance of the proposed method. We compared TICUDA with the method utilizing full source information and the pruning methods using regularization. The results show that TICUDA outperforms them in terms of a combinative score, called Reduction Factor, for model accuracy, model size, and the computation cost.

[1] Mei Wang and Weihong Deng. “Deep visual domain adaptation: A survey”.
Neurocomputing 312 (2018), pp. 135–153.
[2] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. “ImageNet classification
with deep convolutional neural networks”. Commun. ACM 60.6 (2017),
pp. 84–90.
[3] Karen Simonyan and Andrew Zisserman. “Very Deep Convolutional Networks
for Large-Scale Image Recognition”. 3rd International Conference on Learning
Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference
Track Proceedings. Ed. by Yoshua Bengio and Yann LeCun. 2015.
[4] Kaiming He et al. “Deep Residual Learning for Image Recognition”. 2016 IEEE
Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las
Vegas, NV, USA, June 27-30, 2016. IEEE Computer Society, 2016, pp. 770–
778.
[5] Mingsheng Long et al. “Learning Transferable Features with Deep Adaptation
Networks”. Proceedings of the 32nd International Conference on Machine
Learning, ICML 2015, Lille, France, 6-11 July 2015. Ed. by Francis R. Bach
and David M. Blei. Vol. 37. JMLR Workshop and Conference Proceedings.
JMLR.org, 2015, pp. 97–105.
[6] Jindong Wang et al. “Stratified Transfer Learning for Cross-domain Activity
Recognition”. 2018 IEEE International Conference on Pervasive Computing
and Communications, PerCom 2018, Athens, Greece, March 19-23, 2018. IEEE
Computer Society, 2018, pp. 1–10.
[7] Sinno Jialin Pan et al. “Domain Adaptation via Transfer Component Analysis”.
IEEE Trans. Neural Networks 22.2 (2011), pp. 199–210.
[8] Yaroslav Ganin and Victor S. Lempitsky. “Unsupervised Domain Adaptation
by Backpropagation”. Proceedings of the 32nd International Conference on Machine
Learning, ICML 2015, Lille, France, 6-11 July 2015. Ed. by Francis R.
Bach and David M. Blei. Vol. 37. JMLR Workshop and Conference Proceedings.
JMLR.org, 2015, pp. 1180–1189.
[9] Eric Tzeng et al. “Adversarial Discriminative Domain Adaptation”. 2017 IEEE
Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu,
HI, USA, July 21-26, 2017. IEEE Computer Society, 2017, pp. 2962–
2971.
[10] Xavier Glorot, Antoine Bordes, and Yoshua Bengio. “Domain Adaptation for
Large-Scale Sentiment Classification: A Deep Learning Approach”. Proceedings
of the 28th International Conference on Machine Learning, ICML 2011, Bellevue,
Washington, USA, June 28 - July 2, 2011. Ed. by Lise Getoor and Tobias
Scheffer. Omnipress, 2011, pp. 513–520.
[11] Muhammad Ghifary et al. “Deep Reconstruction-Classification Networks for
Unsupervised Domain Adaptation”. Computer Vision - ECCV 2016 - 14th European
Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings,
Part IV. Ed. by Bastian Leibe et al. Vol. 9908. Lecture Notes in Computer
Science. Springer, 2016, pp. 597–613.
[12] Konstantinos Bousmalis et al. “Domain Separation Networks”. Advances in Neural
Information Processing Systems 29: Annual Conference on Neural Information
Processing Systems 2016, December 5-10, 2016, Barcelona, Spain. Ed. by
Daniel D. Lee et al. 2016, pp. 343–351.
[13] Vadim Lebedev et al. “Speeding-up Convolutional Neural Networks Using Finetuned
CP-Decomposition”. 3rd International Conference on Learning Representations,
ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track
Proceedings. Ed. by Yoshua Bengio and Yann LeCun. 2015.
[14] Yong-Deok Kim et al. “Compression of Deep Convolutional Neural Networks
for Fast and Low Power Mobile Applications”. 4th International Conference on
Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016,
Conference Track Proceedings. Ed. by Yoshua Bengio and Yann LeCun. 2016.
[15] Cheng Tai et al. “Convolutional neural networks with low-rank regularization”.
4th International Conference on Learning Representations, ICLR 2016, San
Juan, Puerto Rico, May 2-4, 2016, Conference Track Proceedings. Ed. by Yoshua
Bengio and Yann LeCun. 2016.
[16] Chaohui Yu et al. “Accelerating Deep Unsupervised Domain Adaptation with
Transfer Channel Pruning”. International Joint Conference on Neural Networks,
IJCNN 2019 Budapest, Hungary, July 14-19, 2019. IEEE, 2019, pp. 1–8.
[17] Fuzhen Zhuang et al. “Supervised Representation Learning: Transfer Learning
with Deep Autoencoders”. Proceedings of the Twenty-Fourth International Joint
Conference on Artificial Intelligence, IJCAI 2015, Buenos Aires, Argentina,
July 25-31, 2015. Ed. by Qiang Yang and Michael J. Wooldridge. AAAI Press,
2015, pp. 4119–4125.
[18] Muhammad Ghifary, W. Bastiaan Kleijn, and Mengjie Zhang. “Domain Adaptive
Neural Networks for Object Recognition”. PRICAI 2014: Trends in Artificial
Intelligence - 13th Pacific Rim International Conference on Artificial
Intelligence, Gold Coast, QLD, Australia, December 1-5, 2014. Proceedings. Ed.
by Duc Nghia Pham and Seong-Bae Park. Vol. 8862. Lecture Notes in Computer
Science. Springer, 2014, pp. 898–904.
[19] Mingsheng Long et al. “Deep Transfer Learning with Joint Adaptation Networks”.
Proceedings of the 34th International Conference on Machine Learning,
ICML 2017, Sydney, NSW, Australia, 6-11 August 2017. Ed. by Doina Precup
and Yee Whye Teh. Vol. 70. Proceedings of Machine Learning Research. PMLR,
2017, pp. 2208–2217.
[20] Mingsheng Long et al. “Unsupervised Domain Adaptation with Residual Transfer
Networks”. Advances in Neural Information Processing Systems 29: Annual
Conference on Neural Information Processing Systems 2016, December 5-10,
2016, Barcelona, Spain. Ed. by Daniel D. Lee et al. 2016, pp. 136–144.
[21] Benoit Jacob et al. “Quantization and Training of Neural Networks for Efficient
Integer-Arithmetic-Only Inference”. 2018 IEEE Conference on Computer Vision
and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22,
2018. IEEE Computer Society, 2018, pp. 2704–2713.
[22] Geoffrey E. Hinton, Oriol Vinyals, and Jeffrey Dean. “Distilling the Knowledge
in a Neural Network”. CoRR abs/1503.02531 (2015). arXiv: 1503.02531.
[23] Song Han et al. “Learning both Weights and Connections for Efficient Neural
Networks”. CoRR abs/1506.02626 (2015). arXiv: 1506.02626.
[24] Hengyuan Hu et al. “Network Trimming: A Data-Driven Neuron Pruning Approach
towards Efficient Deep Architectures”. CoRR abs/1607.03250 (2016).
arXiv: 1607.03250.
[25] Jian-Hao Luo, Jianxin Wu, and Weiyao Lin. “ThiNet: A Filter Level Pruning
Method for Deep Neural Network Compression”. IEEE International Conference
on Computer Vision, ICCV 2017, Venice, Italy, October 22-29, 2017. IEEE
Computer Society, 2017, pp. 5068–5076.
[26] Hao Li et al. “Pruning Filters for Efficient ConvNets”. 5th International Conference
on Learning Representations, ICLR 2017, Toulon, France, April 24-26,
2017, Conference Track Proceedings. OpenReview.net, 2017.
[27] Wei Wen et al. “Learning Structured Sparsity in Deep Neural Networks”. Advances
in Neural Information Processing Systems 29: Annual Conference on
Neural Information Processing Systems 2016, December 5-10, 2016, Barcelona,
Spain. Ed. by Daniel D. Lee et al. 2016, pp. 2074–2082.
[28] Yihui He, Xiangyu Zhang, and Jian Sun. “Channel Pruning for Accelerating
Very Deep Neural Networks”. IEEE International Conference on Computer Vision,
ICCV 2017, Venice, Italy, October 22-29, 2017. IEEE Computer Society,
2017, pp. 1398–1406.
[29] Zhuang Liu et al. “Learning Efficient Convolutional Networks through Network
Slimming”. IEEE International Conference on Computer Vision, ICCV 2017,
Venice, Italy, October 22-29, 2017. IEEE Computer Society, 2017, pp. 2755–
2763.
[30] Jianbo Ye et al. “Rethinking the Smaller-Norm-Less-Informative Assumption
in Channel Pruning of Convolution Layers”. 6th International Conference on
Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May
3, 2018, Conference Track Proceedings. OpenReview.net, 2018.
[31] Pavlo Molchanov et al. “Pruning Convolutional Neural Networks for Resource
Efficient Inference”. 5th International Conference on Learning Representations,
ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings.
OpenReview.net, 2017.
[32] Pavlo Molchanov et al. “Importance Estimation for Neural Network Pruning”.
IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019,
Long Beach, CA, USA, June 16-20, 2019. Computer Vision Foundation / IEEE,
2019, pp. 11264–11272.
[33] Sergey Ioffe and Christian Szegedy. “Batch Normalization: Accelerating Deep
Network Training by Reducing Internal Covariate Shift”. Proceedings of the 32nd
International Conference on Machine Learning, ICML 2015, Lille, France, 6-11
July 2015. Ed. by Francis R. Bach and David M. Blei. Vol. 37. JMLR Workshop
and Conference Proceedings. JMLR.org, 2015, pp. 448–456.
[34] Kate Saenko et al. “Adapting Visual Category Models to New Domains”. Computer
Vision - ECCV 2010, 11th European Conference on Computer Vision,
Heraklion, Crete, Greece, September 5-11, 2010, Proceedings, Part IV. Ed. by
Kostas Daniilidis, Petros Maragos, and Nikos Paragios. Vol. 6314. Lecture Notes
in Computer Science. Springer, 2010, pp. 213–226.
[35] Adam Paszke et al. “Automatic Differentiation in PyTorch”. NIPS 2017 Workshop
on Autodiff. Long Beach, California, USA, 2017.
[36] Barbara Caputo et al. “ImageCLEF 2014: Overview and Analysis of the Results”.
Information Access Evaluation. Multilinguality, Multimodality, and Interaction.
Ed. by Evangelos Kanoulas et al. Cham: Springer International Publishing,
2014, pp. 192–211. isbn: 978-3-319-11382-1.