當今深度學習為顯學的年代，大多數自然語言處理任務都因深度學習有更
好的表現。然而文本推論、理解的部分屬於複雜的任務，若想從一個類神經網
絡的文本分類器中，去得到分類的歸因，往往不符合人類思考方式，且解釋性
不佳。因此本論文使用貼近人類思考方式的語言模板(Linguistic Pattern)為基礎，
在文本分類問題任務中，我們將以語言模板作為文章推論原因，並結合上當今
深度學習的方法，使文本分類系統具備高準確率及符合人思維的推論性。本研
究分三階段：類別語言模板生成、基於類別語言模板之文本表示法、基於類神
經網絡之文本分類模型。本研究方法的實驗結果於新聞讀者情緒語料上比對照
組多7%準確率；而於新聞主題語料上F1-score 和對照組比，有20%驚人成長。

Nowadays, the majority of Natural Language Processing (NLP) tasks have witnessed performance improvements due to the advancement of Deep Learning techniques. However, logical inference and language understanding remain difficult tasks in NLP. Unlike the human thinking process, the outcome produced by neural network-based text classifiers are usually difficult to interpret directly, and sometimes even unreasonable. Therefore, we deliver a method based on Linguistic Patterns that are closer to the human thinking process. Moreover, these patterns are easily readable. In this thesis, we will combine the linguistic-based as well as deep learning-based methods and try to achieve both high performances and interpretable inference results. There are three major steps in our method, namely, Linguistic Pattern-based Generation for Domain Knowledge, Domain Knowledge Linguistic Pattern Document Representation, and Text Classification Model based on Deep Neural Networks. Results show that our approach improves upon current state-of-the-art methods on emotion classification and news topic classification. Specifically, we observe a 7% absolute increase on the accuracy of emotion classification, and a 20% absolute improvement on F1-score of the topic classifier.

1. S.K. Bharti and K.S. Babu and S.K. Jena, “Automatic keyword extraction for text summarization: A survey”, arXiv CL., Apr. 2017.
2. G. Salton and M.J. McGill, Introduction to modern information retrieval, New York McGraw-Hill, 1983.
3. D.M. Blei, A.Y. Ng, and M.I. Jordan, “Latent dirichlet allocation”, Journal of Machine Learning Research, Vol. 3, pp. 993-1022, Jan. 2003.
4. Y. Matsuo and M. Ishizuka, “Keyword Extraction from a Single Document using Word Co-occurrence Statistical Information”, semanticscholar IR. FLAIRS, St. Augustine, Florida, USA, Jan. 2004, pp. 157-169.
5. M. Paquot and Y. Bestgen, “Distinctive words in academic writing: A comparison of three statistical tests for keyword extraction”, DIAL IR., 2009.
6. S. and V., Syntax-Based Collocation Extraction, Springer Netherlands, 2011.
7. T. Dunning, “Accurate methods for the statistics of surprise and coincidence”, aclweb CL., Vol. 19, No. 1, pp. 61-74, Jan. 1993.
8. S. Rose and D. Engel and N. Cramer and W. Cowley, Automatic keyword extraction from individual documents, Wiley Online Library, Mar. 2010.
9. T. Gruber, “A translation approach to portable ontology specifications”, Knowledge Acquisition, Vol. 5, No. 2, pp. 199-220, 1993.
10. 鍾明強, 基於Ontology架構之文件分類網路服務研究與建構, 碩士, 資訊工程學系, 國立成功大學, 台南市, 2004.
11. C.O. Alm and D. Roth and R. Sproat, “Emotions from text: machine learning for text-based emotion prediction”, semanticscholar HLT/EMNLP, Vancouver B.C. Canada, Oct. 2005.
12. A. Agarwal and B. Xie and I. Vovsha and O. Rambow and R. Passonneau, “Sentiment analysis of twitter data”, ACM LSM’11 Proceedings of the Workshop on Languages in Social Media, Portland Oregon, June 2011.
13. C.P. Yung, “Automatic Ontology Construction by Using a Chinese Parser and a Lexical Knowledge Base”, ndltd NLP, July 2009.
14. 宋啟聖, 詞網同義詞集的中文語意表達之研究, 碩士, 資訊科學系, 東吳大學, 台北市, 2003.
15. 陳信裕, 利用廣義知網及維基百科於劇本文件之廣告推薦, 碩士, 資訊工程學系, 國立臺灣師範大學, 台北市, 2016.
16. C.R. Li and C.H. Yu and H.H. Chen, “Predicting the Semantic Orientation of Terms in E-HowNet”, Computational Linguistics and Chinese Language Processing, Vol. 17, No. 2, pp. 21-36, June 2012.
17. G.E. Hinton and J.L. McClelland and D.E. Rumelhart, Parallel distributed processing: explorations in the microstructure of cognition, MIT Press Cambridge MA USA, 1986, ch3.
18. T. Mikolov and K. Chen and G. Corrado and J. Dean, “Efficient Estimation of Word Representations in Vector Space”, semanticscholar NLP CoRR, Vol. abs-1301-3781, Sep. 2013.
19. H. Peng and J. Li and Y. Song and Yaopeng Liu, “Incrementally Learning the Hierarchical Softmax Function for Neural Language Models”, AAAI NLP Thirty-First AAAI Conference on Artificial Intelligence, Feb. 2017.
20. Q. Le and T. Mikolov, “Distributed Representations of Sentences and Documents”, arXiv CL., May 2014.
21. C.M. Bishop, Neural Networks for Pattern Recognition, Clarendon Press, 1995.
22. B. Liu, Sentiment analysis and opinion mining, Morgan & Claypool Publishers, May 2012.
23. F. Sebastiani, “Machine learning in automated text categorization”, ACM Computing Surveys, Vol. 34, No. 1, March 2002.
24. X. Wei, W.B. Croft, “LDA-based document models for ad-hoc retrieval”, ciir SIGIR’06, Seattle WA USA, August 2006.
25. Y. Kim, “Convolutional Neural Networks for Sentence Classification”, aclweb Conference on Empirical Methods in Natural Language Processing EMNLP, pp. 1746–1751, Oct. 2014.
26. J.Y. Nie and J. Gao and J. Zhang and M. Zhou, “On the use of words and n-grams for Chinese information retrieval”, IRAL Proceedings of the fifth international workshop on on Information retrieval with Asian languages, pp. 141-148, Oct. 2000.
27. Y. LeCun and Y. Bengio and G. Hinton, “Deep learning”, NATURE, Vol. 521, May 2015.
28. P. Liu and X. Qiu and X. Huang, “Recurrent neural network for text classification with multi-task learning”, IJCAI Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, pp. 2873-2879, July 2016.
29. K. Greff and R.K. Srivastava and J. Koutník and B.R. Steunebrink and J. Schmidhuber, “LSTM: A Search Space Odyssey”, arxiv IEEE Transactions on Neural Networks and Learning Systems, Vol. 28, No. 10, Oct. 2017.
30. R. Dey and F.M. Salemt, “Gate-variants of Gated Recurrent Unit (GRU) neural networks”, IEEE 60th International Midwest Symposium on Circuits and Systems MWSCAS, Boston MA USA, Oct. 2017.
31. B. Karlik and A.V. Olgac, “Performance Analysis of Various Activation Functions in Generalized MLP Architectures of Neural Networks”, International Journal of Artificial Intelligence And Expert Systems IJAE, Vol. 1, No. 4, Aug. 2011.
32. A.L. Maas and A.Y. Hannun and A.Y. Ng, “Rectifier Nonlinearities Improve Neural Network Acoustic Models”, 30th International Conference on Machine Learning, JMLR:W&CP Vol 28, Atlanta Georgia USA, 2013.
33. Y.C. Chang and C.C. Chen and W.L. Hsu, “SEMANTIC FRAME-BASED APPROACH FOR READER- EMOTION DETECTION”, PACIS, 2015.
34. Y.C. Chang and C.C. Chen and W.L. Hsu, “A semantic frame-based intelligent agent for topic detection”, tmu Soft Computing, Vol. 21, No. 2, pp. 391-401, Feb. 2017.
35. K.H. Lin and C. Yang and H. Chen, “What emotions do news articles trigger in their readers?”, ACM SIGIR, pp. 733–734, New York, 2007.
36. J.P. Chen, “Linguistic Pattern-based Distributed Representation Method for Topic Detection on Social Media”, Dec. 2018.