程式碼的註解是理解程式行為的要素，而且註解將可使開發者更容易地維護程式。撰寫註解對於開發人員來說是費時費力的，因此可生成註解的自動化工具將可節省開發人員的精力。近來，數項研究開發自動化生成註解之工具。CODE-NN為最先應用深度學習技術於註解生成之研究，其利用長短期記憶神經網路對SQL指令與C#程式碼進行註解生成，並顯示深度學習方法表現優於資訊檢索方法。

Hybrid-DeepCom為針對Java語言進行註解生成之研究，其使用基於序列對序列架構及門控循環單元，並使用結構走訪方法表達抽象語法樹。然而CODE-NN及Hybrid-DeepCom皆面臨多元識別字導致的未登錄詞問題，以及循環神經網路所導致的梯度消失問題。
為了解決未登錄詞問題以及梯度消失問題，我們提出基於深度學習之方法 (ComCNN) 進行程式碼註解生成，ComCNN使用卷積神經網路、雙向長短期記憶神經網路做為模型架構。卷積神經網路可大幅減緩梯度消失問題，而雙向長短期記憶神經網路可學習雙向記憶。為了獲得更佳的識別字詞意學習效果，並解決未登錄詞問題，ComCNN亦使用識別字切割之方法對訓練資料進行預處理。

最後ComCNN使用數個度量方法，包含BLEU、CIDEr和ROUGE-L， 對生成的註解進行度量。在度量結果中，ComCNN得到優於比較模型 (CODE-NN、DeepCom和Hybrid-DeepCom) 3% ~ 7%的分數表現。在記憶體消耗方面，ComCNN獲得相對於比較模型41%、68%及69%之優化，而在訓練速度之方面，ComCNN擁有相對於比較模型1.7倍、2.9倍及23.9倍的訓練速度。

Source code is the key factor for understanding program behavior, and comments enable developers to maintain programs more easily. Producing comments is labor-consuming for developers; hence, automated tools for generating comments can considerably reduce the effort of developers. Recently, several studies have developed automatic tools to produce code comments. In the first study to apply deep learning techniques to comment generation, CODE-NN was used with the long short-term memory network (LSTM) model to predict the comments for SQL queries and C# code and demonstrated that the deep learning approach outperformed the information retrieval approach.

Hybrid-DeepCom, the research aiming to comment generation for Java language, used sequence-to-sequence architecture and gated recurrent units (GRU) architecture; furthermore, Hybrid-DeepCom employed the structure-based traversal (SBT) approach to represent the abstract syntax tree (AST). However, CODE-NN and Hybrid-DeepCom both face the out-of-vocabulary (OOV) problem caused by diverse identifiers and the vanishing gradient problem caused by recurrent neural networks.
To resolve the OOV and vanishing gradient problem, we proposed another deep learning based approach (ComCNN) to generate the comments for Java code. For ComCNN, convolutional neural networks (CNN) and bidirectional long short-term memory networks (Bi-LSTM) were employed as the model architecture. CNN was able to resolve the vanishing problem, and Bi-LSTM was able to learn the bidirectional memories of the source code. To achieve better semantic learning of identifiers and to resolve the OOV problem, the identifier splitting method was used with ComCNN to preprocess the training data.

Eventually, we employed several metrics, such as BLEU, CIDEr, and ROUGE-L to evaluate the generated comments. For all metrics, ComCNN achieved 3%–7% higher scores than the comparative models, such as CODE-NN, DeepCom, and Hybrid-DeepCom. ComCNN gained 41%, 68%, and 69% improvement on GPU memories and had 1.7×, 2.9×, and 23.9× the training speed of the comparative models.

[1] M. Chiang, C. Huang, C. Wu and C. Tsai, “Analysis of a Fault-Tolerant Framework for Reliability Prediction of Service-Oriented Architecture Systems,” IEEE Transactions on Reliability, Early Access, 2020.
[2] C. Hsu and C. Huang, “Optimal Weighted Combinational Models for Software Reliability Estimation and Analysis,” IEEE Transactions on Reliability, vol. 63, no. 3, pp. 731-749, Sep, 2014.
[3] C. Hsu and C. Huang, “An Adaptive Reliability Analysis Using Path Testing for Complex Component-Based Software Systems,” IEEE Transactions on Reliability, vol. 60, no. 1, pp. 158-170, Mar, 2011.
[4] C. Huang and C. Lin, “Analysis of Software Reliability Modeling Considering Testing Compression Factor and Failure-to-Fault Relationship,” IEEE Transactions on Computers, vol. 59, no. 2, pp. 283-288, Feb, 2010.
[5] S. Schach, Object-Oriented and Classical Software Engineering, Science Engineering & Math, July, 2010.
[6] A. Von Mayrhauser and A. M. Vans, “Program Comprehension during Software Maintenance and Evolution”, Computer, vol. 28, no. 8, pp. 44-55, Aug. 1995.
[7] S. C. B. de Souza, N. Anquetil, et al., “A Study of the Documentation Essential to Software Maintenance,” Proceedings of the 23rd Annual International Conference on Design of Communication: Documenting & Designing for Pervasive Information, pp. 68-75, New York, NY, USA, Sep, 2005.
[8] S. Iyer, I. Konstas, et al., “Summarizing Source Code using a Neural Attention Model,” Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), vol. 1, pp. 2073-2083, Berlin, Germany, Aug, 2016.
[9] X. Hu, G. Li, et al., “Deep Code Comment Generation,” Proceedings of IEEE/ACM International Conference on Program Comprehension (ICPC), pp. 200-210, New York, NY, USA, May, 2018.
[10] X. Hu, G. Li, et al., “Deep Code Comment Generation with Hybrid Lexical and Syntactical Information,” Empirical Software Engineering (ESE), vol. 25, pp. 2179-2217, Jun, 2019.
[11] C. Huang, Arthur, et al., “A Study of Applying Deep Learning-Based Weighted Combinations to Improve Defect Prediction Accuracy and Effectiveness,” Proceedings of IEEE International Conference on Industrial Engineering and Engineering Management (IEEM), pp. 1471-1475, Macao, Macao, Dec, 2019.
[12] Q. Mi, J. Keung, et al., “Improving Code Readability Classification Using Convolutional Neural Networks,” Information and Software Technology (IST), vol. 104, pp. 60-71, Dec, 2018.
[13] Y. Cheng, W. Xu, et al., “Semi-Supervised Learning for Neural Machine Translation,” Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), vol. 1, pp. 1965-1974, Berlin, Germany, Aug, 2016.
[14] Y. Wu, M. Schuster, et al., “Google’s Neural Machine Translation System: Bridging the Gap between Human and Machine Translation,” arXiv:1609.08144, Sep, 2016.
[15] G. Klein, Y. Kim, et al., “OpenNMT: Open-Source Toolkit for Neural Machine Translation,” Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), System Demonstrations, pp. 67-72, Vancouver, Canada, July, 2017.
[16] S. Huda, S. Alyahya, et al., “A Framework for Software Defect Prediction and Metric Selection,” IEEE Access, vol. 6, pp. 2844-2858, Dec, 2017.
[17] J. Li, P. He, et al., “Software Defect Prediction via Convolutional Neural Network,” Proceedings of IEEE International Conference on Software, Quality, Reliability and Security (QRS), pp. 318-328, Prague, Czech Republic, July, 2017.
[18] S. Wang, T. Liu, et al., “Automatically Learning Semantic Features for Defect Prediction,” Proceedings of International Conference on Software Engineering (ICSE), pp. 297-308, Austin, TX, USA, May, 2016.
[19] U. Alon, M. Zilberstein, et al., “code2vec: Learning Distributed Representations of Code,” Proceedings of the ACM on Programming Languages, no. 40, Jan, 2019.
[20] Y. Shido, Y. Kobayashi, et al., “Automatic Source Code Summarization with Extended Tree-LSTM,” Proceedings of International Joint Conference on Neural Networks (IJCNN), pp. 1-8, Budapest, Hungary, July, 2019.
[21] S. Haiduc, J. Aponte, et al., “Supporting Program Comprehension with Source Code Summarization,” Proceedings of ACM/IEEE International Conference on Software Engineering (ICSE), pp. 223-226, Cape Town, South Africa, May, 2010.
[22] S. Rastkar, G. C. Murphy, et al., “Generating Natural Language Summaries for Crosscutting Source Code Concerns,” Proceedings of IEEE International Conference on Software Maintenance (ICSE), pp. 103-112, Williamsburg, VI, USA, Sep, 2011.
[23] D. Movshovitz-Attias and W. W. Cohen, “Natural Language Models for Predicting Programming Comments,” Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), vol. 2, pp. 35-40, Sofia, Bulgaria, Aug, 2013.
[24] P. W. McBurney and C. McMillian, “Automatic Documentation Generation via Source Code Summarization of Method Context,” Proceedings of IEEE/ACM International Conference on Program Comprehension (ICPC), pp. 279-290, New York, NY, USA, Jun, 2014.
[25] T. Luong, I. Sutskever, et al., “Addressing the Rare Word Problem in Neural Machine Translation,” Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), vol. 1, pp. 11-19, Beijing, China, July, 2015.
[26] C. C. Huang, H. C Yen, et al., “Using Sublexical Translations to Handle the OOV Problem in Machine Translation,” ACM Transactions on Asian Language Information Processing (TALIP), vol. 10, iss. 3, no. 16, Sep, 2011.
[27] S. Hochreiter and J. Schmidhuber, “Long Short-Term Memory,” Neural Computation, vol. 9, iss. 8, pp. 1735-1780, Nov, 1997.
[28] N. Kalchbrenner, E. Grefenstette, et al., “A Convolutional Neural Network for Modelling Sentences,” Proceedings of the Association for Computational Linguistics (ACL), vol. 1, pp. 655-665, Baltimore, Maryland, Jun, 2014.
[29] A. Graves and J. Schmidhuber, “Framewise Phoneme Classification with Bidirectional LSTM and Other Neural Network Architectures,” Neural Network, vol. 18, iss. 5-6, pp. 602-610, July, 2005.
[30] M. Schuster and K. K. Paliwal, “Bidirectional Recurrent Neural Network,” IEEE Transactions on Signal Processing, vol. 45, iss. 11, pp. 2673-2681, Nov, 1997.
[31] D. Binkley, M. Davis, et al., “To Camelcase or Under_score,” Proceedings of IEEE International Conference on Program Comprehension (ICPC), pp. 158-167, Vancouver, Canada, May, 2009.
[32] B. Sharif and J. I. Maletic, “An Eye Tracking Study on camelCase and under_score Identifier Styles,” Proceedings of IEEE International Conference on Program Comprehension (ICPC), pp. 196-205, Braga, Minho, Portugal, July, 2010.
[33] Smartcomments, “SmartComments,” 2014. [Online]. Available: http://smartcomments.github.io/#home. [Accessed: 30- Jun- 2020]
[34] X. Wang and B. Zhang, “Comment Generation for Source Code: State of the Art, Challenges and Opportunities,” arXiv:1802.02971, Jan, 2018.
[35] E. Wong, J. Yang, et al., “AutoComment: Mining Question and Answer Sites for Automatic Comment Generation,” Proceedings of IEEE/ACM International Conference on Automated Software Engineering (ASE), pp. 562-567, Silicon Valley, CA, USA, Nov, 2013.
[36] M. Ohba and K. Gondow, “Toward Mining "Concept Keywords" from Identifiers in Large Software Projects,” Proceedings of International Workshop on Mining Software Repositories (MSR), pp. 1-5, New York, NY, USA, May, 2005.
[37] P. Rodeghero, C. McMillan, et al., “Improving Automated Source Code Summarization via an Eye-tracking Study of Programmers”, Proceedings of International Conference on Software Engineering (ICSE), pp. 390-401, New York, NY, USA, May, 2014.
[38] G. Sridhara, E. Hill, et al., “Towards Automatically Generating Summary Comments for Java Methods,” Proceedings of the IEEE/ACM International Conference on Automated Software Engineering (ASE), pp. 43-52, New York, NY, USA, Sep, 2010.
[39] L. Moreno, J. Aponte, et al., “Automatic generation of natural language summaries for Java classes,” Proceedings of International Conference on Program Comprehension (ICPC), San Francisco, CA, USA, pp. 23-32, May, 2013.
[40] V. I. Levenshtein, “Binary Codes Capable of Correcting Deletions, Insertions, and Reversals,” Doklady Physics, vol. 10, no. 8, pp.707-710, Feb, 1966.
[41] T. T. Ying and M. P. Robillard, “Code Fragment Summarization,” Proceedings of Joint Meeting on Foundations of Software Engineering (ESEC/FSE), pp. 655-658, New York, NY, USA, Aug, 2013.
[42] T. Hajie, “Automatic Comment Generation using a Neural Translation Model,” Jun, 2016.
[43] Y. Liang and Q. Zhu, “Automatic Generation of Text Descriptive Comments for Code Blocks,” Proceedings of Thirty-Second AAAI Conference on Artificial Intelligence, New Orleans, LA, USA, Feb, 2018.
[44] A. Takahashi, H. Shiina, et al., “Automatic Generation of Program Comments Based on Problem Statements for Computational Thinking,” Proceedings of International Congress on Advanced Applied Informatics (IIAI-AAI), pp. 629-634, Toyama, Japan, July, 2019.
[45] J. Zheng, X. Wang, et al., “Retrieval-based Neural Source Code Summarization,” Proceedings of 42nd International Conference on Software Engineering (ICSE), Seoul, South Korea, July, 2020.
[46] S. Liu, Y. Chen, et al., “Automatic Code Summarization via Multi-dimensional Semantic Fusing in GNN”, arXiv:2006.05405, Jun, 2020.
[47] K. S. Tai, R. Socher, et al., “Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks,” Proceedings of Annual Meeting of the Association for Computational Linguistics (ACL), vol. 1, pp. 1556-1566, Beijing, China, July, 2015.
[48] T. Mikolov, M. Karafiát, et al., “Recurrent Neural Network Based Language Model,” Proceedings of Conference of International Speech Communication Association (ISCA), pp. 1045-1048, Makuhari, Chiba, Japan, Jan, 2010.
[49] H. Schwenk, A. Rousseau, et al., “Pruned or Continuous Space Language Models on a GPU for Statistical Machine Translation,” Proceedings of the NAACL-HLT 2012 Workshop: Will We Ever Really Replace the N-gram Model? On the Future of Language Modeling for HLT, pp. 11-19, Montréal, Canada, June, 2012.
[50] A. M. Rush, S. Chopra, et al., “A Neural Attention Model for Abstractive Sentence Summarization,” Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 379-389, Lisbon, Portugal, Sep, 2015.
[51] C. Tillmann and H. Ney, “Word Reordering and a Dynamic Programming Beam Search Algorithm for Statistical Machine Translation,” Computational Linguistics, vol. 29, no. 1, pp. 97-133, Mar, 2003.
[52] M. Freitag and Y. Al-Onaizan, “Beam Search Strategies for Neural Machine Translation,” Proceedings of the First Workshop on Neural Machine Translation, pp. 56-60, Vancouver, Canada, Aug, 2017.
[53] P. S. Ow and T. E. Morton, “Filtered Beam Search in Scheduling,” International Journal of Production Research, vol. 26, iss. 1, pp. 35-62, May, 2007.
[54] P. Anderson, X. He, et al., “Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering,” Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6077-6086, Salt Lake City, UT, USA, Jun, 2018.
[55] O. Vinyals, A. Toshev, et al., “Show and Tell: A Neural Image Caption Generator,” Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3156-3164, Boston, MA, USA, Jun, 2015.
[56] D. Bahdanau, K. Cho, et al., “Neural Machine Translation by Jointly Learning to Align and Translate,” arXiv:1409.0473v7, May, 2016.
[57] S. Hochreiter, “The Vanishing Gradient Problem During Learning Recurrent Neural Nets and Problem Solutions,” International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems, vol. 6, no. 2, pp. 107-116, Apr, 1998.
[58] T. Mikolov, I. Sutskever, et al., “Distributed Representations of Words and Phrases and Their Compositionality,” Proceedings of International Conference on Neural Information Processing Systems (NIPS), vol. 2, pp. 3111-3119, Curran Associates Inc., Red Hook, NY, USA, Dec, 2013.
[59] B. O’Neill, Elementary Differential Geometry (Second Edition), Academic Press, 2006.
[60] E. P. Klement and R. Mesiar, Logical, Algebraic, Analytic and Probabilistic Aspects of Triangular Norms, Elsevier Science, 2005.
[61] K. Cho, B.V. Merriënboer, et al., “Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation,” Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1724-1734, Doha, Qatar, Oct, 2014.
[62] c2nes, “Javalang, Pure Python Java Parser and Tools,” 2012. [Online]. Available: https://github.com/c2nes/javalang. [Accessed: 19- Mar- 2020]
[63] H. A. Basit, S. J. Puglisi, et al., “Efficient Token Based Clone Detection with Flexible Tokenization,” Proceedings of the Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering (ESEC-FSE), pp. 513-516, New York, NY, USA, Sep, 2007.
[64] I. Sutskever, O. Vinyals, et al., “Sequence to Sequence Learning with Neural Networks,” Advances in Neural Information Processing Systems, vol. 4, Sep, 2014.
[65] G. Lample and F. Charton, “Deep Learning for Symbolic Mathematics,” arXiv:1912.01412, Dec, 2019.
[66] D. Adiwardana, M. T. Luong, et al., “Towards a Human-like Open-Domain Chatbot,” arXiv:2001.09977, Jan, 2020.
[67] H. Lim, J. Park, et al., “Rare Sound Event Detection Using 1D Convolutional Recurrent Neural Networks,” Proceedings of Detection and Classification of Acoustic Scenes and Events (DCASE), Munich, Germany, Nov, 2017.
[68] J. Wang, Y. Yang, et al., “CNN-RNN: A Unified Framework for Multi-label Image Classification,” Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2285-2294, Las Vegas, NV, USA, Jun, 2016.
[69] A. Ullah, J. Ahmad, et al., “Action Recognition in Video Sequences using Deep Bi-Directional LSTM With CNN Features,” IEEE Access, vol. 6, pp. 1155-1166, Nov, 2017.
[70] E. Grefenstette, P. Blunsom, et al., “A Deep Architecture for Semantic Parsing,” Proceedings of the ACL 2014 Workshop on Semantic Parsing, pp. 22-27, Baltimore, MD, USA, Jun, 2014.
[71] Y. Shen, X. He, et al., “Learning Semantic Representations Using Convolutional Neural Networks for Web Search,” Proceedings of the International Conference on World Wide Web, pp. 373-374, New York, NY, USA, Apr, 2014.
[72] S. Mittal, “A Survey of FPGA-based Accelerators for Convolutional Neural Networks,” Neural Computing and Applications, vol. 32, pp. 1109-1139, Oct, 2018.
[73] A. Giusti, D. C. Cireşan, et al., “Fast Image Scanning with Deep Max-pooling Convolutional Neural Networks,” Proceedings of IEEE International Conference on Image Processing (ICIP), pp. 4034-4038, Melbourne, VIC, Australia, Sep, 2013.
[74] T. Luong, H. Pham, et al., “Effective Approaches to Attention-based Neural Machine Translation,” Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1412-1421, Lisbon, Portugal, Sep, 2015.
[75] M. Sundermeyer, T. Alkhouli, et al., “Translation Modeling with Bidirectional Recurrent Neural Networks,” Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 14-25, Doha, Qatar, Oct, 2014.
[76] M. Berglund, T. Raiko, et al., “Bidirectional Recurrent Neural Networks as Generative Models,” Advances in Neural Information Processing Systems (NIPS), Cambridge, MA, USA, Apr, 2015.
[77] K. Xu, J. Ba, et al., “Show, Attend and Tell: Neural Image Caption Generation with Visual Attention,” Proceedings of the International Conference on Machine Learning (ICML), vol. 37, pp. 2048-2057, July, 2015.
[78] J. Chorowski, D. Bahdanau, et al., “Attention-Based Models for Speech Recognition,” Proceedings of the International Conference on Neural Information Processing Systems (NIPS), vol. 1, pp. 577-585, Cambridge, MA, USA, Dec, 2015.
[79] xing-hu, “Dataset for Comment Generation from DeepCom,” Available: https://github.com/xing-hu/DeepCom, 2018 [Accessed: 25- Mar- 2020]
[80] Google, “Google C++ Style Guide,” Available: https://google.github.io/styleguide/cppguide.html#Write_Short_Functions, [Accessed: 25- Mar- 2020]
[81] R. C. Martin, Clean Code: A Handbook of Agile Software Craftsmanship, Prentice Hall, Aug, 2008.
[82] M. Lippert and S. Roock, Refactoring in Large Software Projects, Wiley, May, May, 2006.
[83] “What Is the Ideal Length of a Method for You,” Available: https://softwareengineering.stackexchange.com/questions/133404/what-is-the-ideal-length-of-a-method-for-you, Feb, 2012 [Accessed: 7- Jul- 2020]
[84] D. P. Kingma and J. Ba, “Adam: A Method for Stochastic Optimization,” Proceedings of International Conference for Learning Representations (ICLR), San Diego, CA, USA, May, 2015.
[85] K. Papineni, S. Roukos, et al., “BLEU: a Method for Automatic Evaluation of Machine Translation,” Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), pp. 311-318, Philadelphia, PA, USA, July, 2002.
[86] R. Vedantam, C. L. Zitnick, et al., “CIDEr: Consensus-based Image Description Evaluation,” Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4566-4575, Boston, MA, USA, Jun, 2015.
[87] C. Y. Lin, “ROUGE: A Package for Automatic Evaluation of Summaries,” Text Summarization Branches Out: Proceedings of the ACL-04 Workshop, pp. 74-81, Barcelona, Spain, July, 2004.
[88] K. S. Jones, “A Statistical Interpretation of Term Specificity and Its Application in Retrieval,” Journal of Documentation, vol. 28, no. 1, pp. 11-21, Jan, 1972.