生物醫學專名識別是生物醫學資料探勘的一項重要的任務，其目的在於自動識別和分類生物醫學專名。近年來，深度神經網路，尤其是預訓練的語言模型，在生物醫學專名識別領域取得了令人巨大的進展。然而，由於缺乏大規模高品質的注釋資料和領域知識，其性能仍然有限。為了解決這個問題，我們提出了一個新的基於預訓練的BioBERT的多任務學習模型；該模型帶有一個新的注意力模組，可以將自動處理過的句法資訊集成到模型中。特別地，我們使用公開的NLP工具包獲取每個輸入句子的自動處理後的句法資訊，如詞性標籤、句法成分或依存關係。我們所提出的注意力模組，稱為組合特徵的注意力 (CFA)，可以從句法資訊中提取合適的特徵，之后對提取到特徵進行加權，以增強生物醫學專名識別的效果。此外，我們所提出的多任務學習 (MTL) 方法可以共用訓練過程中的所有參數，從不同的資料集中獲取有用資訊。我們在多個基準生物醫學專名識別資料集上進行了大量的實驗，並在所有資料集上獲得了最好的結果。我們也提供案例分析進一步表明所提出的CFA模組和完全共用的MTL方法在我們的模型中的重要性。

Biomedical named entity recognition (BioNER) is a basic and important task for biomedical text mining with the purpose of automatically recognizing and classifying biomedical entities. Recently, deep neural networks, especially the pre-trained language models, have made great progress for BioNER. However, because of the lack of high-quality and large-scale annotated datasets and relevant external knowledge, the capability of the BioNER system remains limited. To tackle the problem, we propose a novel multi-task learning model based on the pre-trained BioBERT with a new attention module to integrate the auto-processed syntactic information. We first use the open source NLP toolkits to process the input sentence and then obtain the corresponding syntactic information, e.g., part-of-speech labels, syntactic constituents, and dependency relations. Next, the proposed attention module, named combined feature attention (CFA), extracts appropriate features from the syntactic information and weights these features to enhance our model. Moreover, the proposed multi-task learning (MTL) method shares all parameters in the training step to capture useful information from different datasets. We have conducted numerous experiments on several benchmark BioNER datasets, and the results demonstrate our model outperforms others on all datasets. Case studies are also provided to prove the importance of the proposed CFA module and fully-shared MTL method used in our model.

Akdemir, A., & Shibuya, T. (2020). Analyzing the Effect of Multi-task Learning for Biomedical Named Entity Recognition. arXiv preprint, p. arXiv:2011.00425.
Campos, D., Matos, S., & Oliveira, J. L. (2013). Gimli: open source and high-performance biomedical name recognition. BMC Bioinformatics, p. 14(1):1.
Chai, Z., Jin, H., Shi, S., Zhan, S., Zhuo, L., & Yang, Y. (2022). Hierarchical shared transfer learning for biomedical named entity recognition. BMC bioinformatics, pp. 23(1), 1-14.
Crichton, G., Pyysalo, S., Chiu, B., & Korhonen, A. (2017). A neural network multi-task learning approach to biomedical named entity recognition. BMC bioinformatics, pp. 18(1), 1-14.
Dang, T. H., Le, H. Q., Nguyen, T. M., & Vu, S. T. (2018). D3NER: biomedical named entity recognition using CRF-biLSTM improved with fine-tuned embeddings of various linguistic information. Bioinformatics, pp. 34(20):3539–46.
Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv, p. preprint arXiv:1810.04805.
Doğan, R. I., Leaman, R., & Lu, Z. (2014). NCBI disease corpus: a resource for disease name recognition and concept normalization. Journal of biomedical informatics, pp. 47, 1-10.
Gerner, M., Nenadic, G., & Bergman, C. M. (2010). LINNAEUS: a species name identification system for biomedical literature. BMC bioinformatics, pp. 11(1), 1-17.
Habibi, M., LeonWeber, Neves, M., Wiegandt, D. L., & Leser, U. (2017). Deep learning with word embeddings improves biomedical named entity recognition. Bioinformatics, pp. 33(14), i37-i48.
Hao, B., Zhu, H., & Paschalidis, I. (2020, December). Enhancing clinical bert embedding using a biomedical knowledge base. In Proceedings of the 28th international conference on computational linguistics, (pp. 657-661).
Huang, K., Huang, D., Liu, Z., & Mo, F. (2020, November). A Joint Multiple Criteria Model in Transfer Learning for Cross-domain Chinese Word Segmentation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), (pp. 3873-3882).
Khan, M. R., Ziyadi, M., & AbdelHady, M. (2020). Mt-bioner: Multi-task learning for biomedical named entity recognition using deep bidirectional transformers. arXiv preprint, p. arXiv:2001.08904.
Kingma, D. P., & Ba, J. (2015). Adam: A Method for Stochastic Optimization. Proceedings of the 3rd International Conference on Learning Representations.
Kolchinsky, A., Lourenço, A., Wu, H. Y., Li, L., & Rocha, L. M. (2015). Extraction of pharmacokinetic evidence of drug–drug interactions from the literature. PloS one, pp. 10(5), e0122199.
Kulick, S., Bies, A., Liberman, M., Mandel, M., McDonald, R., & Palmer, M. (2004). Integrated annotation for biomedical information extraction. HLT-NAACL 2004 Workshop: Linking Biological Literature, Ontologies and Databases, (pp. 61–68).
Lee, J., Yoon, W., Kim, S., Kim, D. K., So, C. H., & Kang, J. (2020). BioBERT: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics, pp. 36(4), 1234-1240.
Lee, K. J., Hwang, Y. S., Kim, S., & Rim, H. C. (2004). Biomedical named entity recognition using two-phase model based on SVMs. Journal of biomedical informatics, pp. 37(6), 436-447.
Li, J., Zhang, Z., Li, X., & Chen, H. (2008). Kernel‐based learning for biomedical relation extraction. Journal of the American Society for Information Science and Technology, pp. 59(5), 756-769.
Liao, Z., & Wu, H. (2012). Biomedical named entity recognition based on skip-chain Crfs. In 2012 international conference on industrial control and electronics engineering (pp. 1495-1498). IEEE.
Liao, Z., & Zhang, Z. (2012). A generic classifier-ensemble approach for biomedical named entity recognition. Pacific-Asia Conference on Knowledge Discovery and Data Mining, (pp. 86-97).
Liu, H., Hu, Z. Z., & Torii, M. W. (2006). Quantitative assessment of dictionary-based protein named entity tagging. Journal of the American Medical Informatics Association, 13(5), 497-507.
Liu, S., Tang, B., Chen, Q., & Wang, X. (2016). Drug-drug interaction extraction via convolutional neural networks. Computational and mathematical methods in medicine.
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., & ... & Stoyanov, V. (2019). Roberta: A robustly optimized bert pretraining approach. arXiv, p. preprint arXiv:1907.11692.
Luo, L., Yang, Z., Yang, P., Zhang, Y., Wang, L., Lin, H., & Wang. (2018). An attention-based BiLSTM-CRF approach to document-level chemical named entity recognition. Bioinformatics. Bioinformatics, pp. 34(8), 1381-1388.
Manning, C. D., Surdeanu, M., Bauer, J., Finkel, J. R., Bethard, S., & McClosky, D. (2014, June). The Stanford CoreNLP natural language processing toolkit. In Proceedings of 52nd annual meeting of the association for computational linguistics: system demonstrations, (pp. 55-60).
Miller, A., Fisch, A., Dodge, J., Karimi, A. H., Bordes, A., & Weston, J. (2016). Key-value memory networks for directly reading documents. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, (pp. 1400-1409).
Pafilis, E., Frankild, S. P., Fanini, L., Faulwetter, S., Pavloudi, C., Vasileiadou, A., & Jensen, L. J. (2013). The SPECIES and ORGANISMS resources for fast and accurate identification of taxonomic names in text. PloS one, pp. 8(6), e65390.
Sang, E. F., & De Meulder, F. (2003). Introduction to the CoNLL-2003 shared task: Language-independent named entity recognition. arXiv, p. preprint cs/0306050.
Settles, B. (2004). Biomedical named entity recognition using conditional random fields and rich feature sets. In Proceedings of the international joint workshop on natural language processing in biomedicine and its applications (NLPBA/BioNLP), (pp. 107-110).
Tang, B., Cao, H., Wang, X., Chen, Q., & Xu, H. (2014). Evaluating word representation features in biomedical named entity recognition tasks. BioMed research international.
Tian, Y., Shen, W., Song, Y., & Xia, F. (2020). Improving biomedical named entity recognition with syntactic information. BMC bioinformatics, pp. 21(1), 1-17.
Tian, Y., Song, Y., Ao, X., Xia, F., Quan, X., Zhang, T., & Wang, Y. (2020). Joint Chinese word segmentation and part-of-speech tagging via two-way attentions of auto-analyzed knowledge. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, (pp. 8286-8296).
Tong, Y., Chen, Y., & Shi, X. (2021). A multi-task approach for improving biomedical named entity recognition by incorporating multi-granularity information. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, (pp. 4804-4813).
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., & Polosukhin. (2017). Attention is all you need. Advances in neural information processing systems, (p. 30).
Wang, X., Lyu, J., Dong, L., & Xu, K. (2019). Multitask learning for biomedical named entity recognition with cross-sharing structure. BMC bioinformatics, pp. 20(1), 1-13.
Wang, X., Zhang, Y., Ren, X., Zhang, Y., Zitnik, M., & Shang, J. (2019). Cross-type biomedical named entity recognition with deep multi-task learning. Bioinformatics, pp. 35(10), 1745-1752.
Wright, D. (2019). NormCo: Deep disease normalization for biomedical knowledge base construction. University of California, San Diego.
Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R. R., & Le, Q. V. (2019). Xlnet: Generalized autoregressive pretraining for language understanding. Advances in neural information processing systems, (p. 32).
Yao, L., Liu, H., Liu, Y., Li, X., & Anwar, M. W. (2015). Biomedical named entity recognition based on deep neutral network. Int. J. Hybrid Inf. Technol, pp. 8(8), 279-288.
Zhang, J., Shen, D., Zhou, G., Su, J., & Tan, C. L. (2004). Enhancing HMM-based biomedical named entity recognition by studying special phenomena. Journal of biomedical informatics, pp. 37(6), 411-422.
Zhang, Y., Lin, H., Yang, Z., Wang, J., Zhang, S., Sun, Y., & Yang, L. (2018). A hybrid model based on neural networks for biomedical relation extraction. Journal of biomedical informatics, pp. 81, 83-92.
Zuo, M., & Zhang, Y. (2020). Dataset-aware multi-task learning approaches for biomedical named entity recognition. Bioinformatics, pp. 36(15), 4331-4338.