運用蘇木精和伊紅 (hematoxylin and eosin、H&E)及免疫組織 (chromogenic immunohistochemistry、IHC) 染色可處理大部分外科病理的標準臨床診斷。同時，電腦輔助診斷 (computer-aided diagnosis、CAD) 也迅速發展並運用於圖像分析與協助臨床診斷。國際臨床治療發展越來越多的生物指標 (biomarkers) 與疾病治療選擇或預後有關，當中有些指標的病理評估標準卻是複雜而容易造成診斷結果差異。
H&E和IHC染色可以清楚顯示整個細胞以及組織結構。然而，這些將所有結構都展現出來的特點在開發數位影像分析時，會造成電腦難以明確區別與分割染色訊息。近年來螢光染色技術 (fluorescence、FLUO)，與免疫螢光染色 (immunofluorescence、IF) 可以個別螢光標示不同目標結構並分開訊號分析，已廣泛地運用於基礎臨床研究。但可見光和螢光染色技術之間的顏色差異卻在人工智慧輔助的 (computer-assisted artificial intelligence、AI) 交叉分析中帶來挑戰。
在本研究中，我們開發一套顏色標準化和細胞核提取 (color normalization and nucleus extraction methods) 的AI辨識模式與工作流程用於分析乳癌腫瘤細胞辨識，以解決可見光與螢光染色方式之間的訊號辨識差異。在AI模型的訓練、測試和驗證階段中，病理醫師提供正確的標註，以確保AI模型的準確性並提供改進性能的反饋。對於H&E和FLUO影像的分析，識別腫瘤特徵的準確性分別為89.6%和80.5%。此項以細胞核為焦點的AI辨識模型也可應用於評估乳癌中的間質腫瘤浸潤淋巴細胞 (stromal tumor infiltrating lymphocytes、sTIL)，能準確區分腫瘤細胞和淋巴細胞，並計算淋巴細胞占總腫瘤內間質組織面積的百分比。
本研究結果顯示跨可見光與螢光染色的交叉辨識工作流程具有準確性，可以減低H&E和FLUO圖像辨識的壁壘，不同染色影像的資料庫可以交互運用，以便於擴大發展新的病理AI模型並同時降低開發成本，並輔助精準診斷。儘管AI模型能提供準確的量化評估，卻無法全面理解病理切片中複雜的細節和細胞關聯訊息，導致與真正的人工評估有落差，仍需再進一步改善。目前AI在外科病理領域最大的期望就是協助病理醫師處理複雜的量化或半量化的生物指標評估，提供更高效且精確的診斷，支持臨床決策的制定或是未來治療方針的改進。

Surgical pathology based on hematoxylin and eosin (H&E) stains and chromogenic immunohistochemistry (IHC) can handle the majority of clinical diagnostic work. Meanwhile, computer-aided diagnosis (CAD) is used for image analysis in support of precision clinical diagnosis. At clinical site, an increasing number of biomarkers have been discovered to be related to disease treatment selection or prognosis, and evaluation of certain factors can be quite complex.
H&E and IHC stains reveal entire tissue and cellular structures, which make these images difficult to distinctly distinguish and segment staining signals in developing digital image analysis. Alternatively, fluorescent (FLUO) and immunofluorescent (IF) staining technology with advantages such as channel independence and multiplex labeling is widely used in clinical research. The differences in coloration between visible and fluorescent dyes staining technologies can create challenges in cross-analysis, particularly when using computer-assisted artificial intelligence (AI) algorithms.
In this study, color normalization and nucleus extraction methods and a workflow for analyzing fluorescent nucleus staining images in breast cancer (BC) tumor recognition we developed to address the differences between staining technologies. In the training, testing and validation phases of AI tumor recognition models, correct annotations are provided by pathologists, which helps ensure the accuracy of the AI models and provides feedback to improve their performance. The workflow achieved an accuracy of 89.6% and 80.5% in identifying specific tumor features in H&E- and fluorescent-stained pathological images, respectively. This nuclei-focus recognition AI model was applied to assess tumor infiltrating lymphocytes (TIL) in BC, which accurately distinguished tumor cells and lymphocytes, and calculated the percentage of area occupied by lymphocytes over total intratumoral stromal area.
Our findings demonstrate that the barriers in identifying H&E and fluorescent images can be reduced by utilizing this AI model and workflow. The databases of different staining images can be mutually utilized, facilitating the expansion and development of novel pathology AI models while reducing development costs. However, there are still some limitations for AI in fully understanding the details and intercellular relationships present in tissue slides, and leads to discrepancies in interpretation between pathologists and AI.
Currently, the greatest expectation for AI in surgical pathology is to assist pathologists in handling complex quantitative or semi-quantitative biomarker evaluations. This aims to provide more efficient and precise diagnoses, supporting the formulation of clinical decisions or improving future treatment strategies.

1. Suvarna K, Layton C, Bancroft J. The Hematoxylins and Eosin. Theory and Practice of Histological Techniques (Eighth Edition): Elsevier; 2018.
2. Scarani P. Trees with blood-colored wood. Pathologica 2000; 92(4): 298-300.
3. Rosai J. The H&E technique: old mistress apologue. Pathologica 1998; 90(6): 739-42.
4. Rosai J. Why microscopy will remain a cornerstone of surgical pathology. Lab Invest 2007; 87(5): 403-8.
5. Coons AH, Creech HJ, Jones RN. Immunological Properties of an Antibody Containing a Fluorescent Group. Proceedings of the Society for Experimental Biology and Medicine 1941; 47(2): 200-2.
6. National Comprehensive Cancer Network. https://www.nccn.org/guidelines/category_1.
7. Hsu SM, Raine L, Fanger H. Use of avidin-biotin-peroxidase complex (ABC) in immunoperoxidase techniques: a comparison between ABC and unlabeled antibody (PAP) procedures. The journal of histochemistry and cytochemistry : official journal of the Histochemistry Society 1981; 29(4): 577-80.
8. Sheibani K, Tubbs RR. Enzyme immunohistochemistry: technical aspects. Seminars in diagnostic pathology 1984; 1(4): 235-50.
9. Kos Z, Roblin E, Kim RS, et al. Pitfalls in assessing stromal tumor infiltrating lymphocytes (sTILs) in breast cancer. NPJ Breast Cancer 2020; 6: 17.
10. Cross SS, Dennis T, Start RD. Telepathology: current status and future prospects in diagnostic histopathology. Histopathology 2002; 41(2): 91-109.
11. Weinstein RS, Descour MR, Liang C, et al. Telepathology overview: from concept to implementation. Hum Pathol 2001; 32(12): 1283-99.
12. Pantanowitz L, Sharma A, Carter AB, et al. Twenty Years of Digital Pathology: An Overview of the Road Travelled, What is on the Horizon, and the Emergence of Vendor-Neutral Archives. Journal of pathology informatics 2018; 9: 40.
13. Abels E, Pantanowitz L, Aeffner F, et al. Computational pathology definitions, best practices, and recommendations for regulatory guidance: a white paper from the Digital Pathology Association. The Journal of pathology 2019; 249(3): 286-94.
14. Fuchs TJ, Buhmann JM. Computational pathology: challenges and promises for tissue analysis. Computerized medical imaging and graphics : the official journal of the Computerized Medical Imaging Society 2011; 35(7-8): 515-30.
15. Hou L, Samaras D, Kurc TM, et al. Patch-based Convolutional Neural Network for Whole Slide Tissue Image Classification. Proceedings IEEE Computer Society Conference on Computer Vision and Pattern Recognition 2016; 2016: 2424-33.
16. U.S. Food and Drug Administration. https://www.fda.gov/.
17. Chen M, Zhang B, Topatana W, et al. Classification and mutation prediction based on histopathology H&E images in liver cancer using deep learning. npj Precision Oncology 2020; 4(1): 14.
18. Coudray N, Ocampo PS, Sakellaropoulos T, et al. Classification and mutation prediction from non-small cell lung cancer histopathology images using deep learning. Nat Med 2018; 24(10): 1559-67.
19. Kather JN, Pearson AT, Halama N, et al. Deep learning can predict microsatellite instability directly from histology in gastrointestinal cancer. Nat Med 2019; 25(7): 1054-6.
20. Courtiol P, Maussion C, Moarii M, et al. Deep learning-based classification of mesothelioma improves prediction of patient outcome. Nature Medicine 2019; 25(10): 1519-25.
21. Beck AH, Sangoi AR, Leung S, et al. Systematic analysis of breast cancer morphology uncovers stromal features associated with survival. Science translational medicine 2011; 3(108): 108ra13.
22. Wu J, Liu C, Liu X, et al. Artificial intelligence-assisted system for precision diagnosis of PD-L1 expression in non-small cell lung cancer. Mod Pathol 2022; 35(3): 403-11.
23. Baxi V, Edwards R, Montalto M, et al. Digital pathology and artificial intelligence in translational medicine and clinical practice. Mod Pathol 2022; 35(1): 23-32.
24. Sung H, Ferlay J, Siegel RL, et al. Global Cancer Statistics 2020: GLOBOCAN Estimates of Incidence and Mortality Worldwide for 36 Cancers in 185 Countries. CA Cancer J Clin 2021; 71(3): 209-49.
25. WHO Classification of Breast Tumours. 5th ed: International Agency of Research on Cancer (IARC); 2019.
26. Maiorano E, Regan MM, Viale G, et al. Prognostic and predictive impact of central necrosis and fibrosis in early breast cancer: results from two International Breast Cancer Study Group randomized trials of chemoendocrine adjuvant therapy. Breast Cancer Res Treat 2010; 121(1): 211-8.
27. Van den Eynden GG, Colpaert CG, Couvelard A, et al. A fibrotic focus is a prognostic factor and a surrogate marker for hypoxia and (lymph)angiogenesis in breast cancer: review of the literature and proposal on the criteria of evaluation. Histopathology 2007; 51(4): 440-51.
28. Fridman WH, Pages F, Sautes-Fridman C, et al. The immune contexture in human tumours: impact on clinical outcome. Nat Rev Cancer 2012; 12(4): 298-306.
29. Patel SP, Kurzrock R. PD-L1 Expression as a Predictive Biomarker in Cancer Immunotherapy. Mol Cancer Ther 2015; 14(4): 847-56.
30. Ohaegbulam KC, Assal A, Lazar-Molnar E, et al. Human cancer immunotherapy with antibodies to the PD-1 and PD-L1 pathway. Trends Mol Med 2015; 21(1): 24-33.
31. Núñez Abad M, Calabuig-Fariñas S, Lobo de Mena M, et al. Programmed Death-Ligand 1 (PD-L1) as Immunotherapy Biomarker in Breast Cancer. 2022; 14(2): 307.
32. Planes-Laine G, Rochigneux P, Bertucci F, et al. PD-1/PD-L1 Targeting in Breast Cancer: The First Clinical Evidences Are Emerging. A Literature Review. Cancers 2019; 11(7).
33. International Immuno-Oncology Biomarker Working Group on Breast Cancer. https://www.tilsinbreastcancer.org/.
34. Loi S, Michiels S, Adams S, et al. The journey of tumor-infiltrating lymphocytes as a biomarker in breast cancer: clinical utility in an era of checkpoint inhibition. Ann Oncol 2021; 32(10): 1236-44.
35. Denkert C, von Minckwitz G, Darb-Esfahani S, et al. Tumour-infiltrating lymphocytes and prognosis in different subtypes of breast cancer: a pooled analysis of 3771 patients treated with neoadjuvant therapy. Lancet Oncol 2018; 19(1): 40-50.
36. Denkert C, Wienert S, Poterie A, et al. Standardized evaluation of tumor-infiltrating lymphocytes in breast cancer: results of the ring studies of the international immuno-oncology biomarker working group. Mod Pathol 2016; 29(10): 1155-64.
37. Kim RS, Song N, Gavin PG, et al. Stromal Tumor-infiltrating Lymphocytes in NRG Oncology/NSABP B-31 Adjuvant Trial for Early-Stage HER2-Positive Breast Cancer. J Natl Cancer Inst 2019; 111(8): 867-71.
38. Huang J, Chen X, Fei X, et al. Changes of Tumor Infiltrating Lymphocytes after Core Needle Biopsy and the Prognostic Implications in Early Stage Breast Cancer: A Retrospective Study. Cancer research and treatment 2019; 51(4): 1336-46.
39. Cha YJ, Ahn SG, Bae SJ, et al. Comparison of tumor-infiltrating lymphocytes of breast cancer in core needle biopsies and resected specimens: a retrospective analysis. Breast Cancer Res Treat 2018; 171(2): 295-302.
40. Amgad M, Stovgaard ES, Balslev E, et al. Report on computational assessment of Tumor Infiltrating Lymphocytes from the International Immuno-Oncology Biomarker Working Group. NPJ Breast Cancer 2020; 6: 16.
41. Thomssen C, Balic M, Harbeck N, et al. St. Gallen/Vienna 2021: A Brief Summary of the Consensus Discussion on Customizing Therapies for Women with Early Breast Cancer. Breast Care (Basel) 2021; 16(2): 135-43.
42. Agilent Technologies Inc. PD-L1 IHC 22C3 pharmDx Interpretation Manual – Head and Neck Squamous Cell Carcinoma (HNSCC) 2019.
43. Agilent Technologies Inc. PD-L1 IHC 28-8 pharmDx Interpretation Manual - US Version 2015.
44. Ventana Medical Systems Inc. and Roche Diagnostics International, Inc. VENTANA PD-L1 (SP142) Assay Interpretation Guide for Non-Small Cell Lung Cancer ≥ 50% TC or ≥ 10% IC Stepwise Scoring Algorithm 2020.
45. Ventana Medical Systems Inc. VENTANA PD-L1 (SP263) Assay Staining in Urothelial Carcinoma Interpretation Guide 2017.
46. Zarella MD, Yeoh C, Breen DE, et al. An alternative reference space for H&E color normalization. PloS one 2017; 12(3): e0174489.
47. Vijh S, Saraswat M, Kumar S. A new complete color normalization method for H&E stained histopatholgical images. Applied Intelligence 2021; 51(11): 7735-48.
48. Boschman J, Farahani H, Darbandsari A, et al. The utility of color normalization for AI-based diagnosis of hematoxylin and eosin-stained pathology images. The Journal of pathology 2022; 256(1): 15-24.
49. Rosai J. Rosai and Ackerman's Surgical Pathology E-Book: Elsevier Health Sciences; 2011.
50. Sandler M, Howard A, Zhu M, et al. Mobilenetv2: Inverted residuals and linear bottlenecks. Proceedings of the IEEE conference on computer vision and pattern recognition 2018.
51. Ronneberger O, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. International Conference on Medical image computing and computer-assisted intervention 2015.
52. Mouroutis T, Roberts SJ, Bharath AA. Robust cell nuclei segmentation using statistical modelling. Bioimaging 2001; 6(2): 79-91.
53. Begelman G, Gur E, Rivlin E, et al. Cell nuclei segmentation using fuzzy logic engine. 2004 International Conference on Image Processing, 2004 ICIP '04; 2004. p. 2937-40.
54. Yang X, Li H, Zhou X. Nuclei Segmentation Using Marker-Controlled Watershed, Tracking Using Mean-Shift, and Kalman Filter in Time-Lapse Microscopy. IEEE Transactions on Circuits and Systems I: Regular Papers 2006; 53(11): 2405-14.
55. Stanton SE, Disis ML. Clinical significance of tumor-infiltrating lymphocytes in breast cancer. J Immunother Cancer 2016; 4: 59.
56. El Bairi K, Haynes HR, Blackley E, et al. The tale of TILs in breast cancer: A report from The International Immuno-Oncology Biomarker Working Group. NPJ Breast Cancer 2021; 7(1): 150.
57. Mahmoud SM, Paish EC, Powe DG, et al. Tumor-infiltrating CD8+ lymphocytes predict clinical outcome in breast cancer. J Clin Oncol 2011; 29(15): 1949-55.
58. Liu S, Lachapelle J, Leung S, et al. CD8+ lymphocyte infiltration is an independent favorable prognostic indicator in basal-like breast cancer. Breast Cancer Research 2012; 14(2): R48.
59. Gu-Trantien C, Loi S, Garaud S, et al. CD4⁺ follicular helper T cell infiltration predicts breast cancer survival. The Journal of clinical investigation 2013; 123(7): 2873-92.
60. Shou J, Zhang Z, Lai Y, et al. Worse outcome in breast cancer with higher tumor-infiltrating FOXP3+ Tregs : a systematic review and meta-analysis. BMC Cancer 2016; 16(1): 687.
61. Mahmoud SM, Lee AH, Paish EC, et al. The prognostic significance of B lymphocytes in invasive carcinoma of the breast. Breast Cancer Res Treat 2012; 132(2): 545-53.
62. Tsou P, Katayama H, Ostrin EJ, et al. The Emerging Role of B Cells in Tumor Immunity. Cancer Res 2016; 76(19): 5597-601.
63. Siliņa K, Rulle U, Kalniņa Z, et al. Manipulation of tumour-infiltrating B cells and tertiary lymphoid structures: a novel anti-cancer treatment avenue? Cancer immunology, immunotherapy : CII 2014; 63(7): 643-62.
64. Dieu-Nosjean MC, Antoine M, Danel C, et al. Long-term survival for patients with non-small-cell lung cancer with intratumoral lymphoid structures. J Clin Oncol 2008; 26(27): 4410-7.
65. Coppola D, Nebozhyn M, Khalil F, et al. Unique ectopic lymph node-like structures present in human primary colorectal carcinoma are identified by immune gene array profiling. The American journal of pathology 2011; 179(1): 37-45.
66. Visvader JE, Stingl J. Mammary stem cells and the differentiation hierarchy: current status and perspectives. Genes Dev 2014; 28(11): 1143-58.
67. Huang PW, Ouyang H, Hsu BY, et al. Deep-learning based breast cancer detection for cross-staining histopathology images. Heliyon 2023; 9(2): e13171.
68. Ventana Medical Systems Inc. VENTANA PD-L1 (SP263) Assay Interpretation Guide for Non-Small Cell Lung Cancer 1% Tumor Cell (TC) Scoring Algorithm 2021.
69. Taiwan society of pathology. https://www.twiap.org.tw/.
70. National Health Insurance Administration, Ministry of Health and Welfare. https://www.nhi.gov.tw/.