研究生: |
于妍之 Yu, Yen-Chih |
---|---|
論文名稱: |
基於FPGA定量微分相位差顯微鏡模組實現以深度學習優化之活細胞成像 FPGA-based Quantitative Differential Phase Contrast Microscopy for Live Cell-imaging optimized by Deep Learning |
指導教授: |
葉哲良
Yeh, J. Andrew 駱遠 Luo, Yuan |
口試委員: |
江振國
Chiang, Chen-Kuo 蔡孟勳 Tsai, Meng-Shiun 王玉麟 Wang, Yu-Lin 陳文翔 Chen, Wen-Shiang 黃宣銘 Huang, Hsuan-Ming |
學位類別: |
碩士 Master |
系所名稱: |
工學院 - 奈米工程與微系統研究所 Institute of NanoEngineering and MicroSystems |
論文出版年: | 2022 |
畢業學年度: | 110 |
語文別: | 英文 |
論文頁數: | 116 |
中文關鍵詞: | 各向同性定量微分相位差顯微術 、現場可程式化邏輯閘陣列 、深度學習 、活細胞成像 |
外文關鍵詞: | Isotropic Quantitative Differential Phase Contrast Microscopy, Field Programmable Gate Array, Deep Learning, Live Cell Imaging |
相關次數: | 點閱:3 下載:0 |
分享至: |
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報 |
各向同性定量微分相位差顯微術 ( Isotropic qDPC )於免標記的情況下可得到高解析度定量相位影像,適用於長時活細胞檢測及藥物治療相關研究。本論文目標為提升Isotropic qDPC系統之成像效率,並分別從顯微鏡系統模組、深度學習成像方法及生物醫學應用三個層面進行討論。
顯微鏡模組結合光學系統、電子硬體以及後處理計算單元。透過開發現場可程式化邏輯閘陣列(FPGA)可攜式顯微鏡模組,執行三個基本功能:照明控制、影像採集和影像重建。模組化後系統提升10倍的影像擷取速度,並達到輕巧化之目的。
深度學習成像方法使用U-net架構,分別訓練以一軸與兩軸強度影像作為輸入資訊的深度學習模型,實現微分相位差(DPC)強度影像至定量微分相位差(qDPC)之影像轉換,收集更完整的活細胞動態資訊。此方法可減少所需強度影像張數,並取代演算法複雜計算與參數設定,結合顯微鏡模組後大幅減少影像擷取時間,以優化成像效率。
生物醫學應用從成像條件、細胞三維資訊及影像動態變化等方面探討可結合之活細胞相關應用,並以Isotropic qDPC系統進行 COVID-19病毒感染細胞後細胞融合的長時觀測初步研究。研究結果顯示,與傳統的亮場顯微鏡相比,Isotropic qDPC成像可獲得厚度、外觀型態、內部結構以及乾物質總重等資訊用以評估細胞條件,說明Isotropic qDPC系統應用於生物醫學研究之潛力。
Quantitative phase imaging is becoming an important tool for biomedical research. It’s a challenging task to obtaining accurate quantitative structural information for thin and transparent biological objects. Isotropic Quantitative Differential Phase Contrast Microscopy is a newly label-free optical imaging method. A typical qDPC microscopic imaging requires an optical setup with electronically controlled dynamic illumination and a phase retrieval algorithm for image reconstruction. To reconstruct a single-phase image, multiple intensity measurements are acquired and post processed. Here, we propose a compact FPGA-based qDPC microscopy module with high speed image acquisition.
In addition to the development of FPGA-based qDPC microscope module, we implemented supervised deep learning methods for the single axis qDPC microscopy. Our trained model has potential to be directly integrated in FPGA-based qDPC module for high speed biomedical imaging.
We performed initial studies related to observing the biological changes in COVID-infected cells through the qDPC system. Our results show quantitative phase imaging performs better in evaluating the cell conditions in comparison to conventional bright field microscopy, and provide new directions for virus infected cell biological research.
[1] Jensen, E.C. "Overview of Live-Cell Imaging: Requirements and Methods Used." The Anatomical Record, 2013. 296(1), p. 1-8.
[2] Kasprowicz, R., Suman, R., and O'Toole, P. "Characterising live cell behaviour: Traditional label-free and quantitative phase imaging approaches." The international journal of biochemistry & cell biology, 2017. 84, p. 89-95.
[3] Lin, Y.H., Li, A.C., Vyas, S., Huang, Y.Y., Yeh, J.A., and Luo, Y. "Isotropic quantitative differential phase contrast microscopy using radially asymmetric color-encoded pupil." Journal of Physics: Photonics, 2021. 3(3), p. 035001.
[4] Nagy, G., Kiraly, G., and Banfalvi, G. "Chapter 7 - Optimization of Cell Cycle Measurement by Time-Lapse Microscopy." in Methods in Cell Biology. Vol. 112. Academic Press, 2012, p. 143-161.
[5] Lin, Y.Z., Huang, K.Y., and Luo, Y. "Quantitative differential phase contrast imaging at high resolution with radially asymmetric illumination." Opt Lett, 2018. 43(12), p. 2973-2976.
[6] Gabriel, M., Balle, D., Bigault, S., Pornin, C., Gétin, S., Perraut, F., Block, M.R., Chatelain, F., Picollet-D’hahan, N., Gidrol, X., and Haguet, V. " Time-lapse contact microscopy of cell cultures based on non-coherent illumination." Scientific Reports, 2015. 5(1), p. 14532.
[7] Schmidt, A.G., Weisz, G., and French, M. "Evaluating Rapid Application Development with Python for Heterogeneous Processor-Based FPGAs." in 2017 IEEE 25th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM), 2017.
[8] Dhanabal, R., Sahoo, S.K., Bharathi, V., Varma, B.S.R.P., Kalyan, D., and Divya, V. " FPGA based image processing unit." in 2015 IEEE 9th International Conference on Intelligent Systems and Control (ISCO), 2015.
[9] Yan, B., Sun, Y., Ding, F., and Yuan, H. " Design of CMOS image acquisition system based on FPGA." in 2011 6th IEEE Conference on Industrial Electronics and Applications, 2011.
[10] Jin, X., Jian, G., Xinfeng, Y., Hongkai, W., and Huangrong, X. "Implementing real time image processing algorithm on FPGA." in Proc.SPIE, 2020.
[11] Yu, Y.C., Yeh, J. A.,and Luo, Y. "FPGA control module for Quantitative Differential Phase Contrast Microscope." in Frontiers in Optics + Laser Science 2021. 2021.
[12] Jansen, B., Zimprich, P., and Hubner, M. "A dynamic partial reconfigurable overlay concept for PYNQ.", 2017, p. 1-4.
[13] Li, A.C., Vyas, S., Lin, Y.H., Huang, Y.Y., Huang, H.M., and Luo, Y. "Patch-based U-net Model for Isotropic Quantitative Differential Phase Contrast Imaging." IEEE Transactions on Medical Imaging, 2021. 40(11) , p. 3229-3237.
[14] Tian, L., Hunt, B., Bell, M.A.L., Yi, J., Smith, J.T., Ochoa, M., Intes, X., and Durr, N.J. "Deep Learning in Biomedical Optics." Lasers in Surgery and Medicine, 2021. 53(6), p. 748-775.
[15] Falk, T., Mai, D., Bensch, R., Çiçek, Ö., Abdulkadir, A., Marrakchi, Y., Böhm, A., Deubner, J., Jäckel, Z., Seiwald, K., Dovzhenko, A., Tietz, O., Dal Bosco, C., Walsh, S., Saltukoglu, D., Tay, T.L., Prinz, M., Palme, K., Simons, M., Diester, I., Brox, T., and Ronneberger, O. "U-Net: deep learning for cell counting, detection, and morphometry." Nature Methods, 2019. 16(1), p. 67-70.
[16] Wang, H., Rivenson, Y., Jin, Y., Wei, Z., Gao, R., Günaydın, H., Bentolila, L.A., Kural, C., and Ozcan, A. "Deep learning enables cross-modality super-resolution in fluorescence microscopy." Nature Methods, 2019. 16(1), p. 103-110.
[17] Sinha, A., Lee, J., Li, S., and Barbastathis, G. "Lensless computational imaging through deep learning." Optica, 2017. 4(9), p. 1117-1125.
[18] McCann, M.T., Jin, K.H., and Unser, M. "Convolutional Neural Networks for Inverse Problems in Imaging: A Review." IEEE Signal Processing Magazine, 2017. 34(6), p. 85-95.
[19] Rivenson, Y., Zhang, Y., Günaydın, H., Teng, D., and Ozcan, A. "Phase recovery and holographic image reconstruction using deep learning in neural networks." Light: Science & Applications, 2018. 7(2), p. 17141.
[20] Chen, H.-H., Lin, Y.Z., and Luo, Y. "Isotropic differential phase contrast microscopy for quantitative phase bio-imaging." Journal of Biophotonics, 2018. 11(8), p. e201700364.
[21] Liu, Y., Mollaeian, K., Shamim, M.H., and Ren, J. "Effect of F-actin and Microtubules on Cellular Mechanical Behavior Studied Using Atomic Force Microscope and an Image Recognition-Based Cytoskeleton Quantification Approach." International journal of molecular sciences, 2020, 21(2) , p. 392
[22] Pesapane, F., Codari, M., and Sardanelli, F. "Artificial intelligence in medical imaging: threat or opportunity? Radiologists again at the forefront of innovation in medicine." European Radiology Experimental, 2018. 2(1), p. 35.
[23] Diederich, B., Wartmann, R., Schadwinkel, H., and Heintzmann, R. "Using machine-learning to optimize phase contrast in a low-cost cellphone microscope." PloS one, 2018. 13(3), p. e0192937.
[24] Jo, Y., Cho, H., Lee, S.Y., Choi, G., Kim, G., Min, H.S., and Park, Y. " Quantitative Phase Imaging and Artificial Intelligence: A Review." IEEE Journal of Selected Topics in Quantum Electronics, 2019. 25, p. 2859234.
[25] Isola, P., Zhu, J., Zhou, T., and Efros, A.A. "Image-to-Image Translation with Conditional Adversarial Networks." in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
[26] Ronneberger, O., Fischer, P., and Brox, T. "U-Net: Convolutional Networks for Biomedical Image Segmentation." in 2015 International Conference on Medical image computing and computer-assisted intervention. 2015, p. 234-241.
[27] Driskell, R. R., Clavel, C., Rendl, M., and Watt, F. M. "Hair follicle dermal papilla cells at a glance." Journal of cell science, 2011. 124(8), p. 1179-1182.
[28] Pavez Lorie, E., Stricker, N., Plitta-Michalak, B., Chen, I.P., Volkmer, B., Greinert, R., Jauch, A., Boukamp, P., and Rapp, A. "Characterisation of the novel spontaneously immortalized and invasively growing human skin keratinocyte line HaSKpw." Scientific Reports, 2020. 10(1), p. 15196.
[29] Zhang, L., and Chan, C. "Isolation and enrichment of rat mesenchymal stem cells (MSCs) and separation of single-colony derived MSCs." Journal of Visualized Experiments, 2010. 37, p. e1852.
[30] Liu, T., Han, C., Wang, S., Fang, P., Ma, Z., Xu, L., and Yin, R. "Cancer-associated fibroblasts: an emerging target of anti-cancer immunotherapy." Journal of Hematology & Oncology, 2019. 12(1), p. 86.
[31] Groeger, S., and Meyle, J. "Oral Mucosal Epithelial Cells." Frontiers in Immunology, 2019. 10, p. 208.
[32] Siracusa, R., Fusco, R., and Cuzzocrea, S. "Astrocytes: Role and Functions in Brain Pathologies." Frontiers in Pharmacology, 2019. 10, p. 1114.
[33] Ryoo, S.-R., Kim, Y.K., Kim, M. H., and Min, D.H. "Behaviors of NIH-3T3 Fibroblasts on Graphene/Carbon Nanotubes: Proliferation, Focal Adhesion, and Gene Transfection Studies." ACS nano, 2020, 4(11), p. 6587-6598.
[34] Chen, W. J., Ho, C. C., Chang, Y. L., Chen, H. Y., Lin, C. A., Ling, T. Y., ... and Yang, P. C. "Cancer-associated fibroblasts regulate the plasticity of lung cancer stemness via paracrine signalling." Nature Communications, 2014. 5(1), p. 3472.
[35] Berger, K., Bangen, J.M., Hammerich, L., Liedtke, C., Floege, J., Smeets, B., and Moeller, M.J. "Origin of regenerating tubular cells after acute kidney injury." Proceedings of the National Academy of Sciences, 2014. 111(4), p. 1533-1538.
[36] Feng, X., Tustison, N.J., Patel, S.H., and Meyer, C.H. "Brain Tumor Segmentation Using an Ensemble of 3D U-Nets and Overall Survival Prediction Using Radiomic Features." Frontiers in Computational Neuroscience, 2020. 14.
[37] Hou, L., Samaras, D., Kurc, T.M., Gao, Y., Davis, J.E., and Saltz, J.H. "Patch-Based Convolutional Neural Network for Whole Slide Tissue Image Classification." in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2016.
[38] Kingma, D.P., and Ba, J. "Adam: A Method for Stochastic Optimization.", 2014.
[39] Horé, A., and Ziou, D. "Image Quality Metrics: PSNR vs. SSIM." in 2010 20th International Conference on Pattern Recognition, 2010.
[40] Zhou, W., Bovik, A.C., Sheikh, H.R., and Simoncelli, E.P. "Image quality assessment: from error visibility to structural similarity." IEEE Transactions on Image Processing, 2004. 13(4), p. 600-612.
[41] Aknoun, S., Yonnet, M., Djabari, Z., Graslin, F., Taylor, M., Pourcher, T., Wattellier, B., and Pognonec, P. "Quantitative phase microscopy for non-invasive live cell population monitoring." Scientific Reports, 2021. 11(1), p. 4409.
[42] Barer, R. "Determination of Dry Mass, Thickness, Solid and Water Concentration in Living Cells." Nature, 1953. 172(4389), p. 1097-1098.
[43] Liu, P.Y., Chin, L.K., Ser, W., Chen, H.F., Hsieh, C.M., Lee, C.H., Sung, K.B., Ayi, T.C., Yap, P.H., Liedberg, B., Wang, K., Bourouina, T., and Leprince-Wang, Y. " Cell refractive index for cell biology and disease diagnosis: past, present and future. " Lab on a Chip, 2016. 16(4), p. 634-644.
[44] Hu, C., and Popescu, G. "Quantitative Phase Imaging (QPI) in Neuroscience." IEEE Journal of Selected Topics in Quantum Electronics, 2019. 25, p. 1-9.
[45] Park, Y., Depeursinge, C., and Popescu, G. "Quantitative phase imaging in biomedicine." Nature Photonics, 2018. 12(10), p. 578-589.
[46] Cacace, T., Bianco, V., and Ferraro, P. "Quantitative phase imaging trends in biomedical applications." Optics and Lasers in Engineering, 2020. 135, p. 106188.
[47] Kreft, M., Vardjan, N., Stenovec, M., and Zorec, R. "Lateral drift correction in time-lapse images by the particle-tracking algorithm." European Biophysics Journal, 2008. 37(7), p. 1119.
[48] Gallego, R.D., Remohí, J., and Meseguer, M. "Time-lapse imaging: the state of the art." Biol Reprod, 2019. 101(6), p. 1146-1154.
[49] Khodjakov, A., and Rieder, C.L. "Imaging the division process in living tissue culture cells." Methods, 2006. 38(1), p. 2-16.
[50] Adler, J., and Pagakis, S.N. "Reducing image distortions due to temperature-related microscope stage drift." Journal of Microscopy, 2003. 210(2), p. 131-137.
[51] Park, J., Brureau, A., Kernan, K., Starks, A., Gulati, S., Ogunnaike, B., Schwaber, J., and Vadigepalli, R. "Inputs drive cell phenotype variability." Genome research, 2014. 24(6), p. 930-941.
[52] Quesenberry, P.J., and Aliotta, J.M. "Cellular phenotype switching and microvesicles." Advanced drug delivery reviews, 2010. 62(12), p. 1141-1148.
[53] Sommer, C.J. "Ischemic stroke: experimental models and reality." Acta Neuropathol, 2017. 133(2), p. 245-261.
[54] Tornabene, E., Helms, H.C.C., Pedersen, S.F., and Brodin, B. "Effects of oxygen-glucose deprivation (OGD) on barrier properties and mRNA transcript levels of selected marker proteins in brain endothelial cells/astrocyte co-cultures." PloS one, 2019. 14(8), p. e0221103-e0221103.
[55] Strazielle, N., and Ghersi-Egea, J.F. "Physiology of Blood–Brain Interfaces in Relation to Brain Disposition of Small Compounds and Macromolecules." Molecular Pharmaceutics, 2013. 10(5), p. 1473-1491.
[56] Elmore, S. "Apoptosis: a review of programmed cell death." Toxicologic pathology, 2007. 35(4), p. 495-516.
[57] Fink, S.L., and Cookson, B.T. "Apoptosis, Pyroptosis, and Necrosis: Mechanistic Description of Dead and Dying Eukaryotic Cells." Infection and Immunity, 2005. 73(4), p. 1907-
[58] Lowe, S.W., and Lin, A.W. "Apoptosis in cancer." Carcinogenesis, 2000. 21(3), p. 485-495.
[59] Fischer, U., and Schulze-Osthoff, K. "Apoptosis-based therapies and drug targets." Cell Death & Differentiation, 2005. 12(1), p. 942-961.
[60] Barker, K.L., Boucher, K.M., and Judson-Torres, R.L. "Label-Free Classification of Apoptosis, Ferroptosis and Necroptosis Using Digital Holographic Cytometry." Applied Sciences, 2020. 10(13), p. 4439.
[61] Chen, Y., Song, Y., Du, W., Gong, L., Chang, H., and Zou, Z. "Tumor-associated macrophages: an accomplice in solid tumor progression." Journal of Biomedical Science, 2019. 26(1), p. 78.
[62] Zhou, X., Liu, X., and Huang, L. "Macrophage-Mediated Tumor Cell Phagocytosis: Opportunity for Nanomedicine Intervention." Advanced Functional Materials, 2021. 31(5), p. 2006220.
[63] Liu, X., Wei, L., Xu, F., Zhao, F., Huang, Y., Fan, Z., Mei, S., Hu, Y., Zhai, L., Guo, J., Zheng, A., Cen, S., Liang, C., and Guo, F. "SARS-CoV-2 spike protein–induced cell fusion activates the cGAS-STING pathway and the interferon response." Science Signaling, 2022. 15(729), p. eabg8744.
[64] Park, S.B., Irvin, P., Hu, Z., Khan, M., Hu, X., Zeng, Q., Chen, C., Xu, M., Leek, M., Zang, R., Case, J.B., Zheng, W., Ding, S., Liang, T.J., and Shenk, T. "Targeting the Fusion Process of SARS-CoV-2 Infection by Small Molecule Inhibitors." MBio, 2022. 13(1), p. e03238-21.
[65] Hsieh, M.H., Beirag, N., Murugaiah, V., Chou, Y.-C., Kuo, W.-S., Kao, H.F., Madan, T., Kishore, U., and Wang, J.Y. "Human Surfactant Protein D Binds Spike Protein and Acts as an Entry Inhibitor of SARS-CoV-2 Pseudotyped Viral Particles." Frontiers in Immunology, 2021. 12, p. 1613.