對於立體的研究生物樣本，共軛焦顯微鏡是一種強而有力的工具。相較於其它類型的顯微鏡，共軛焦顯微鏡能提供解析度跟對比都比較好的影像。然而，相較於其橫向的解析度，共軛焦顯微鏡在Z軸方向擁有較差的解析度。這個現象限制了重建之三維生物樣本的體資料在空間上的可信度。為了提高Z軸方向的解析度，其中一種方法為Tilted-view Microscopy。藉由旋轉生物樣本，透過傳統的共軛焦顯微鏡可以獲得從不同視角觀測而得之生物樣本影像。原本因為Z軸解析度較差的關係而無法從單一視角觀察到之生物樣本資訊，現在有機會透過其他視角來取得。這些從不同視角觀測而得之生物樣本影像經過整合即可重建一個擁有相同橫向與縱向解析度之生物樣本體資料。
我們提出了一種影像融合的演算法來整合透過Tilted-view Microscopy所獲得之多角度影像，進而重建出擁有相同橫向與縱向解析度之生物樣本體資料。在這個演算法中，影像堆疊首先經過隨深度變化的反疊積(deconvolution)演算法來修復因為點擴散函數而產生的影像失真。然後，這些經過反疊積處理的影像堆疊再透過以特徵為基礎的對位演算法來進行整合。最後，沒有被這些多角度影像堆疊所擷取之生物樣本資訊則透過以亮度為基礎之內插演算法來進行估測。經過上述步驟以後，我們可以重建一個擁有相同橫向與縱向解析度之生物樣本體資料，其中擁有被多角度影像堆疊所擷取之真實生物樣本資訊和透過估測所產生之原本無法從多角度影像堆疊所獲得之生物樣本資訊。

Confocal laser scanning microscope (CLSM) is a powerful tool for studying biological specimens three-dimensionally. Compared with other microscopes, CLSM can provide images with higher resolution and better contrast. However, the resolution along the Z-axis (the optical axis) is much lower than that along the lateral directions. This phenomenon may hamper the spatial reliability of the reconstructed three dimensional volume data of the specimen. One way to increase the resolution in the Z-axis direction is Tilted-view Microscopy. By rotating the specimen, image stacks from different observation angles can be acquired with conventional CLSM. Missing information that can't be recorded from a single direction, due to the poor Z-axial resolution, can be recorded from other directions. Images derived from different observation angles are then combined to reconstruct one volume data with equal lateral and axial resolutions.
We propose an image fusion algorithm for the multi-angle image stacks derived by Tilted-view Microscopy to reconstruct a 3D volume data of the specimen that has equal lateral and axial resolutions. In this algorithm, image stacks are first deconvoluted with a depth-variant deconvolution method to recover the distortions caused by point spread functions. Then, deconvoluted image stacks are integrated through a feature-based registration algorithm. Finally, an intensity-based interpolation is applied to predict the absent information that is not recorded by these multi-angle images. As a result, a 3D volume data of the specimen with equal lateral and axial resolutions, which has the real points from multi-angle images and the predicted points that are not recorded by these images, is reconstructed.

[1] M. Minsky, “Microscopy Apparatus,” US Patent 3013467, 1961.
[2] P. Davidovits and M. D. Egger, “Scanning Optical Microscope,” US Patent 3643015, 1972.
[3] P. Davidovits and M. D. Egger, “Photomicrography of Corneal Endothelial Cells in vivo,” Nature, vol. 244, pp. 366–367, 1973.
[4] S. Wilhelm, B. Gröbler, M. Gluch, and H. Heinz, “Confocal Laser Scanning Microscopy. Principles,” Carl Zeiss, Inc., Jena, Germany.
[5] A. Egner and S. W. Hell, “Fluorescence microscopy with super-resolved optical sections,” TRENDS in Cell Biology, vol. 15, pp. 207–215, 2005.
[6] S. Herculano-Houzel, “The human brain in numbers: a linearly scaled-up primate brain,” Front. Hum. Neurosci., vol. 3, article 31, pp. 1–11, 2009.
[7] P. Kvello, B. B. Lofaldli, J. Rybak, R. Menzel, and H. Mustaparta, “Digital, three-dimensional average shaped atlas of the Heliothis virescens brain with integrated gustatory and olfactory neurons,” Front. Syst. Neurosci., vol. 3, article 14, 2009.
[8] B. B. Lofaldli, P. Kvello, and H. Mustaparta, “Integration of the antennal lobe glomeruli and three projection neurons in the standard brain atlas of the moth Heliothis virescens,” Front. Syst. Neurosci., vol. 4, article 5, 2010.
[9] K. Rein, M. Zockler, M. T. Mader, C. Grubel, and M. Heisenberg, “The Drosophila Standard Brain,” Current Biology, vol. 12, pp. 227–231, 2002.
[10] H. H. Lin, Jason S. Y. Lai, A. L. Chin, Y. C. Chen, and A. S. Chiang, “A Map of Olfactory Representation in the Drosophila Mushroom Body,” Cell, vol. 128, pp. 1205–1217, 2007.
[11] A. S. Chiang et al., “Three-dimensional reconstruction of brain-wide wiring networks in Drosophila at single-cell resolution,” Current Biology, vol. 21, pp. 1–11, 2011.
[12] R. Brandt, T. Rohlfing, J. Rybak, S. Krofczik, A. Maye, M. Westerhoff, H. C. Hege, and R. Menzel, “Three-dimensional Average-Shape Atlas of the Honeybee Brain and Its Applications,” J. Comparative Neurol., vol. 429, pp. 1–19, 2005.
[13] J. Rybaki, A. Kuß, H. Lamecker, S. Zachow, H. C. Hege, M. Lienhard, J. Singer, K. Neubert, and R. Menzel, “The digital bee brain: integrating and managing neurons in a common 3D reference system,” Front. Syst. Neurosci., vol. 4, article 30, 2010.
[14] A. E. Kurylas, T. Rohlfing, S. Krofczik, A. Jenett, and U. Homberg, “Standardized atlas of the brain of the desert locust, Schistocerca gregaria,” Cell Tissue Res., vol. 333, pp. 125–145, 2008.
[15] H. Peng, “Bioimage informatics: a new area of engineering biology,” Bioinformatics, vol. 24, pp. 1827–1836, 2008.
[16] H. Wei, B. el Jundi, U. Homberg, and M. Stengl, “Implementation of Pigment-Dispersing Factor-Immunoreactive Neurons in a Standardized Atlas of the Brain of the Cockroach Leucophaea maderae,” J. Comparative Neurol., vol. 518, pp. 4113–4133, 2010.
[17] S. R. Olsen and R. I. Wilson, “Cracking neural circuits in a tiny brain: new approaches for understanding the neural circuitry of Drosophila,” Trends in Neurosciences, vol. 31, pp. 512–520, 2008.
[18] J. D. Armstrong and J. I. van Hemert, “Towards a virtual fly brain,” Philos. Transact. A Math. Phys. Eng. Sci., vol. 367, pp. 2387–2397, 2009.
[19] J. H. Simpson, “Mapping and manipulating neural circuits in the fly brain,” Adv. Genet., vol. 65, pp. 79–143, 2009.
[20] B. el Jundi, W. Huetteroth, A. E. Kurylas, and J. Schachtner, “Anisometric Brain Dimorphism Revisited: Implementation of a Volumetric 3D Standard Brain in Manduca sexta,” J. Comparative Neurol., vol. 517, no. 20, pp. 210–225, 2009.
[21] Y. C. Liu and A. S. Chiang, “High-resolution confocal imaging and three-dimensional rendering,” Methods, vol. 30, pp. 86–93, 2003.
[22] E. A. Coutsias, C. Seok, and K. A. Dill, “Using Quaternions to Calculate RMSD,” Journal of Computational Chemistry, vol. 25, pp. 1849–1857, 2004.
[23] C. C. Wu, G. Y. Chen, Y. C. Chen, H. C. Shao, H. M. Chang, and Y. C. Chen, “Algorithm for the Creation of the Standard Drosophila Brain Model and its Coordinate System,” VIE 2008, pp. 478–483, 2008.
[24] Y. C. Chen, Y. C. Chen, and A. S. Chiang, “Two-level model averaging techniques in Drosophila brain imaging,” Proceeding of 2002 IEEE International Conference on Imaging Processing, vol. 2, pp. 941–944, 2002.
[25] M. G. L. Gustafsson, “Extended resolution flourescence microscopy,” Current Opinion in Structural Biology, vol. 9, pp. 627–634, 1999.
[26] S. Hell and E. H. K. Stelzer, “Properties of a 4Pi confocal fluorescence microscope,” J. Opt. Soc. Am. A, vol. 9, pp. 2159–2166, 1992.
[27] E. H. K. Stelzer and S. Lindek, “Fundamental reduction of the observation volume in far-field light microscopy by detection orthogonal to the illumination axis: confocal theta microscopy,” Opt. Commun., vol. 111, pp. 536–547, 1994.
[28] J. Bewersdorf, R. Schmidt, and S. W. Hell, “Comparison of I5M and 4Pi-microscopy,” Journal of Microscopy, vol. 222, pp. 105–117, 2006.
[29] M. G. L. Gustafsson, D. A. Agarg, and J. W. Sedat, “I5M: 3D widefield light microscopy with better than 100 nm axial resolution,” Journal of Microscopy, vol. 195, pp. 10–16, 1999.
[30] D. Laksameethanasan, S. S. Brandt, P. Engelhardt, O. Renaud, and S. L. Shorte, “A Bayesian Reconstruction Method for Micro-Rotation Imaging in Light Microscopy,” Microscopy Research and Technique, vol. 71, pp. 158–167, 2008.
[31] R. Heintzmann and C. Cremer, “Axial tomographic confocal fluorescence microscopy,” Journal of Microscopy, vol. 206, pp. 7–23, 2002.
[32] J. Bradl, B. Rinke, B. Schneider, P. Edelmann, H. Krieger, M. Hausmann, and C. Cremer, “Resolution improvement in 3-D microscopy by object tilting,” Microscopy and Analysis, vol. 44, pp. 9–11, 1996.
[33] C. J. Cogswell, K. G. Larkin, and H. U. Klemm, “Fluorescence microtomography: Multi-angle image acquisition and 3D digital reconstruction,” Proc. SPIE, vol. 2655, pp. 109–115, 1996.
[34] P. J. Shaw, D. A. Agard, Y. Hiraoka, and J. W. Sedat, “Tilted view reconstruction in optical microscopy: Three-dimensional reconstruction of Drosophila melanogaster embryo nuclei,” Biophysical Journal, vol. 55, pp. 101–110, 1989.
[35] R. Heintzmann, G. Kreth, and C. Cremer, “Reconstruction of axial tomographic high resolution data from confocal fluorescence microscopy: a method for improving 3D FISH images,” Anal. Cell. Pathol., vol. 20, pp. 7–15, 2000.
[36] C. H. Hsu, “Novel Confocal Microscopy System with Revolvable stage,” Master's thesis, National Tsing Hua University, Hsinchu, Taiwan, 2010.
[37] P. Matula, M. Kozubek, F. Staier, and M. Hausmann, “Precise 3D image alignment in micro-axial tomography,” Journal of Microscopy, vol. 209, pp. 126–142, 2003.
[38] D. G. Lowe, “Object Recongnition from Local Scale-Invariant Features,” In Proceedings of the International Conference on Computer Vision, pp. 1150–1157, 1999.
[39] D. G. Lowe, “Distinctive Image Features from Scale-Invariant Keypoints,” Int. J. Comput. Vision, vol. 60, pp. 91–110, 2004.
[40] A. Diaspro, S. Annunziata, M, Raimondo, P. Ramoino, and M. Robello, “A Single-Pinhole Confocal Laser Scanning Microscope for 3-D Imaging of Biostructures,” IEEE Eng. Med. Biol., vol. 18, pp. 106–110, 1999.
[41] B. Zhang, J. Zerubia, and J.-C. Olivo-Marin, “Gaussian approximations of fluorescence microscope point-spread function models,” Applied Optics, vol. 46, pp. 1819–1829, 2007.
[42] F. Aguet, D. Van De Ville, and M. Unser, “An accurate PSF model with few parameters for axially shift-variant deconvolution,” Proceedings of IEEE International Symposium on Biomedical Imaging: From Nano to Macro, pp. 157–160, 2008.
[43] S. F. Gibson and F. Lanni, “Experimental test of an analytical model of aberration in an oil-immersion objective lens used in three-dimensional light microscopy,” J. Opt. Soc. Am. A, vol. 8, pp. 1601–1613, 1991.
[44] P. Török, S. J. Hewlett, and P. Varga, “The role of specimen-induced spherical aberration in confocal microscopy,” Journal of Microscopy, vol. 188, pp. 158–172, 1997.
[45] A. Marian, F. Charrière, T. Colomb, F. Montfort, J. Kühn, P. Marquet, and C. Depeursinge, “On the complex three-dimensional amplitude point spread function of lenses and microscope objectives: theoretical aspects, simulations and measurements by digital holography,” Journal of Microscopy, vol. 255, pp. 156–169, 2006.
[46] P. Pankajakshan, L. Blanc-Féraud, Z. Kam, and J. Zerubia, “Point-spread function retrieval for fluorescence microscopy,” Proceedings of IEEE International Symposium on Biomedical Imaging: From Nano to Macro, pp. 1095–1098, 2009.
[47] P. Pankajakshan, “Blind Deconvolution for Confocal Laser Scanning Microscopy,” Ph. D. thesis, University of Nice Sophia-Antipolis, France, 2009.
[48] C. Preza and J.-A. Conchello, “Depth-variant maximum-likelihood restoration for three-dimensional fluorescence microscopy,” J. Opt. Soc. Am. A, vol. 21, pp. 1593–1601, 2004.
[49] P. Sarder and A. Nehorai, “Deconvolution Methods for 3-D Fluorescence Microscopy Images,” IEEE Sig. Proc. Mag. , vol. 23, pp. 32–45, 2006.
[50] W. Wallac, L. H. Schaefer, and J. R. Swedlow, “A Workingperson's Guide to Deconvolution in Light Microscopy,” Biotechniques, vol. 31, pp. 1076–1097, 2001.
[51] J. G. McNally, T. Karpova, J. Cooper, and J. A. Conchello, “Three-Dimensional Imaging by Deconvolution Microscopy,” Methods, vol. 19, pp. 373–385, 1999.
[52] J. Markham and J.-A. Conchello, “Parametric blind deconvolution: a robust method for the simultaneous estimation of image and blur,” J. Opt. Soc. Am. A, vol. 16, pp. 2377–2391, 1999.
[53] W. Kabsch, “A discussion of the solution for the best rotation to relate two sets of vectors,” Acta Crystallographica Section A, vol.34, pp. 827–828, 1978.
[54] N. S. Claxton, T. J. Fellers, and M. W. Davidson, “Laser scanning confocal microscopy,” http://www.olympusfluoview.com/theory/LSCMIntro.pdf.
[55] Y. C. Chen, “3D Deconvolution of FocusClear® Processed Transparent Biological Microscopic Images,” Master's thesis, National Tsing Hua University, Hsinchu, Taiwan, 2010.
[56] K. W. Wang, “An Image Registration Method for Aligning Different Views of Slice Image Stacks from Confocal Microscope,” Master's thesis, National Tsing Hua University, Hsinchu, Taiwan, 2010.
[57] H. A. Hsu, “Z-Axis Interpolation for Confocal Microscopy Images,” Master's thesis, National Tsing Hua University, Hsinchu, Taiwan, 2010.
[58] T. J. Holmes, “Blind deconvolution quantum-limited incoherent imagery: maximum-likelihood approach,” J. Opt. Soc. Am. A, vol. 9, pp. 1052–1061, 1992.
[59] W. Kabsch, “A Solution for The Best Rotation to Relate Two Sets of Vectors,” Acta Crystallographica Section A, vol.32, pp. 922–923, 1976.
[60] J. Kennedy and R. Eberhart, “Particle Swarm Optimization,” Proceedings of IEEE International Conference on Neural Networks, vol. 4, pp. 1942–1948, 1995.
[61] S. F. Wang, “Depth-variant Deconvolution of FocusClear® Processed Biological Microscopic Images,” Master's thesis, National Tsing Hua University, Hsinchu, Taiwan, 2011.
[62] M. A. Fischler and R. C. Bolles, “Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography,” Comm. ACM, vol. 24, pp. 381–395, 1981.
[63] M. Blume, D. Zikic, W. Wein, and N. Navab, “A New and General Method for Blind Shift-Variant Deconvolution of Biomedical Images,” Proc. MICCAI, Part I, pp. 743–750, 2007.
[64] N. Snavely, S. M. Seitz, and R. Szeliski, “Modeling the World from Internet Photo Collections,” International Journal of Computer Vision, vol. 80, pp. 189–210, 2008.
[65] S. Se and P. Jasiobedzki, “Stereo-Vision Based 3D Modeling and Localization for Unmanned Vehicles,” International Journal of Intelligent Control and Systems, vol. 13, pp. 46–57, 2008.
[66] J. S. Beis and D. G. Lowe, “Shape Indexing Using Approximate Nearest-Neighbour Search in High-Dimensional Spaces,” In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1000–1006, 1997.
[67] J. L. Bentley, “Multidimensional Binary Search Trees Used for Associative Searching,” Comm. ACM, vol. 18, pp. 509–517, 1975.