依據空間與時間相關性之適應性視差估算產生中間景物的分析與硬體設計

簡易檢索 / 詳目顯示

回結果列表

研究生：	李俊宜 Chun-Yi Li
論文名稱：	依據空間與時間相關性之適應性視差估算產生中間景物的分析與硬體設計 Analysis and Hardware Design of Intermediate View Generation Using Adaptive Disparity Estimation Based on Spatial and Temporal Correlation
指導教授：	陳永昌 Yung-Chang Chen
口試委員:
學位類別：	碩士 Master
系所名稱：	電機資訊學院 - 電機工程學系 Department of Electrical Engineering
論文出版年：	2007
畢業學年度：	95
語文別：	英文
論文頁數：	84
中文關鍵詞：	三維電視、視差估算、中間影像合成、方塊匹配、立體影像
外文關鍵詞：	3DTV, Disparity Estimation, Intermediate View Synthesis, Block Matching, Stereoscopic Image
相關次數：	點閱：2 下載：0
分享至:	分享至facebook 分享至twitter

查詢本校圖書館目錄查詢臺灣博碩士論文知識加值系統勘誤回報

3維電視(3D-TV)將會是高畫質電視(HDTV)下一步要走的路線。許多國內外知名大學與研究機構已經開始著力於如何設計具兼容性與彈性化的3DTV廣播系統。視訊的壓縮跟網路上的強健傳輸在3D-TV更是重要的課題之一。另外，立體影像如何呈現也是另一個熱門話題。未來，3D-TV技術可透過廣告看板、電影院、家庭電視、電腦螢幕等呈現，影像內容將會變得更逼真生動。但，由於頻寬有限，將所有視角的影像全部都傳輸過來勢必是行不通的，所以必須借助中間影像的合成，來符合人類視線環繞或者移動視差的特性，讓我們的眼睛更加舒服。
要算出立體影像的中間影像首先必須先估算出兩張影像的視差(Disparity)，而估算視差的方法有很多種，較適合硬體實現的不外乎是方塊匹配(Block Matching)。目前的文獻中，方塊匹配包括下列幾種方法：沿Epipolar line做全搜尋(Full Search)；二，使用階層式搜尋(Hierarchical Search)；三，使用四元樹搜尋(Quadtree-based Search)。第三個方法，比其他方法更可以降低合成之中間影像內物體邊緣閃爍(flicker)效應，提升PSNR比。
我們將使用方塊與方塊在空間與時間相關性質。在空間相關性，可以用來判斷我們方塊的搜尋範圍，算出更精確的視差估計(Disparity Estimation)；在時間相關性，我們的方法可以用來大幅度降低閃爍效應，另外在補洞(Hole-filling)部分會留下邊緣的特性。最後，會將中間影像合成的演算法使用硬體描述語言去設計硬體架構，並在Xilinx多媒體版來實現設計的原型。

Three dimensional television (3D-TV) will be the next step following high definition television (HDTV). Many famous universities and research organizations all over the world have concentrated on how to design a compatible and flexible broadcasting system of 3D-TV. Besides, another popular topic is the stereo image rendering. In the future, the technology of 3D-TV will be presented by advertisement boards, movie theaters, home TVs, and LCD monitors. Their contents become more photographic than before. But, it is impractical to transmit images of all view angles due to the limitation of bandwidth. In order to provide “look-around” and “motion-parallax” feeling to our eyes and let our eyes more comfortable, we use intermediate view synthesis.
First of all, we estimate the disparity of the left and right images before interpolating the intermediate view. There are many methods to estimate disparity. Block Matching is suitable for hardware implementation. Block Matching mainly includes the following methods: 1, Full Search along epipolar line. 2, Hierarchical Search. 3, Quadtree-based Search. The third method is better at reducing the flicker effect in edge part than other methods and gets higher PSNR.
We use the spatial correlation and temporal correlation between blocks. In the spatial correlation, we decide the search range adaptively to calculate the more correct disparity. In the temporal correlation, our method is good at reducing the flicker effect. In Hole-filling part, we retain the property of edge. Finally, we use Verilog HDL to develop the hardware architecture of the proposed algorithm and implement the prototype in Xilinx multimedia board.

Chapter 1: Introduction                                1
1 Overview of Stereo Vision                          1
2 Motivation                                         4
3 Thesis organization                                5
Chapter 2: Overview of Intermediate View Synthesis     6
1 Stereoscopic System                                6
1.1    Arbitrary Camera Model                        6
1.2    Parallel Camera Model                         8
2 Disparity Estimation Algorithm                     9
2.1    Overview of Disparity Estimation Algorithms   9
2.2    Block Matching Methods                        9
3 Intermediate View Synthesis (IVS)                  11
4 Main Challenge                                     14
Chapter 3: Adaptive Disparity Estimation for Intermediate  View Synthesis                                         18
1 Double Check Spatial and Temporal SAD              18
1.1    Adaptive disparity estimation on spatial correlation                                            18
1.2    Adaptive disparity estimation on temporal correlation                                            22
2 Two-Step Hole-Filling Algorithm                    24
2.1    The first step of hole-filling algorithm      24
2.2    The second step of hole-filling algorithm     25
Chapter 4: Hardware Design for Intermediate View Synthesis
                                                       27
1 Overview of the Hardware Architecture              28
2 The Access of External Memory                      32
3 Architecture of Adaptive Disparity Estimation      34
3.1    Architecture of AB2                           34
3.2    Adaptive Search Range Buffer                  35
3.3    Adaptive DE Controller                        36
4 Architecture of DCST                               37
4.1    Disparity Vector Buffer                       37
4.2    Spatial and Temporal SAD Calculation          39
4.3    DCST Controller                               39
5 Architecture of Synthesis                          41
5.1    View Synthesis                                41
5.2    Intermediate View Synthesis Buffer            42
5.3    Synthesis Controller                          43
6 Architecture of HFinFIFO                           44
6.1    Hole Detection Buffer                         44
6.2    Horizontal Bilinear Interpolator              45
6.3    Horizontal Bilinear Interpolation Controller  47
7 Architecture of HFinZBT                            48
7.1    Edge Dependent Bilinear Interpolator          48
7.2    Edge Dependent Bilinear Interpolation Controller                                             49
8 Operating Frequency                                51
Chapter 5: Simulation Results                          54
1 Software Simulation Environment                    55
2 The Results of Software Simulation                 57
2.1    Four Algorithms for Puppy and Worldcup2002    58
2.2    The Comparison of Our Proposed Method and [12]                                                   67
2.3    Real PSNR of Tri-Image Pair                   72
3 Environment Setting for Simulated Verification     75
3.1    Software Simulation for HW Verification       76
4 Environment of Xilinx FPGA Verification            77
Chapter 6: Conclusions and Future Works                80
1 Conclusions                                        80
2 Future works                                       80
References                                             82

                                

[1] Peter Hohenstatt, Leonardo da Vinci, 1998.
[2] Ubersicht, http://www.3d-historisch.de/
[3] Ubersicht, http://www.stereoviews.com
[4] http://www.inition.co.uk
[5] http://sharp-world.com
[6] Y. Wang, J. Ostermann, and Y. Zhang, “Video Processing and Communications,” Prentice Hall, pp. 374-396, 2002.
[7] A. Redert, E. Hendriks, and J. Biemond, “Correspondence Estimation in Image Pairs,” IEEE Signal Processing Magazine 16, pp. 29-46, May 1999.
[8] J. Karathanasis, D. Kalivas, and J. Vlontzos, “Disparity Estimation Using Block Matching and Dynamic Programming,” IEEE International Conference on Electronics, Circuits, and Systems (ICECS 1996), Vol. 2, 13-16, pp. 728 – 731, Oct. 1996.
[9] D. Fleet, A. Jepson, and M. Jenkin, “Phase-based Disparity Measurement,” CVGIP: Image Understanding 53, pp. 198-210, 1991.
[10] S. Chien, S. Yu, L. Ding, Y. Huang, and L. Chen, “Fast Disparity Estimation Algorithm for Mesh-based Stereo Image/Video Compression with Two-stage Hybrid Approach,” Visual Communications and Image Processing 2003. Proceedings of SPIE, Vol. 5150, pp. 1521-1530, June 2003.
[11] M. Accame, F. Natale, and D. Giusto, “Hierarchical Block Matching for Disparity Estimation in Stereo Sequences,” International Conference on Image Processing, 1995, Vol. 2, pp. 374-377, Oct. 1995.
[12] J. Sung, S. Lee, S. Kim, and J. Kim, “Quadtree-based Disparity Estimation for Intermediate View Synthesis of Stereoscopic Image Sequences,” Optical Engineering. Proceedings of SPIE, Vol. 44, Issue 3, pp. 034002-1-12, Mar. 2005.
[13] E. Izquierdo and J. Ohm, “Image-based Rendering and 3D Modeling: A Complete Framework,” Signal Processing: Image Communication, Vol. 15, Issue 10, pp. 817-858, Aug. 2000.
[14] M. Kim and K. Sohn, “Edge-preserving Directional Regularization Technique for Disparity Estimation of Stereoscopic Images,” IEEE Transaction on Consumer Electronics, Vol. 45, Issue 3, pp. 804-811, Aug. 1999.
[15] J. Ohm and E. Izquierdo, “An Object-based System for Stereoscopic Viewpoint Synthesis,” IEEE Transactions on Circuits and Systems for Video Technology, Vol. 7, No. 5, pp. 801 – 811, Oct. 1997.
[16] J. Oh and R. Park, “Reconstruction of Intermediate Views from Stereoscopic Images Using Disparity Vectors Estimated by the Geometrical Constraint,” IEEE Transactions on Circuits and Systems for Video Technology, Vol. 16, No. 5, pp. 638 –641, May 2006.
[17] M. Perez, C. Paqliari, and T. Dennis, “Stereo-based Intermediate View Synthesis with Realistic ‘Lookaround’ Capability,” Electronics Letters, Vol. 34, Issue 19, pp. 1840-1841, Sep. 1998.
[18] C. Paqliari, M. Perez, and T. Dennis, “Reconstruction of Intermediate Views from Stereoscopic Images Using A Rational Filter,” Proceedings of 1998 International Conference on Image Processing, 1998 (ICIP 98), Vol. 2, pp. 627-631, Oct. 1997.
[19] N. Shukia, S. Senqupta, and M. Chakraborty, “Intermediate View Synthesis in Wide-baseline Stereoscopic Video for Immersive Telepresence,” The 2nd European Workshop on the Integration of Knowledge, Semantics and Digital Media Technology, 2005. EWIMT 2005. (Ref. No. 2005/11099), pp. 83-88, Date. 30 Nov. - 1 Dec 2005.
[20] http://reasearch.microsoft.com/vision
[21] T. Komarek and P. Pirsch, “Array Architecture for Block Matching Algorithm,” IEEE Transactions on Circuits and Systems, Vol. 36, No. 10, pp. 1301 – 1308, Oct. 1989.
[22] https://www.3dtv-research.org
[23] C. Tsai and A. Katsaggelos, “Dense Disparity Estimation with A Divide-and-conquer Disparity Space Image Technique,” IEEE Transactions on Multimedia, Vol. 1, Issue.1, pp. 18-29, Mar. 1999.
[24] W. Ho and N. Ahuja, “Surfaces from Stereo: Integrating Feature Matching, Disparity Estimation, and Contour Detection,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 11, Issue. 2, pp. 121- 136, Feb. 1989.
[25] K. Bae, J Kim and E. Kim, “New Disparity Estimation Scheme Based on Adaptive Matching Windows for Intermediate View Reconstruction,” Society of Photo-Optical Instrumentation Engineers, Vol.42, Issue 6, pp. 1778-1786, June 2003.
[26] XILINX, “Virtex-Ⅱ Platform FPGA User Guide”, UG002(v1.4) Nov. 1, 2002.
[27] XILINX, “MicroBlaze and Multimedia Development Board User Guide”, UG020(v1.0) Aug. 29, 2002.

全文公開日期本全文未授權公開 (校內網路)
全文公開日期本全文未授權公開 (校外網路)

簡易檢索 / 詳目顯示

相關論文