相比於傳統光柵式與透鏡式的3D顯示方式，雙層分解式3D顯示技術可提供更高解析度且全方向性的3D立體視覺效果，並可藉由高刷新率的液晶螢幕進一步提升影像還原品質與3D深度的連續性，實現更為擬真並親臨實境的3D視覺饗宴。然而，因其所使用的迭代式光場分解演算法需要龐大記憶體頻寬存取與高運算複雜度，使得此3D顯示技術很難應用於一般行動裝置與消費性電子產品。在本論文中，我們延續實驗室先前提出的即時光場分解演算法硬體架構設計，進一步優化其核心運算單元面積與功耗，並設計高效率的外部記憶體資料存取方式，完成第一顆高速且低功耗的光場處理晶片並支援即時的雙層分解式3D立體顯示系統。
基於先前的光場分解演算法定點數硬體架構設計，我們進一步分析各運算階段的數值分布範圍，並根據光場資料的分解特性，對內積運算單元做乘法前量化以及最小化除法單元的平均量化誤差，同時改善二階內積運算的暫存器分割方式與提高核心運算的時脈閘控比例，最後實作的核心運算單元相較於先前版本可提升20%的時脈速度以及節省29%的面積。另一方面，我們分析此處理器對外部記憶體的半區塊式資料存取效率，並於FPGA端設計半區塊資料重分佈單元，以多個半區塊合併連續存取的方式進一步減少AXI指令數量以及指令間的延遲時間，最後實作的結果可減少90-96%的外部記憶體切換指令延遲時間並提升16-59%的AXI頻寬存取效率。
此晶片以台積電40奈米CMOS製程來製作，晶片面積為2.7×2.7 mm^2，並使用5.9 M的邏輯閘以及75.1 KB的晶片內部記憶體。其在秩數1/2/4的分解模式分別可操作在200/200/180 MHz，消耗283/442/971 mW，並達到HD (720p) 33/34/31 fps的光場分解速度，實現高速且低功耗的光場分解。另外，此晶片更可搭配我們高效率光場資料存取的FPGA開發平台，以及我們額外設計的雙層分解式3D立體顯示器原型機，實現即時的裸視3D立體顯示技術。

Compared to traditional 3D display technology (e.g., parallax barriers and lenticular lens), the dual-layer-factored 3D display technology can provide higher image resolution and 3D perception in all viewing directions, i.e., full-parallax 3D. This technology can improve image fidelity and 3D depth continuity with high-refresh-rate LCDs and give an immersive 3D viewing experience. However, the iterative light-field factorization algorithm for the factored display requires massive DRAM bandwidth and large computational complexity, which is difficult to support in portable devices and consumer electronics. In this thesis, we improve the area and power consumption of the light-field factorization engine based on previous work and design a high-efficiency DRAM access flow. We implement the first real-time low-power light-field factorization processor and demonstrate a real-time dual-layer-factored 3D display subsystem.

Based on the fixed-point light-field factorization engine in previous work, we analyze the dynamic range of each part, propose a factorized input truncation method before the inner products, and minimize the average truncation errors in the fixed-point division. We also refine the pipeline structure of two-stage inner products and increase the operand isolation and clock-gating ratio. Our implementation result can reduce the critical path and area of the light-field factorization engine by 20% and 29%, respectively. Furthermore, we analyze the efficiency of DRAM read/write commands with the half-block-based address mapping, propose a multi-block access flow and deploy a half-block rearrangement unit at FPGA to increase the AXI bus utilization and reduce the DRAM row-changed frequency. The half-block rearrangement method can reduce the bandwidth of DRAM activation and precharge commands by 90-96% and increase the AXI bus utilization of factorized data by 16-59%.

This chip is fabricated in TSMC 40nm CMOS technology. The die area is 2.7×2.7 mm^2 with 5.9 M gate counts and 75.1 KB on-chip SRAM. The chip supports rank-1/-2/-4 factorization at core frequency 200/200/180 MHz with consuming 283/442/971 mW. The maximum throughputs are HD (720p) 33/34/31 fps, which provides a high-speed, low-power light-field factorization. In addition, this chip can collaborate with our high-efficiency data-accessing FPGA platform and our dual-layer-factored 3D display prototype to demonstrate the real-time 3D display technology.

[1] Neil Dodgson, “Autostereoscopic 3D Displays,” in Computer, 2005, vol. 38, pp. 31–36.
[2] Jason Geng, “Three-dimensional display technologies,” in Adv. Opt. Photon., 2013, vol. 5, pp. 456–535.
[3] Kunio Sakamoto and Tsutomu Morii, “Multi-view 3D display using parallax barrier combined with polarizer,” in Advanced Free-Space Optical Communication Techniques/Applications II and Photonic Components/Architectures for Microwave Systems and Displays, 2006, vol. 6399, pp. 214–221.
[4] Tsubasa Kanebako and Yasuhiro Takaki, “Time-multiplexing display module for high-density directional display,” in Stereoscopic Displays and Applications XIX, 2008, vol. 6803, pp. 240–247.
[5] Hee-Jin Im, Byung-Joo Lee, Hyungki Hong, and Hyun-Ho Shin, “Autostereoscopic 60 view 3D using slanted lenticular lens arrays,” in Journal of Information Display, 2007, vol. 8, pp. 23–26.
[6] A. Stern and B. Javidi, “Three-Dimensional Image Sensing, Visualization, and Processing Using Integral Imaging,” in Proceedings of the IEEE, 2006, vol. 94, pp. 591–607.
[7] Douglas Lanman, Matthew Hirsch, Yunhee Kim, and Ramesh Raskar, “Content-Adaptive Parallax Barriers: Optimizing Dual-Layer 3D Displays Using Low-Rank Light Field Factorization,” in ACM SIGGRAPH Asia, 2010, p. 10.
[8] Gordon Wetzstein, Douglas Lanman, Matthew Hirsch, and Ramesh Raskar, “Tensor Displays: Compressive Light Field Synthesis Using Multilayer Displays with Directional Backlighting,” in ACM Trans. Graph., 2012, vol. 31, p. 11.
[9] Douglas Lanman, Gordon Wetzstein, Matthew Hirsch, Wolfgang Heidrich, and Ramesh Raskar, “Polarization Fields: Dynamic Light Field Display Using Multi-Layer LCDs,” in Proceedings of the 2011 SIGGRAPH Asia Conference, 2011, p. 10.
[10] Xuan Cao, Zheng Geng, Tuotuo Li, Mei Zhang, and Zhaoxing Zhang, “Accelerating decomposition of light field video for compressive multi-layer display,” in Opt. Express, 2015, vol. 23, pp. 34007–34022.
[11] K. Takahashi, Y. Kobayashi, and T. Fujii, “From Focal Stack to Tensor Light-Field Display,” in IEEE Transactions on Image Processing, 2018, vol. 27, pp. 4571–4584.
[12] Li-De Chen, “Temporal Fusion: Continuous Time Light Field Video Factorization,” in To be submitted to Transaction on Image Processing, 2022.
[13] Hao-Chien Cheng, “VLSI Architecture and Chip Design of Real-time Full-HD Light Field Factorization for Automultiscopic 3D Display,” in NTHU, 2020.
[14] An-You Zheng, “VLSI Architecture of Non-Negative Matrix Factorization Engine for Light Field Factorization,” in NTHU, 2020.
[15] Hai-Wei Chen, Jiun-Haw Lee, Bo-Yen Lin, Stanley Chen, and Shin-Tson Wu, “Liquid crystal display and organic light-emitting diode display: present status and future perspectives,” in Light: Science & Applications, 2017, vol. 7, p. 17168.
[16] Yuanhe Tang, Ruixia Zhang, Haiyang Gao, Kai Liu, Gaoxiang Zhao, Xusan Yang, Qing Li, Yuan Liang, Na Ye, Hanchen Liu, and Shulin Liu, “Partially light-controlled imaging system based on High Temperature Poly-Silicon Thin Film Transistor-Liquid Crystal Display,” in Opt. Express, 2010, vol. 18, pp. 10616–10626.
[17] Ying Tang, Zhenyang Zhu, Masahiro Toyoura, Kentaro Go, Kenji Kashiwagi, Issei Fujishiro, and Xiaoyang Mao, “ALCC-Glasses: Arriving Light Chroma Controllable Optical See-Through Head-Mounted Display System for Color Vision Deficiency Compensation,” in Applied Sciences, 2020, vol. 10.
[18] G Wetzstein, D Lanman, M Hirsch, and R Raskar, “Real-time Image Generation for Compressive Light Field Displays,” in Journal of Physics: Conference Series, 2013, vol. 415, p. 12045.
[19] A. Gershun, “The Light Field,” in Journal of mathematics and physics, 1939, vol. 18.
[20] Michael Landy and J. Anthony Movshon, “The Plenoptic Function and the Elements of Early Vision,” in Computational Models of Visual Processing, 1991, pp. 3–20.
[21] Marc Levoy and Pat Hanrahan, “Light Field Rendering,” in Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques, 1996, pp. 31–42.
[22] L-Q. Weng, L-D. Chen, H-C. Cheng, A-Y. Zheng, K-P. Lin, and C-T. Huang, “A HD 31fps 7×7-View Light-Field Factorization Processor for Dual-Layer 3D Factored Display,” in IEEE International Solid-State Circuits Conference, 2022.
[23] Daniel Lee and H. Sebastian Seung, “Algorithms for Non-negative Matrix Factorization,” in Advances in Neural Information Processing Systems, 2001, vol. 13.
[24] Vincent D. Blondel, Ngoc-Diep Ho, and Paul Van Dooren, “Weighted Nonnegative Matrix Factorization and Face Feature Extraction,” in Image and Vision Computing, 2007.
[25] J. Zhang, Z. Fan, D. Sun, and H. Liao, “Unified Mathematical Model for Multilayer-Multiframe Compressive Light Field Displays Using LCDs,” in IEEE Transactions on Visualization and Computer Graphics, 2019, vol. 25, pp. 1603–1614.
[26] Synthetic Light Field Archive [Online], “Available: https://web.media.mit.edu/~gordonw/SyntheticLightFields/index.php,” Accessed on: Feb. 19, 2022.
[27] Katrin Honauer, Ole Johannsen, Daniel Kondermann, and Bastian Goldluecke, “A dataset and evaluation methodology for depth estimation on 4D light fields,” in Asian Conference on Computer Vision, 2016.
[28] Low power light-field factorization and real-time dual-layer 3D factored display demonstration [Online], “Available: https://www.ee.nthu.edu.tw/vcslab/lfd_processor/index.html,” Accessed on: Feb. 22, 2022.