光場顯示器不需要額外的眼部裝置，並且能夠在不犧牲解析度的前提之下，播放支援全視角的立體影像。由於擬真的立體觀影體驗，讓光場顯示器成為未來三維顯示器市場中的有力競爭者。然而，要做出能達到即時播放的光場顯示器仍有難度，因為每一個播放的立體影像背後都需要經過光場分解運算，而由於大量的光場資料以及遞迴式的運算，讓光場分解的運算複雜度與外部記憶體頻寬需求都很高。在本論文中，我們提出了演算法與系統架構的共同設計，讓此系統能夠提供足夠快的光場分解，並且只需要合理的外部記憶體頻寬，我們提出的系統架構將能夠支援即時播放的光場顯示器。

我們採用了區塊式的光場分解來避免遞迴式的存取外部記憶體，但如果想達到即時播放的規格，運算複雜度與外部記憶體頻寬需求還是太高。因此，我們修正了原本的更新流程來進一步減少最佳化時冗餘的光場資料以及遞迴次數。我們用稀疏化限制來減少最佳化的光場資料，以及色彩通道初始化來減少遞迴次數。接著，我們基於修改過的更新流程，提出了硬體實作的系統架構。為了提高運算速度，系統搭載了高吞吐量的光場分解引擎，進行以列為單位的平行更新。並且為了提高引擎的使用效率，此系統會透過半區塊記憶體輪轉的方式，重複使用區塊之間重疊的部分。

我們將此系統架構實做成台積電40 奈米製程的晶片。此晶片使用了6.5M的邏輯閘以及75.1KB 的晶片內部記憶體，並且總面積為3.9x3.9mm^2。當運行在200MHz 時，此晶片能夠在秩數1 的分解下提供每秒24 張的光場分解，並且需要的外部記憶體頻寬為3.4 GB/s，與以幀為單位更新的流程相比減少了約98%。

Light field displays, as a member of automultiscopic 3D display, can provide a full-parallax 3D scene to viewers without sacrificing resolution. The realistic 3D perception makes light field display to be a powerful candidate in future 3D display market. However, the light field factorization behind the displays leads to high computation complexity and DRAM bandwidth, which come from the dense constraints and iterative update, and makes the real-time light field display challenging. In this thesis, we propose an algorithm-architecture co-design to provide fast enough light field factorization with a reasonable DRAM bandwidth. The proposed system is able to support the real-time full-HD light field display.

We adopt Block-Based Light Field Factorization to avoid iteratively accessing DRAM, but the DRAM bandwidth and computation complexity is still high for real-time specification. Therefore, we propose a modified update flow to further optimize the redundant constraints and iteration. The constraint issue is solved by proposed Sparsified Constraint, and the iteration is reduced by Color Channel Initialization. Then, we propose a system architecture which accommodates to the modified flow. To increase the processing speed, the proposed system is equipped with high-throughput light field factorization engines with row-parallelism. And to maximize the utilization of engines, the system reuses the block overlap by half-block memory rotation.

Finally, we implement the proposed system as an ASIC in TSMC 40nm process. The chip uses 6.5M logic gates and 75.1KB on-chip memory, and the chip size is 3.9x3.9 mm^2. When operating at 200 MHz, our chip can provide up to 24 fps in rank 1 factorization. The required DRAM bandwidth for rank 1 factorization is 3.4 GB/s, which is 98% off comparing to the frame-based update.

[1] Frederic Eugene Ives, “Parallax stereogram and process of making same,”
U.S. Patent 725567, 1903.
[2] Kunio Sakamoto and Tsutomu Morii, “Multi-view 3D display using parallax
barrier combined with polarizer,” in Advanced Free-Space Optical Communication
Techniques/Applications II and Photonic Components/Architectures
for Microwave Systems and Displays. International Society for Optics and
Photonics, 2006, vol. 6399, pp. 214 – 221, SPIE.
[3] G. Lippmann, “Épreuves réversibles donnant la sensation du relief,” J. Phys.
Theor. Appl., vol. 7, no. 1, pp. 821–825, 1908.
[4] Dick K. G. de Boer, Martin G. H. Hiddink, Maarten Sluijter, Oscar H.
Willemsen, and Siebe T. de Zwart, “Switchable lenticular based 2D/3D
displays,” in Stereoscopic Displays and Virtual Reality Systems XIV. International
Society for Optics and Photonics, 2007, vol. 6490, pp. 276 – 283,
SPIE.
[5] I. Sexton and P. Surman, “Stereoscopic and autostereoscopic display systems,”
IEEE Signal Processing Magazine, vol. 16, no. 3, pp. 85–99, 1999.
[6] Ken Perlin, Salvatore Paxia, and Joel S. Kollin, “An autostereoscopic display,”
in Proceedings of the 27th Annual Conference on Computer Graphics
and Interactive Techniques(SIGGRAPH ’00). 2000, p. 319–326, ACM
Press/Addison-Wesley Publishing Co.
[7] T. Peterka, R. L. Kooima, D. J. Sandin, A. Johnson, J. Leigh, and T. A.
DeFanti, “Advances in the dynallax solid-state dynamic parallax barrier autostereoscopic visualization display system,” IEEE Transactions on Visualization
and Computer Graphics, vol. 14, no. 3, pp. 487–499, 2008.
[8] Yunhee Kim, Joohwan Kim, Jin-Mo Kang, Jae-Hyun Jung, Heejin Choi, and
Byoungho Lee, “Point light source integral imaging with improved resolution
and viewing angle by the use of electrically movable pinhole array,” Opt.
Express, vol. 15, no. 26, pp. 18253–18267, 2007.
[9] Douglas Lanman, Matthew Hirsch, Yunhee Kim, and Ramesh Raskar,
“Content-adaptive parallax barriers: Optimizing dual-layer 3d displays using
low-rank light field factorization,” in ACM SIGGRAPH Asia 2010 Papers.
2010, SIGGRAPH ASIA ’10, Association for Computing Machinery.
[10] Douglas Lanman, Gordon Wetzstein, Matthew Hirsch, Wolfgang Heidrich,
and Ramesh Raskar, “Polarization fields: Dynamic light field display using
multi-layer lcds,” in Proceedings of the 2011 SIGGRAPH Asia Conference.
2011, SA ’11, Association for Computing Machinery.
[11] Gordon Wetzstein, Douglas Lanman, Wolfgang Heidrich, and Ramesh
Raskar, “Layered 3d: Tomographic image synthesis for attenuation-based
light field and high dynamic range displays,” ACM Trans. Graph., vol. 30,
no. 4, 2011.
[12] Gordon Wetzstein, Douglas Lanman, Matthew Hirsch, and Ramesh Raskar,
“Tensor displays: Compressive light field synthesis using multilayer displays
with directional backlighting,” ACM Trans. Graph., vol. 31, no. 4, 2012.
[13] K. Takahashi, Y. Kobayashi, and T. Fujii, “From focal stack to tensor lightfield
display,” IEEE Transactions on Image Processing, vol. 27, no. 9, pp.
4571–4584, 2018.
[14] Xuan Cao, Zheng Geng, Tuotuo Li, Mei Zhang, and Zhaoxing Zhang, “Accelerating
decomposition of light field video for compressive multi-layer display,”
Opt. Express, vol. 23, no. 26, pp. 34007–34022, 2015.
[15] Li-De Chen, “Temporal fusion: Continuous time light field video factorization,”
To be submitted to SIGGRAGH 2021, 2020.
[16] Hai-Wei Chen, Jiun-Haw Lee, Bo-Yen Lin, Stanley Chen, and Shin-Tson
Wu, “Liquid crystal display and organic light-emitting diode display: present
status and future perspectives,” Light: Science & Applications, vol. 7, pp.
17168, 2017.
[17] Daniel Lee and H. Seung, “Learning the parts of objects by non-negative
matrix factorization,” Nature, vol. 401, pp. 788–91, 11 1999.
[18] Vincent D. Blondel, Ngoc diep Ho, Paul Dooren, Universite ́ Catholique De
Louvain, and Av. Georges Lemâıtre, “Weighted nonnegative matrix factorization
and face feature extraction,” in In Image and Vision Computing, 2008,
pp. 1–17.
[19] A.H. Andersen and A.C. Kak, “Simultaneous algebraic reconstruction technique
(sart): A superior implementation of the art algorithm,” Ultrasonic
Imaging, vol. 6, no. 1, pp. 81 – 94, 1984.
[20] G Wetzstein, D Lanman, M Hirsch, and R Raskar, “Real-time image generation
for compressive light field displays,” Journal of Physics: Conference
Series, vol. 415, pp. 012045, feb 2013.
[21] J. Zhang, Z. Fan, D. Sun, and H. Liao, “Unified mathematical model for
multilayer-multiframe compressive light field displays using lcds,” IEEE
Transactions on Visualization and Computer Graphics, vol. 25, no. 3, pp.
1603–1614, 2019.
[22] K. Maruyama, Y. Inagaki, K. Takahashi, T. Fujii, and H. Nagahara, “A
3-d display pipeline from coded-aperture camera to tensor light-field display
through cnn,” in 2019 IEEE International Conference on Image Processing
(ICIP), 2019, pp. 1064–1068.
[23] K. Maruyama, K. Takahashi, and T. Fujii, “Comparison of layer operations
and optimization methods for light field display,” IEEE Access, vol. 8, pp.
38767–38775, 2020.
[24] Jiangtao Yin, Lixin Gao, and Zhongfei (Mark) Zhang, “Scalable nonnegative
matrix factorization with block-wise updates,” in Machine Learning and Knowledge Discovery in Databases. 2014, pp. 337–352, Springer Berlin
Heidelberg.
[25] Synthetic Light Field Archive, “https://web.media.mit.edu/ gordonw/
syntheticlightfields/index.php,” .
[26] Katrin Honauer, Ole Johannsen, Daniel Kondermann, and Bastian Goldluecke,
“A dataset and evaluation methodology for depth estimation on 4d
light fields,” in Asian Conference on Computer Vision. Springer, 2016.
[27] Lytro Camera, “https://www.lytro.com/,” .
[28] An-You Zheng, “Vlsi architecture of non-negative matrix factorization engine
for light field factorization,” in NTHU, 2020.