在這篇論文中，我們使用卷積神經網絡建構三維點雲多物件辨識系統~(3D point cloud multi-object detection system)~。首先，探討三維點雲在深度學習領域不同的應用，包含：點雲分類、場景分割和物件辨識。不同於目前多數的物件辨識網路模型使用二維影像進行模型的訓練和辨識，我們僅以三維點雲資料作為模型訓練的資料來源。在完成合適的訓練後，偵測器~(detector)~在立體空間中可成功辨識不同的目標和其所在位置。當目標物受干擾，例如：遮蔽和稀疏化，利用部分的的點雲進行辨識，在具有足夠的特徵點雲時，偵測器亦具有良好的辨識能力。此外，為避免訓練模型發生過擬合~(overfitting )~和提升訓練資料的多樣性，我們加入了點雲的資料增強層~(data augmentation layer)~，對於標記資料進行調整，以提升模型的訓練效率。最後，作為點雲辨識技術的應用案例，我們以預測邊界框量測魚體長度進行示範。

In this thesis, we have developed a multi-object detection system using 3D point cloud from a chaos lidar based on a convolutional neural network. In contrast to most object recognition models that use 2D photos for recognition and other recognition models that use photos combined with depth data, only 3D point cloud information is used in our training model. To expand the diversity of limited point clouds for training, we add data augmentation layers and train them together with the original labeled data. Through our optimized training network model, multiple objects can be successfully identified even with partial occlusion or sparsity in data. By detecting fish models with different sizes, we demonstrate the feasibility of our multi-object detection system.

[1] R. Halterman, M. Bruch, “Velodyne HDL-64E lidar for unmanned surface vehicle
obstacle detection,” Proc. SPIE, vol. 7692, pp. 224-231, 2010.
[2] J. Redmon, S. Divvala, R. Girshick and A. Farhadi, “You only look once: unified, real-time object detection,” IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 779-788, 2016.
[3] A. H. Lang, S. Vora, H. Caesar, L. Zhou, J. Yang and O. Beijbom, “PointPillars: fast encoders for object detection from point clouds,” IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 12689-12697, 2019.
[4] Y. Guo, H. Wang, Q. Hu, H. Liu, L. Liu and M. Bennamoun, “Deep learning for 3D Point Clouds: a survey,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 43, no. 12, pp. 4338-4364, 1 Dec. 2021.
[5] Nancy Kenney, “Laser scan may one day aid Notre Dame’s restorers,”THE ART NEWSPAPER, 2019.
[6] A. Voulodimos, N. Doulamis, Anastasios Doulamis, E. Protopapadakis, “Deep learning for computer vision: a brief review,” Computational Intelligence and Neuroscience, 2018.
[7] Z. Wu, S. Song, A. Khosla, F. Yu, L. Zhang, X. Tang and J. Xiao, “ 3D ShapeNets: A Deep Representation for Volumetric Shapes,” IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015.
[8] A. Geiger, P. Lenz, C. Stiller, and R. Urtasun, “Vision meets robotics: the KITTI dataset,” The International Journal of Robotics Research, pp. 1231– 1237, 2013.
[9] H. Su, S. Maji, E. Kalogerakis, E. Learned-Miller,“Multi-View convolutional neural networks for 3D shape recognition,” IEEE International Conference on Computer Vision (ICCV), pp. 945-953, 2015.
[10] CR. Qi, H. Su, K. Mo, Leonidas J. Guibas, “PointNet: deep learning on point sets for 3D classification and segmentation,” IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
[11] CR. Qi, L. Yi, H. Su, LJ. Guibas, “Pointnet++: deep hierarchical feature learning on point sets in a metric space,” Advances in neural information processing systems 30 (NIPS 2017), 2017.
[12] D. Maturana and S. Scherer, “VoxNet: A 3D Convolutional neural network for real-time object recognition,” IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 922-928, 2015.
[13] MD. Deuge, A. Quadros, C. Hung, B. Douillard, “Unsupervised feature learning for classification of outdoor 3D scans,” In Australasian Conference on Robotics and Automation (ACRA), 2013.
[14] N. Varney, VK. Asari, Q. Graehling, “DALES: A large-scale aerial LiDAR data set for semantic segmentation,” IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
[15] Z. Wu, et al, “Cia-ssd: confident iou-aware single-stage object detector from point cloud,” AAAI conference on artificial intelligence, Vol 35, 2021.
[16] H. L. Ho, J. D. Chen, C. A. Yang, C. C. Liu, C. T. Lee, Y. H. Lai, and F. Y. Lin, “High-speed 3D imaging using a chaos lidar system,” The European Physical Journal Special Topics, pp. 435-441, 2022.
[17] P. Xiao et al., “PandaSet: advanced sensor suite dataset for autonomous driving,” IEEE International Intelligent Transportation Systems Conference (ITSC), pp. 3095-3101, 2021.
[18] Caesar, Holger, et al, “nuscenes: a multimodal dataset for autonomous driving,” IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
[19] Y. Yan, Y. Mao, and B. Li, “Second: sparsely embedded convolutional detection,” Sensors, 2018.
[20] Y. Zhou, O. Tuzel, “Voxelnet: end-to-end learning for point cloud based 3d object detection,” IEEE conference on computer vision and pattern recognition (CVPR), 2018.