臺灣博碩士論文加值系統

在現今的環境下，自動駕駛的技術已經逐漸成熟，並且當今自駕車上的感測器以不 局限於影像鏡頭，更多的是同時搭載了多種的感測器，如 3D 光達、雷達等，都是為了 能夠更好的保障駕駛及用路人的安全。而為了能夠更好的利用多種感測器進行物件偵測， 本研究將焦點放在了同時使用 3D 光達及影像鏡頭的物件偵測網路上，在參考了許多現 有的方法後，設計出了 FrustumNet-CSP 的網路架構。
FrustumNet-CSP 可以分為兩個部分，第一個部分使用 Mask R-CNN 產生 2D 的區域 提議，並且使用這些區域提議通過映射到三維的點雲圖中生成視錐體，通過這些視錐體 來將點雲數據分組並且輸入至 PointNet 中進行特徵提取，以產生特徵圖讓後續的網路能 夠進行訓練。第二部分為全卷積網路 (FCN,FullyConvolution Networks) ，全卷積網路將 接收由 PointNet 所產生的點雲特徵圖，通過多個卷積區塊以及 CSP 模塊來進行特徵的 訓練，並且使用批量歸一化以及 SiLU 激勵函數組合成 CBS 模塊，藉此來使網路能更快 的收斂，最後輸出物件偵測的結果。
實驗結果部分在現有的 KITTI 數據集上進行測試，KITTI 數據集為目前大多數三維 物件偵測網路所廣泛使用的數據集。在實驗上本論文與其他現有的同時 3D 光達與影像 鏡頭的方法進行了比較，比較了物件偵測的 3D AP 與鳥瞰圖 AP，本論文的網路皆取得 了最優秀的物件偵測準確度，藉此驗證了本論文之方法的有效性。

In the current context, autonomous driving technology has gradually matured, and modern self-driving cars are equipped not only with visual cameras but also with various other sensors such as 3D LiDAR, radar, etc., all aimed at ensuring better driving and pedestrian safety. To make better use of multiple sensors for object detection, this study focuses on a network architecture that combines 3D LiDAR and visual camera inputs. After referencing several existing methods, the research team designed the FrustumNet-CSP network architecture.
FrustumNet-CSP can be divided into two parts. The first part utilizes Mask R-CNN to generate 2D region proposals and maps them to 3D point cloud to create frustums. These frustums are used to group the point cloud data, which is then fed into PointNet for feature extraction to produce feature maps for subsequent network training. The second part consists of a Fully Convolutional Network (FCN) that takes the point cloud feature maps generated by PointNet. The FCN employs multiple convolutional blocks and CSP modules for feature training, along with batch normalization and SiLU activation functions combined into CBS modules to expedite network convergence. Finally, the FCN outputs the results of object detection.
The experimental results were conducted using the KITTI dataset, which is widely used for most 3D object detection networks. In these experiments, the proposed approach was compared with other existing methods that also utilize both 3D LiDAR and visual camera inputs. Object detection 3D AP and bird's eye view AP were compared, and the proposed FrustumNetCSP consistently achieved the highest accuracy in object detection, thus validating the effectiveness of this research approach.

摘要.............................................................................................................................................i
Abstract.......................................................................................................................................ii
誌謝...........................................................................................................................................iii
目錄...........................................................................................................................................iv
表格目錄....................................................................................................................................v
圖目錄.......................................................................................................................................vi
第一章 緒論........................................................................................................................1
1.1 研究背景................................................................................................................1
1.2 研究動機與目的....................................................................................................2
1.3 論文架構................................................................................................................3
第二章 相關研究................................................................................................................4
2.1 文獻回顧................................................................................................................4
2.2 卷積神經網路........................................................................................................6
2.3 區域提議網路........................................................................................................9
2.4 感測器融合網路..................................................................................................10
第三章 影像嵌入點雲的錐體網路..................................................................................13
3.1 整體網路架構......................................................................................................14
3.2 點雲數據取樣......................................................................................................14
3.3 點雲數據組合......................................................................................................17
3.4 全卷積網路..........................................................................................................19
第四章 實驗結果與分析..................................................................................................24
4.1 實驗設置與數據集..............................................................................................24
4.2 實驗結果與探討..................................................................................................26
4.3 消融實驗..............................................................................................................29
第五章 結論與未來展望..................................................................................................30
參考文獻..................................................................................................................................31
Extended Abstract.....................................................................................................................34
Chapter 1 Introduction..............................................................................................................35
Chapter 2 Related Work............................................................................................................35
Chapter 3 FrustumNet-CSP......................................................................................................36
Chapter 4 Experimental Results and Analysis..........................................................................36
Chapter 5 Conclusions..............................................................................................................37

[1] X. Chen, H. Ma, J. Wan, B. Li and T. Xia, "Multi-View 3D Object Detection Network
for Autonomous Driving," IEEE Conference on Computer Vision and Pattern
Recognition (CVPR), pp. 6526-6534, 2017.
[2] K. J. Zhixin Wang, "Frustum ConvNet: Sliding Frustums to Aggregate Local Point-Wise
Features for Amodal 3D Object Detection," IEEE/RSJ International Conference on
Intelligent Robots and Systems (IROS), pp. 1742-1749, 2019.
[3] Z. Y and T. O, "Voxelnet: End-to-end Learning for Point Cloud Based 3D Object
Detection," IEEE/CVF Conference on Computer Vision and Pattern Recognition
(CVPR), pp. 4490-4499, 2018.
[4] A. H. Lang, S. Vora, H. Caesar, L. Zhou, J. Yang and O. and Beijbom, "PointPillars:
Fast Encoders for Object Detection from Point Clouds," IEEE/CVF Conference on
Computer Vision and Pattern Recognition (CVPR), pp. 12689-12697, 2019.
[5] Y. Yan, Y. Mao and B. and Li, "SECOND: Sparsely Embedded Convolutional
Detection," Sensors, 2018.
[6] X. Z. T. H. R. H. Y. Z. X. B. Zhe Liu, "TANet: Robust 3D Object Detection from Point
Clouds with Triple Attention," AAAI Conference on Artificial Intelligence, pp. 11677-
11684, 2019.
[7] R. R. Weijing Shi, "Point-GNN: Graph Neural Network for 3D Object Detection in a
Point Cloud," IEEE/CVF Conference on Computer Vision and Pattern Recognition
(CVPR), pp. 1708-1716, 2020.
[8] X. Y. X. T. J. F. Z. X. E. D. S. W. Liang Du, “Associate-3Ddet: Perceptual-to-Conceptual Association for 3D Point Cloud Object Detection,” IEEE/CVF Conference
on Computer Vision and Pattern Recognition (CVPR), pp. 13326-13335, 2020.
[9] W. T. S. C. L. J. C.-W. F. Wu Zheng, “CIA-SSD: Confident IoU-Aware Single-Stage
Object Detector From Point Cloud,” Proceedings of the AAAI Conference on Artificial
Intelligence, pp. 3555-3562, 2020.
[10] X. W. H. L. Shaoshuai Shi, “PointRCNN: 3D Object Proposal Generation and
Detection from Point Cloud,” IEEE/CVF Conference on Computer Vision and Pattern
Recognition (CVPR), pp. 770-779, 2019.
[11] B. Jiang, R. Luo, J. Mao, T. Xiao and Y. and Jiang, "Acquisition of Localization
Confidence for Accurate Object Detection," Proceedings of the European Conference
(ECCV), pp. 784-799, 2018.
[12] Y. S. S. L. X. S. J. J. Zetong Yang, “STD: Sparse-to-Dense 3D Object Detector for
Point Cloud,” IEEE/CVF International Conference on Computer Vision (ICCV), pp.
1951-1960, 2019.
[13] C. G. L. J. Z. W. J. S. X. W. H. L. Shaoshuai Shi, "PV-RCNN: Point-Voxel Feature Set
Abstraction for 3D Object Detection," IEEE/CVF Conference on Computer Vision and
Pattern Recognition (CVPR), pp. 10526-10535, 2020.
[14] C. He, H. Zeng, J. Huang, X.-S. Hua and L. Zhang, "Structure Aware Single-stage 3D
Object Detection from Point Cloud," IEEE/CVF Conference on Computer Vision and
Pattern Recognition (CVPR), pp. 11870-11879, 2020.
[15] Y. S. S. L. J. J. Zetong Yang, "3DSSD: Point-based 3D Single Stage Object Detector,"
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.
11037-11045, 2020.
[16] Y. K. J. K. J. W. C. Jin Hyeok Yoo, "3D-CVF: Generating Joint Camera and LiDAR
Features Using Cross-View Spatial Feature Fusion for 3D Object Detection," Proc. 16th
Eur. Conf. Comput. Vis. (ECCV), p. 720–736, 2020.
[17] M. M. J. L. A. H. S. W. Jason Ku, “Joint 3D Proposal Generation and Object
Detection from View Aggregation,” IEEE/RSJ International Conference on Intelligent
Robots and Systems (IROS), pp. 1-8, 2018.
[18] D. M. H. R. Su Pang, "CLOCs: Camera-LiDAR Object Candidates Fusion for 3D
Object Detection," IEEE/RSJ International Conference on Intelligent Robots and
Systems (IROS), pp. 10386-10393, 2020.
[19] K. H. R. G. J. S. Shaoqing Ren, "Faster R-CNN: Towards Real-Time Object Detection
with Region Proposal Networks," IEEE Transactions on Pattern Analysis and Machine
Intelligence, pp. 1137-1149, 2017.
[20] R. Girshick, "Fast R-CNN," IEEE International Conference on Computer Vision
(ICCV), pp. 1440-1448, 2015.
[21] G. G. P. D. R. G. Kaiming He, “Mask R-CNN,” IEEE International Conference on
Computer Vision (ICCV), pp. 2980-2988, 2018.
[22] H. S. K. M. L. J. G. Charles R. Qi, “PointNet: Deep Learning on Point Sets for 3D
Classification and Segmentation,” Proc. IEEE Comput.Vis. Pattern Recognit. (CVPR),
pp. 77-85, 2017.
[23] W. L. C. W. H. S. L. J. G. Charles R. Qi, “Frustum PointNets for 3D Object Detection
from RGB-D Data,” Proceedings of the IEEE Conferenceon on Computer Vision and
Pattern Recognition (CVPR), pp. 918-927, 2018.
[24] P. L. C. S. a. R. U. Andreas Geiger, “Vision meets robotics: The KITTI dataset,”
International Journal of Robotics Research (IJJR), pp. 1231-1237, 2013.
[25] Z. W. J. S. X. W. H. L. Shaoshuai Shi, “From Points to Parts: 3D Object Detection
from Point Cloud with Part-aware and Part-aggregation Network,” IEEE Transactions
on Pattern Analysis and Machine Intelligence, pp. 2647-2664, 2021.