臺灣博碩士論文加值系統

現有的3D點雲物件偵測在考慮到運算量與速度的前提下多僅使用點雲數據來進行訓練，而在訓練點雲物件時通常將物件偵測(Object Detection)和物件辨識視為兩個獨立的任務，因此在定位準確度和分類器的置信度兩者數據可能會無法匹配。為了解決這個問題，本研究將焦點放在單級偵測器(One Stage)上，參考了前面已經有的一些方法之後，本研究設計出了SS-SSD (Semantic Segmentation-Based Single-Stage 3D Object Detection)的模型架構。
SS-SSD大致分為三個部分，第一個部分使用稀疏卷積(Sparse Convolution)來處理輸入的點雲點，並藉以將其轉換成更好進行資料處理體素(Voxel)並進行訓練，第二部分為區域提取網路(RPN,Region Proposal Networks)，重新編排過後的體素網路架構可以很快地找到三維數據的特徵空間，第三部分為語義分割模型，本研究的語義分割模型直接輸出語義分割後的淺層體素網路，再把處理過後的語義分割模型以及RPN網路輸出的模型進行資料合併，最後用DIoU(Distance Intersection over Union)來收斂損失函數(Loss Function)，並得以進行物件偵測以及物件辨識。

Existing 3D point cloud object detection mostly only uses point cloud data for training under the premise of considering the amount of calculation and speed, while object detection and object recognition are usually regarded as two when training point cloud objects. It is a separate task, so the data may not match between the localization accuracy and the confidence of the classifier. In order to solve this problem, this study focuses on the single-stage detector. After referring to some of the previous methods, this study designs SS-SSD (Semantic Segmentation-Based Single-Stage 3D Object Detection) model architecture.
SS-SSD is roughly divided into three parts. The first part uses sparse convolution to process the input point cloud points and convert them into Voxels for better data processing and training. The second part is the Region Proposal Networks. The rearranged voxel network architecture can quickly find the feature space of the three-dimensional data. The third part is the semantic segmentation model, the semantic segmentation model of this study. Directly output the shallow voxel network after semantic segmentation, then merge the processed semantic segmentation model and the model output by the RPN network, and finally use DIoU (Distance Intersection over Union) to converge the loss function, And can perform object detection and object identification.

摘要...................................i
Abstract..............................ii
誌謝..................................iii
目錄...................................iv
表格目錄...............................vi
圖目錄.................................vii
第一章 緒論..............................1
1.1 研究背景............................1
1.2 研究動機與目的.......................2
1.3 論文架構............................3
第二章 相關研究..........................4
2.1 文獻回顧.............................4
2.2 深度學習.............................7
2.3 類神經網路...........................8
2.4 卷積神經網路.........................13
第三章 語義分割之單級三維物件偵測..........18
3.1 點雲體素資料.........................19
3.2 稀疏卷積.............................21
3.3 語義分割和區域提取網路的融合...........25
3.4 快速收斂邊界框........................26
第四章 實驗結果與分析......................30
4.1 實驗設置..............................30
4.2 資料庫統整............................31
4.3 實驗數據與情形探討.....................34
4.4 實驗結果與探討.........................38
第五章結論與未來展望........................41
參考文獻...................................43
Extended Abstract.........................46
Chapter 1 Introduction.....................47
Chapter 2 Related work.....................47
Chapter 3 SS-SSD block.....................48
Chapter 4 Experimental Results and Analysis....48
Chapter 5 Conclusions......................49

[1]Chen, X.; Ma, H.; Wan, J.; Li, B.; and Xia, T. 2017. MultiView 3D Object Detection Network for Autonomous Driving. In CVPR, 1907–1915.
[2]Chen, Y.; Liu, S.; Shen, X.; and Jia, J. 2019. Fast Point RCNN. In ICCV, 9775–9784.
[3]D., W. D.; W., M. I.; and L., S. R. 2008. Multivariate Probability Distributions. In Mathematical Statistics with Applications, 223–295. Belmont, California: Brooks/Cole, 7 edition.
[4]Prarthana Bhattacharyya and Krzysztof Czarnecki. Deformable PV-RCNN: Improving 3D object detection with learned deformations.
[5]Graham, B.; Engelcke, M.; and Van Der Maaten, L. 2018. 3D Semantic Segmentation with Submanifold Sparse Convolutional Networks. In CVPR, 9224–9232.
[6]Shi, W.; and Rajkumar, R. 2020. Point-GNN: Graph Neural Network for 3D Object Detection in a Point Cloud. In CVPR, 1711–1719.
[7]Liu, Z.; Zhao, X.; Huang, T.; Hu, R.; Zhou, Y.; and Bai, X. 2020. TANet: Robust 3D Object Detection from Point Clouds with Triple Attention. In AAAI, 11677–11684.
[8]Chenhang He, Hui Zeng, Jianqiang Huang, Xian-Sheng Hua, and Lei Zhang. Structure aware single-stage 3D object detection from point cloud. In CVPR, pages
[9]Du, L.; Ye, X.; Tan, X.; Feng, J.; Xu, Z.; Ding, E.; and Wen, S. 2020. Associate-3Ddet: Perceptual-to-Conceptual Association for 3D Point Cloud Object Detection. In CVPR, 13329–13338
[10]Liu, W.; Anguelov, D.; Erhan, D.; Szegedy, C.; Reed, S.; Fu, C.-Y.; and Berg, A. C. 2016. SSD: Single Shot Multibox Detector. In ECCV, 21–37.
[11]Kingma, D. P.; and Ba, J. 2014. Adam: A Method for Stochastic Optimization. arXiv preprint arXiv:1412.6980 .
[12]Yoo, J. H.; Kim, Y.; Kim, J. S.; and Choi, J. W. 2020. 3DCVF: Generating Joint Camera and LiDAR Features Using Cross-View Spatial Feature Fusion for 3D Object Detection. In ECCV, 720–736.
[13]Zhou, D.; Fang, J.; Song, X.; Guan, C.; Yin, J.; Dai, Y.; and Yang, R. 2019. IoU Loss for 2D/3D Object Detection. In 3DV, 85–94.
[14]Ming Liang, Bin Yang, Shenlong Wang, and Raquel Urtasun. Deep continuous fusion for multi-sensor 3D object detection. In ECCV, 2018.
[15]Shi, S.; Wang, X.; and Li, H. 2019. PointRCNN: 3D Object Proposal Generation and Detection from Point Cloud. In CVPR, 770–779.
[16]Charles R. Qi, Wei Liu, Chenxia Wu, Hao Su, and Leonidas J. Guibas. Frustum pointnets for 3D object detection from RGB-D data. CoRR, 2017
[17]Jiang, B.; Luo, R.; Mao, J.; Xiao, T.; and Jiang, Y. 2018. Acquisition of Localization Confidence for Accurate Object Detection. In ECCV, 784–799.
[18]Yang, Z.; Sun, Y.; Liu, S.; Shen, X.; and Jia, J. 2019. STD: Sparse-to-Dense 3D Object Detector for Point Cloud. In ICCV, 1951–1960.
[19]Shaoshuai Shi, Chaoxu Guo, Li Jiang, Zhe Wang, Jianping Shi, Xiaogang Wang, and Hongsheng Li. PV-RCNN: Pointvoxel feature set abstraction for 3D object detection. In CVPR, pages 10529–10538, 2020
[20]Jin Hyeok Yoo, Yeocheol Kim, Ji Song Kim, and Jun Won Choi. 3D-CVF: Generating joint camera and LiDAR fea9 tures using cross-view spatial feature fusion for 3D object detection. In ECCV, 2020.
[21]Su Pang, Daniel Morris, and Hayder Radha. CLOCs: Camera-LiDAR object candidates fusion for 3D object detection. arXiv preprint arXiv:2009.00784, 2020
[22]Zhou, Y.; and Tuzel, O. 2018. Voxelnet: End-to-end Learning for Point Cloud Based 3D Object Detection. In CVPR, 4490–4499
[23]Lang, A. H.; Vora, S.; Caesar, H.; Zhou, L.; Yang, J.; and Beijbom, O. 2019. PointPillars: Fast Encoders for Object Detection from Point Clouds. In CVPR, 12697–12705.
[24]Yan, Y.; Mao, Y.; and Li, B. 2018. SECOND: Sparsely Embedded Convolutional Detection. Sensors 18(10): 3337.
[25]Weijing Shi and Raj Rajkumar. Point-GNN: Graph neural network for 3D object detection in a point cloud. In CVPR, pages 1711–1719, 2020.
[26]Yang, Z.; Sun, Y.; Liu, S.; and Jia, J. 2020. 3DSSD: Pointbased 3D Single Stage Object Detector. In CVPR, 11040– 11048
[27]Wu Zheng, Weiliang Tang, Sijin Chen, Li Jiang, and ChiWing Fu. CIA-SSD: Confident IoU-aware single-stage object detector from point cloud. In AAAI, 2021
[28]Geiger, A.; Lenz, P.; Stiller, C.; and Urtasun, R. 2013. Vision Meets Robotics: The KITTI Dataset. The International Journal of Robotics Research 32(11): 1231–1237.
[29]Ku, J.; Mozifian, M.; Lee, J.; Harakeh, A.; and Waslander, S. L. 2018. Joint 3D Proposal Generation and Object Detection from View Aggregation. In IROS, 1–8.
[30]Lehner, J.; Mitterecker, A.; Adler, T.; Hofmarcher, M.; Nessler, B.; and Hochreiter, S. 2019. Patch Refinement–Localized 3D Object Detection. arXiv preprint arXiv:1910.04093 .
[31]Li, J.; Luo, S.; Zhu, Z.; Dai, H.; Krylov, A. S.; Ding, Y.; and Shao, L. 2020. 3D IoU-Net: IoU Guided 3D Object Detector for Point Clouds. arXiv preprint arXiv:2004.04962 .
[32]Liang, M.; Yang, B.; Chen, Y.; Hu, R.; and Urtasun, R. 2019. Multi-task Multi-sensor Fusion for 3D Object Detection. In CVPR, 7345–7353.
[33]Liang, M.; Yang, B.; Wang, S.; and Urtasun, R. 2018. Deep Continuous Fusion for Multi-sensor 3D Object Detection. In ECCV, 641–656.
[34]Lin, T.-Y.; Goyal, P.; Girshick, R.; He, K.; and Dollar, P. ´ 2017. Focal Loss for Dense Object Detection. In ICCV, 2980–2988.
[35]Liu, B.; Wang, M.; Foroosh, H.; Tappen, M.; and Pensky, M. 2015. Sparse Convolutional Neural Networks. In CVPR, 806–814
[36]Loshchilov, I.; and Hutter, F. 2016. SGDR: Stochastic Gradient Descent with Warm Restarts. arXiv preprint arXiv:1608.03983 .
[37]Qi, C. R.; Liu, W.; Wu, C.; Su, H.; and Guibas, L. J. 2018. Frustum PointNets for 3D Object Detection from RGB-D Data. In CVPR, 918–927.
[38]Qi, C. R.; Yi, L.; Su, H.; and Guibas, L. J. 2017. PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space. In NeurIPS, 5099–5108.
[39]Shi, S.; Guo, C.; Jiang, L.; Wang, Z.; Shi, J.; Wang, X.; and Li, H. 2020a. PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection. In CVPR, 10529–10538.
[40]Shi, S.; Wang, Z.; Shi, J.; Wang, X.; and Li, H. 2020b. From Points to Parts: 3D Object Detection from Point Cloud with Part-Aware and Part-Aggregation Network. IEEE Transactions on Pattern Analysis and Machine Intelligence .
[41]Wang, Z.; and Jia, K. 2019. Frustum ConvNet: Sliding Frustums to Aggregate Local Point-Wise Features for Amodal 3D Object Detection. In IROS, 1742–1749.
[42]Xie, L.; Xiang, C.; Yu, Z.; Xu, G.; Yang, Z.; Cai, D.; and He, X. 2020. PI-RCNN: An Efficient Multi-Sensor 3D Object Detector with Point-Based Attentive Cont-Conv Fusion Module. In AAAI, 12460–12467.