臺灣博碩士論文加值系統

每逢水果的採收期，果園需要大量人手來協助水果採集及外觀進行篩選。不同的外觀品質的用途各不相同，但採集人力之間不一定有相同標準，如此水果的品質便會參差不齊。因此本論文利用深度學習來判斷水果的外觀，基於完全卷積網路(FCN, Fully Convolutional Networks)的架構方法，FCN是用於語意分割的像素級預測框架。原始FCN模型較難辨識出水果表面缺陷位置，因此提出利用二次訓練的方法來改善模型，第一次先訓練背景及水果，第二次僅針對水果大小資料訓練水果及缺陷，提高辨識出水果外觀缺陷位置的機率，相較傳統電腦視覺辨識所要求的嚴苛取像條件，深度學習的方法較容易突破環境限制，最後將影像推論分割出水果和缺陷來表示檢測結果，計算水果總面積及缺陷面積比例設定閾值判斷分級，讓分級有統一標準。本改良方法可搭配使用擴增實境(AR, Augmented Reality)眼鏡做為研究方法之應用裝置，讓AR眼鏡拍攝水果影像資料後，傳輸至嵌入式系統進行自動檢測，回傳圖像分割結果，不論是果樹上或已採摘的水果都能進行自動檢測判斷應用。

Once fruits are harvested, orchard needs a lot of manpower to perform fruit collection and screen regarding the appearance of the fruits. The uses of appearance qualities are different, but there is no objective standard for manual collecting, so the quality of the fruits will be uneven. Therefore, this thesis uses deep learning to judge the appearance of fruits, based on the deep learning Fully Convolutional Networks, FCN is an approach with a typical pixel-based semantic segmentation method, but it is difficult to identify the defect location under the original model. Therefore, a modified two-pass training method is proposed to improve the model. The first pass training focuses on background and fruit segmentation; the second pass training tries to distinguish the fruit and the defects on the detected fruit instances, aiming at the improvement of the probability of identifying the defect location of the fruit appearance. Compared with traditional computer vision recognition, which requires a strict background condition, the deep learning method is easier to break through the obstacle of a constrained environment. Finally, image inference is used to classify the fruits and the defects for calculating the total area of fruits and the proportion of defective areas which are used to set a threshold for judgment and grading, so that the grading has a uniform standard. Combined with augmented reality glasses for automated fruit defects detection, it is a simple and wearable device for human working for fruits harvest. After receiving the fruit images on the lens of the AR glasses, it is transmitted to a back-end system for detection calculation and returns the segmentation result, whether it is on the fruit tree or the picked fruit, both can be judged and applied using the proposed approach.

Abstract......i
摘要......iii
Acknowledgements......iv
Table of Contents......v
List of Tables......vii
List of Figures......viii
Chapter 1 Introduction......1
1.1 Background......1
1.2 Motivation and objective......1
1.3 Organization of the thesis......3
Chapter 2 Literature Review......4
2.1 Overview......4
2.2 Fruit grading......4
2.3 Problem formulation......6
2.4 Defect detection......6
2.4.1 Removal of background......6
2.4.2 Segmentation algorithm......7
2.4.3 Convolution Neural Network......8
Chapter 3 Computer Vision Methods......10
3.1 Background segmentation......10
3.2 Sliding comparison window......13
3.3 Detection of surface defect......14
Chapter 4 Deep Learning Methods......17
4.1 Fully Convolutional Networks......17
4.2 Data pre-processing......18
4.3 Networks architecture......21
4.3.1 FCN-32s......22
4.3.2 FCN-16s......22
4.3.3 FCN-8s......23
4.4 VGG16 model......23
Chapter 5 FCN two-pass learning......26
5.1 Dataset classification......26
5.2 First pass training......27
5.3 Second pass training......28
Chapter 6 Experiment Results......31
6.1 Environment settings......31
6.2 Computer vision segmentation results......31
6.3 FCN training results......35
6.4 Compare the two methods......46
6.5 Experimental application......46
Chapter 7 Conclusion and Future Research......48
Reference......49
Extended Abstract......51

[1]Dian Rong, XiuqinRao, Yibin Ying, “Computer vision detection of surface defect on oranges by means of a sliding comparison window local segmentation algorithm,” Computers and Electronics in Agriculture Vol. 137, pp. 59-68, May 2017.
[2]勞動統計查詢網. 勞動部. [Online] Available: https://statfy.mol.gov.tw/statistic_DB.aspx. Accessed July 12, 2020
[3]重量式分級機. 合興農業機有限公司. [Online] Available: https://www.hohsing-yung.com.tw/product-detail-299368.html. Accessed July 12, 2020
[4]Evan Shelhamer, Jonathan Long, and Trevor Darrell, “Fully Convolutional Networks for Semantic Segmentation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 39, No. 4, Apr. 2017.
[5]Dian Rong, Yibin Ying, Xiuqin Rao, “Embedded vision detection of defective orange by fast adaptive lightness correction algorithm,” Computers and Electronics in Agriculture Vol. 138, pp. 48-59, Jun. 2017.
[6]Maimunah Mohd Alia, Norhashila Hashima, Ahmad Shahid Abdul Hamida, “Combination of laser-light backscattering imaging and computer vision for rapid determination of oil palm fresh fruit bunches maturity,” Computers and Electronics in Agriculture, Vol. 169, 105235, Feb. 2020.
[7]Shuxiang Fand, Jiangbo Li, Yunhe Zhang, Xi Tian, Qingyan Wang, Xin He, Chi Zhang, Wenqian Huang, “On line detection of defective apples using computer vision system combined with deep learning methods,” Journal of Food Engineering, Vol. 286, 110102, Dec. 2020.
[8]Arthur Z.da Costa, Hugo E.H.Figueroa, Juliana A.Fracarolli “Computer vision based detection of external defects on tomatoes using deep learning,” Biosystems Engineering, Vol. 190, pp 131-144, Feb. 2020.
[9]ImageNet. official web site. [Online] Available: http://image-net.org/index. Accessed May 9, 2018
[10]Alex Krizhevsky, Ilya Sutskever, Geoffrey E Hinton, “Imagenet classification with deep convolutional neural networks,” In Advances in neural information processing systems, pp 1097-1105, 2014
[11]Karen Simonyan, Andrew Zisserman, “Very deep convolutional networks for large-scale image recognition,” Computer Vision and Pattern Recognition, Sep. 2014.
[12]Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, Andrew Rabinovich “Going deeper with convolutions,” Computer Vision and Pattern Recognition, Sep. 2014.
[13]Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun “Deep Residual Learning for Image Recognition,” Computer Vision and Pattern Recognition, Dec. 2015.
[14]OpenCV. official web site. [Online] Available: https://opencv.org/ . Accessed Jul 8, 2020
[15]John Canny “A Computational Approach to Edge Detection” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. PAMI-8, Issue. 6, pp. 679-698, Nov.1986.
[16]Labelme. Wkentaro. [Online] Available: https://github.com/wkentaro/labelme. Accessed Oct. 14, 2019
[17]vgg16 models. François Chollet. [Online] Available: https://github.com/fchollet/deep-learning-models/releases/download/v0.1/vgg16_weights_tf_dim_ordering_tf_kernels_notop.h5. Accessed Mar. 9, 2020
[18]TensorFlow. official web site. [Online] Available: https://www.tensorflow.org/?hl=zh-tw . Accessed Mar. 14, 2020
[19]Moverio BT-350. EPSON. [Online] Available: https://www.epson.com.tw/家用系列/智慧穿戴裝置/智慧眼鏡/Moverio-BT-350/p/V11H837054. Accessed Jun. 20, 2020