臺灣博碩士論文加值系統

時代更迭機器人技術逐漸進步，各大企業行業中機器人技術不斷引入新技術，而近幾年VR設備與相關產品漸漸普及在了眾人眼前，隨著Apple Vision Pro的問世更是掀起了一波風潮，現今VR遠程操控應用在各個方面，能夠實現多種不同的功能和任務，使用VR操作機械手臂的方式打造更容易操控，與更加安全的操控方法，例如遠端手術、機器人操作、遠端教育和培訓、危險環境勘測。而深度攝影機在各種不同的任務中扮演由為重要角色，能夠更準確地的偵測物件距離，相當於人的的一雙眼睛，本論文將使用深度攝影機結合VR虛擬實境打造一處場景，透過深度攝影機使虛擬實境內的虛擬物件與現實物件同步。
本論文著重於在虛擬現實模擬中整合深度攝影機與YOLOv5物件檢測，以實現虛擬與現實物件在模擬環境中的同步。深度攝影機能夠捕捉場景中物件的精確深度信息，其應用使得虛擬物件在模擬場景中的準確，提升使用者的沉浸式體驗。本論文探討實現此同步的技術層面和方法，在影像辨識方面將使用YOLOv5物件檢測模型進行物件偵測，透過深度影像技術的整合獲取物件在現實中的位置。以提供在虛擬場景中虛擬物件所需的位子資料。
本論文包含架設了機械手臂控制端的伺服器並且在虛擬世界中建立操作機械手臂之模擬工作站，讓使用者透過VR裝置所進入的虛擬世界之機械手臂模擬工作站能及時演示現實中機械手臂的狀態以及其相對應的動作流程，以提供使用者一個安全空間進行觀察與監測，這種同步技術的成功應用不僅有望擴展虛擬現實的應用範疇，還有助於各領域中模擬和訓練的進一步發展。

As robotics technology advances with the changing times, various industries continually introduce new technologies into robotics. In recent years, VR devices and related products have gradually become popular, with the release of Apple Vision Pro igniting a trend. VR remote control applications now span various domains, offering diverse functionalities and capabilities. VR operation of mechanical arms facilitates easier and safer control methods, enabling applications such as remote surgery, robot operation, remote education and training, and hazardous environment surveying. Depth cameras play a crucial role in various tasks, accurately detecting object distances akin to human eyes. This thesis aims to synchronize virtual and real objects through the integration of depth image cameras with VR virtual environments.
This paper focuses on integrating depth image cameras and the YOLOv5 object detection model in virtual reality simulations to achieve synchronization between virtual and real objects in the simulation environment. Depth cameras capture precise depth information of objects in the scene, enhancing the accuracy of virtual objects in the simulation environment and improving users' immersive experiences. This thesis explores the technological aspects and methods to achieve this synchronization. Object detection using the YOLOv5 model is employed for object detection, while integration of depth image technology retrieves object positions in reality to provide the necessary positional data for virtual objects in the virtual scene.
The thesis includes setting up a server for controlling the mechanical arm and establishing a simulated workstation for operating the mechanical arm in the virtual world. Users can enter the virtual world through VR devices to observe and monitor the real-time demonstration of the mechanical arm's status and corresponding action processes, providing users with a safe space for observation and monitoring. The successful application of this synchronization technology not only expands the scope of virtual reality applications but also contributes to the further development of simulation and training in various fields.

摘要...................................i
Abstract...................................ii
誌謝...................................iii
目錄...................................iv
表目錄...................................vii
圖目錄...................................viii
第一章 緒論...................................1
1.1 研究背景與動機...................................1
1.2 研究範圍與限制...................................2
1.3 文獻探討...................................3
1.3.1 虛擬實境(Virtual Reality)...................................3
1.3.2 YOLO算法的演進與應用...................................6
1.3.3 機械手臂於虛擬實境的應用...................................8
第二章 系統架構...................................11
2.1 硬體系統架構...................................11
2.2 硬體系統之介紹...................................12
2.2.1 TM5-900機械手臂...................................12
2.2.2 頭戴式VR裝置...................................14
2.2.3 深度攝影機...................................15
2.3 軟體系統介紹...................................16
2.3.1 虛擬實境引擎介紹與Unity Engine...................................16
2.3.2 Oculus應用程式...................................17
2.3.3 Unity插件應用...................................18
2.3.4 Socket網路介面通訊技術...................................22
2.3.5 物體檢測模型YOLOv5...................................23
2.3.6 3D CAD 軟體 Onshape...................................24
第三章 研究方法...................................25
3.1 機械手臂工作平台...................................25
3.2 機械手臂電控箱及專案設置...................................26
3.3 機械手臂控制系統...................................27
3.3.1 機械手臂控制模組...................................28
3.3.2 夾爪控制模組...................................29
3.3.3 Socket連線模組...................................30
3.4 Unity虛擬實境環境設置...................................31
3.5 Unity場景架設...................................32
3.5.1 XR Interaction Toolkit...................................32
3.5.2 機械手臂與物件導入...................................34
3.6 YOLOv5模型訓練...................................36
3.6.1 混淆矩陣（Confusion Matrix）...................................38
3.6.2 精確率-召回率曲線（Precision-Recall curve）.....................38
3.6.3 置信度曲線 (F1-Confidence Curve)...................................39
3.6.4 精確率-置信度曲線 (Precision-Confidence Curve).....................40
3.6.5 召回率-置信度曲線(Recall-Confidence Curve).......................40
3.6.6 YOLOv5模型訓練與驗證結果...................................41
3.7 3D物件偵測...................................43
3.8 現實與虛擬同步...................................44
3.8.1 機械手臂同步...................................45
3.8.2 虛擬物件同步...................................47
第四章 實驗成果與測試...................................52
4.1 裝置連線...................................53
4.2 任務布置之組裝椅子...................................54
第五章 結論與未來展望...................................58
5.1 結論...................................58
5.2 未來展望...................................58
參考文獻...................................59
Extended Abstract...................................61

[1] 0x資訊, “記住 VRML：1995 年的元宇宙,” [線上]. Available: https://0xzx.com/zh-tw/2022011220182002200.html. [存取日期: 1 5 2024].
[2] K. SCHWARZE, “It's Like Teleporting" — Facebook Buys Virtual-Reality Maker Oculus,” [線上]. Available: https://www.popsugar.com/tech/What-Oculus-Rift-34441871. [存取日期: 1 5 2024].
[3] M. Andronico, “I just tried the Meta Quest 3 — and it’s the VR headset I’ve always wanted,” [線上]. Available: https://edition.cnn.com/cnn-underscored/electronics/meta-quest-3-hands-on. [存取日期: 1 5 2024].
[4] N. I. Lab., “Virtual Reality,” [線上]. Available: http://ip.csie.ncu.edu.tw/course/VR/VR1100p.pdf. [存取日期: 1 5 2024].
[5] D.Heaney., “How virtual reality positional tracking works,” [線上]. Available: https://venturebeat.com/games/how-virtual-reality-positional-tracking-works/. [存取日期: 1 5 2024].
[6] VRPinea., “5年過去了，VR現在是Inside-out的天下了嗎？,” [線上]. Available: https://www.gushiciku.cn/dl/02Ld2/zh-tw. [存取日期: 1 5 2024].
[7] RizwanMunawar、glenn-jocher、AyushExel, “ultralytics,” [線上]. Available: https://docs.ultralytics.com/zh#yolo-a-brief-history. [存取日期: 1 5 2024].
[8] 張家銘, “YOLOv4 產業應用,” [線上]. Available: https://aiacademy.tw/yolo-v4-intro/. [存取日期: 1 5 2024].
[9] 工業技術研究院.鄒明珆., “工業技術與資訊月刊_建立元宇宙智慧製造生態系,” [線上]. Available: https://www.itri.org.tw/ListStyle.aspx?DisplayStyle=18_content&SiteID=1&MmmID=1036452026061075714&MGID=1164137104732107451. [存取日期: 1 5 2024].
[10] 世紀貿易, “元宇宙工廠,” [線上]. Available: https://www.centra.com.tw/sb01?id=8. [存取日期: 1 5 2024].
[11] 高永洲, “未來科技館,” [線上]. Available: https://www.futuretech.org.tw/futuretech/index.php?action=product_detail&prod_no=P0008700003748. [存取日期: 1 5 2024].
[12] T. robot., “TM robot 常規負載機器人系列,” [線上]. Available: https://www.tm-robot.com/zh-hant/regular-payload/. [存取日期: 1 5 2024].
[13] M.QUEST., “META QUEST3,” [線上]. Available: https://www.meta.com/tw/quest/quest-3/. [存取日期: 1 5 2024].
[14] StereoLabs., “Products ZED 2i,” [線上]. Available: https://www.stereolabs.com/products/zed-2. [存取日期: 1 5 2024].
[15] UNITY, “Unity,” [線上]. Available: https://unity.com/cn. [存取日期: 1 5 2024].
[16] B. Lang, “Latest Updates Enable Quest 2 to Run Oculus Link at 90Hz – Here’s How,” [線上]. Available: https://www.roadtovr.com/oculus-quest-2-90hz-oculus-link/. [存取日期: 1 5 2024].
[17] 薛. 邱建宏, “探索元宇宙開放框架－OpenXR標準,” [線上]. Available: https://ictjournal.itri.org.tw/xcdoc/cont?xsmsid=0M236556470056558161&sid=0M264424321560844954. [存取日期: 1 5 2024].
[18] Andrew, “XR Toolkit 2.0 Crash Course using Unity 2021,” [線上]. Available: https://www.youtube.com/watch?app=desktop&v=5ZBkEYUyBWQ. [存取日期: 1 5 2024].
[19] Hecomi, “uWindowCapture,” [線上]. Available: https://github.com/hecomi/uWindowCapture. [存取日期: 1 5 2024].
[20] M. Robotics, “Robotic Arm with Inverse Kinematics,” [線上]. Available: https://assetstore.unity.com/packages/templates/packs/robotic-arm-with-inverse-kinematics-161155. [存取日期: 1 5 2024].
[21] Jeanboydev., “一篇文章看明白 TCP/IP，TCP，UDP，IP，Socket 之间的关系,” [線上]. Available: https://blog.csdn.net/freekiteyu/article/details/72236734. [存取日期: 1 5 2024].
[22] D. Davies, “Windows上的YOLOv5對象檢測（分步教程）,” [線上]. Available: https://wandb.ai/wandb_fc/chinese/reports/Windows-YOLOv5---VmlldzoxODQ3NzQ1. [存取日期: 1 5 2024].
[23] ONSHAPE, “Onshape,” [線上]. Available: https://www.onshape.com/en/. [存取日期: 1 5 2024].
[24] peifeng-unity, “Unity Robotics Demos,” [線上]. Available: https://github.com/Unity-Technologies/articulations-robot-demo. [存取日期: 1 5 2024].