臺灣博碩士論文加值系統

隨著科技的不斷進步，以一般只能觀賞的虛擬實境越來越難以提供使用者更好的虛實體驗。為了提升使用者體驗虛擬實境時的互動性，虛擬實境結合了多種不同的技術，其目的不外乎是為了提供更好的虛實體驗，讓使用者能在虛擬世界中以雙手進行控制、雙腳進行移動。然而，常見的虛擬實境設備大多都以控制手把面向、觸發控制手把上的按鈕來進行操作，造成使用者無法以本身的身體部件來進行直觀的操作。

本研究希望能讓使用者長距離移動的情況下最大化提升虛擬實境的互動性，因此本研究選擇以手機作為開發平台。基於 Google ARCore 函式庫，我們可以藉由九軸感測器提供的加速度、磁感測、陀螺儀數值，並使用慣性測量單元(Inertial Measurement Unit，IMU)及 Madgwick 的姿態參考系統(Attitude and Heading Reference System，AHRS)之演算法進行身體位置的動態捕捉，讓手機能精準的感知使用者身體各節點的三維移動量。

為了加強虛擬與現實空間中環境的關聯性，本研究以擴增實境中的環境感知技術結合人體的姿態捕捉技術讓使用者可以用手腳與虛擬物件互動並達到虛擬整合的效果，並且實作了自定義感測器敏感度與其校準系統，讓不同身型的使用者只用手機與本研究中的感測系統就可以在任意環境中，體驗到可用四肢與虛擬物件互動的虛擬實境體驗。

With the progress of the technology, the Virtual Reality (VR) that only provides viewing becomes more and more difficultly to give users better VR experience. To enhance VR interaction user experiencing, VR combines with various technologies so as to improve better VR experience such as users can control and move by their hand and feet. However, most of the VR equipment only operate by controller’s direction and the triggers on the controller button, it cause user cannot use their own body to have an intuitive control.

This study focuses on maximizing the interactivity of VR of long-range moving, thus cellphone is chosen as our development platform. Based on Google ARCore library, the values of accelerometer, magnetism sensor and gyroscope from 9-axis sensor can be got. Furthermore, the Inertial Measurement Unit (IMU) and Madgwick’s Attitude and Heading Reference System algorithm are used to calculate information and motion tracking of body. Therefore, the
cellphone can detect the body node on the user’s three-dimensional movement accurately.

To reinforce the relationship between virtual and real environment, this study combines environment understanding and body motion tracking technology to let user can interact with the virtual object by their hands and feet. Moreover, this study implements a customize sensitivity system of sensors and calibration system to fit the body shape of everyone. Therefore, the proposed system provides users have the VR experience that can interact with virtual objects by their limbs in any environment.

摘要......i
Abstract......ii
誌謝......iii
目錄......iv
表目錄......vi
圖目錄......vii
第一章 緒論......1
1.1 研究動機......1
1.2 研究回顧......2
1.2.1 無線體域網(WBSNs)與動作姿態感測（Motion Capture）......2
1.2.2 姿態參考系統（AHRS）演算法及其應用......4
1.2.3 擴增實境(MR)......4
1.2.4 不同實境差異概述......4
1.3 研究目的......5
1.4 論文架構說明......5
第二章 相關技術分析與文獻探討......6
2.1 虛擬實境概述......6
2.2 虛擬實境投射原理......7
2.3 手機混合實境投射原理及深度感知能力問題概述......8
2.4 虛擬實境裝置分析......10
2.5 影像辨識定位技術分析......13
2.6 Sensors定位技術分析......15
2.7 各類型九軸感測器比較......17
2.8 PocketCard......19
第三章 系統設計......21
3.1 系統架構......21
3.2 以九軸感測器測量、模擬人體肌肉運動狀態......22
3.3 本研究之感測位置......25
3.4 虛擬角色肢體角度校正與初始化......27
3.5 九軸感測器之濾波與數值校正......29
3.6 姿態探測......34
3.7 以姿態探測進行投擲力道判斷......35
3.8 整合ARCore......37
3.9 虛實整合與互動......39
3.10 校正介面......44
3.11 Android 10以上環境整合......46
第四章 實驗結果......47
4.1 PocketCard設置......47
4.2 PocketCard連接狀態介面......48
4.3 PocketCard初始化校正......48
4.4 飄移補償與校正之前後差異......54
4.4.1 前手臂飄移處理前之數據折線圖......54
4.4.2 後手臂飄移處理前後之數據折線圖......57
4.5 卡爾曼濾波前後差異......60
4.6 動作及時捕捉實測......65
4.7 模式切換介面......71
4.8 互動功能實測-投擲與拿取......71
4.9 互動實測-瑜珈......75
4.10 互動實測-高爾夫球......80
第五章 結論......86
5.1 結論......86
5.2 未來展望......87
參考文獻......88

1.Hao, Y. and R. Foster, Wireless body sensor networks for health-monitoring applications. Physiological measurement, 2008. 29(11): p. R27.
2.Vlasic, D., et al., Practical motion capture in everyday surroundings. ACM transactions on graphics (TOG), 2007. 26(3): p. 35-es.
3.Leap Motion. Available from : https://www.rockpapershotgun.com/2014/12/02/leap-motion-3d-jam/.
4.Xbox Kinect. Available from : https://zh.taiwanbeats.tw/archives/15210/xbox-360-kinect-standalone.
5.Skog, I. and P. Händel. Calibration of a MEMS inertial measurement unit. in XVII IMEKO world congress. 2006.
6.Roh, M.-S. and B.-S. Kang, Dynamic accuracy improvement of a MEMS AHRS for small UAVs. International Journal of Precision Engineering and Manufacturing, 2018. 19(10): p. 1457-1466.
7.Madgwick, S.O., A.J. Harrison, and R. Vaidyanathan. Estimation of IMU and MARG orientation using a gradient descent algorithm. in 2011 IEEE international conference on rehabilitation robotics. 2011. IEEE.
8.Mimicking arm movement with a pan-and-tilt unit and Fitbit. Available from: https://automateeverythingsite.wordpress.com/2019/03/13/mimicking-arm-movement-with-a-pan-and-tilt-unit-and-fitbit/.
9.GAO, Z.-y., X.-j. NIU, and M.-f. GUO, Quaternion-based Kalman filter for micro-machined strapdown attitude heading reference system. Chinese Journal of Aeronautics, 2002. 15(3): p. 171-175.
10.Fuentes-Pacheco, J., J. Ruiz-Ascencio, and J.M. Rendón-Mancha, Visual simultaneous localization and mapping: a survey. Artificial intelligence review, 2015. 43(1): p. 55-81.
11.The differences between VR, AR & MR. Available from: https://www.linkedin.com/pulse/differences-between-vr-ar-mr-avi-barel.
12.VR Box.Available from: https://www.ebay.com.au/itm/VR-Headset-Version-Virtual-Reality-Box-3D-VR-Glasses-With-Controller-For-Phone-/274508440829.
13.Google Daydream. Available from: https://vr.google.com/daydream/.
14.VR-View-Method. Available from: http://news.eeworld.com.cn/qrs/article_2016030926849.html.
15.李文揚 LI, W.-Y., 基於三維重建技術之混合式虛擬與擴增實境體驗設計與實作，The Design and Implementation of Mixed Reality and 3D Reconstruction Technologies. 臺灣碩博士論文加值系統, 2019.
16.Virtuix Omni VR concepts. Available from: https://fortune.com/2016/01/08/virtuix-vr-esports-tournament-ces/.
17.Virtuix Omni Product Introduction. Available from: https://www.virtuix.com/products/.
18.HTC Vive Product Introduction.Available form: https://blog.vive.com/tw/2016/02/22/htc-vive-mwc-2016-preorder/.
19.Google Cardboard Product Introduction. Available from: https://www.tomsguide.com/uk/us/google-cardboard,review-4207.html.
20.VR Control device. Available from: http://shop.ebooks-cct.com.tw/SalePage/Index/2716774.
21.ArUco Pose estimation. Available from: https://answers.opencv.org/question/215377/aruco-orientation-using-the-function-arucoestimateposesinglemarkers/.
22.Leap Motion Controller Review. Available from: https://www.slashgear.com/leap-motion-controller-review-airspace-and-apps-22291233/.
23.Simulating Kinect Infrared and Depth Images. Available from: https://www.researchgate.net/figure/The-Kinect-transmitter-and-receiver-position-and-orientation-are-displayed-as-well-as_fig1_283734301
24.Motus Product Introduction. Available from: https://www.chatsports.com/sports-tech/a/source/motus-seeks-to-protect-pitchers-from-majors-to-little-league-13818916.
25.Xenoma E-Skin DK. Availble from: https://xenoma.com/.
26.MPU6050 6 axis IMU. Available from: http://elec.egeniuses.net/product/mpu6050-6-axis-imu/.
27.GY-80 - Multi Sensor Board. Available from: https://www.nskelectronics.in/index.php?route=product/product&product_id=478.
28.Micro:bit Product Introduction. Available from: https://www.taiwansensor.com.tw/product/microbit-board/.
29.KSB052 PocketCard. Available from: https://www.buyic.com.tw/product_info.php?products_id=6874&Twesid=qo2q6uffqcti8mav9m9fkngo27.
30.KSB040 micro:bit Lithium Battery Board.Available from: http://www.kaise.com.tw/index.php?route=product/product&product_id=207.
31.Foxlin, E. and Y. Altshuler, Motion-tracking. 2006, Google Patents.
32.Bottema, O. and B. Roth, Theoretical kinematics. Vol. 24. 1990: Courier Corporation.
33.Grochow, K., et al., Style-based inverse kinematics, in ACM SIGGRAPH 2004 Papers. 2004. p. 522-531.
34.A sensor-to-segment calibration method for motion capture system based on low cost MIMU. Available from: https://www.sciencedirect.com/science/article/pii/S0263224118307176.
35.Shoemake, K. Animating rotation with quaternion curves. in Proceedings of the 12th annual conference on Computer graphics and interactive techniques. 1985.
36.Diebel, J., Representing attitude: Euler angles, unit quaternions, and rotation vectors. Matrix, 2006. 58(15-16): p. 1-35.
37.Hanson, A.J., Visualizing quaternions, in ACM SIGGRAPH 2005 Courses. 2005. p. 1-es.
38.Halici, S., On fibonacci quaternions. Advances in applied Clifford algebras, 2012. 22(2): p. 321-327.
39.Shoemakeț, K., Animating rotation with quaternion curves. 1985.
40.Fan, Q., et al., An improved INS/PDR/UWB integrated positioning method for indoor foot-mounted pedestrians. Sensor Review, 2019.
41.Xsens MVN: Consistent Tracking of Human Motion Using Inertial Sensing.Available from: https://www.researchgate.net/figure/Xsens-MVN-avatar-in-N-pose-left-and-T-pose-right-In-the-T-pose-all-segment_fig2_324007368.
42.林晉裕LIN, J.-Y., IMU感測系統於運動軌跡的應用 The Application of IMU System to the Tracking. 臺灣碩博士論文加值系統, 2018.
43.Lee, 李.S.-C., 瞬間變速訓練對棒球選手投球球速與上肢肌力之影響 The Effect of Instantaneous speed Change Training on Baseball Players’Pitching Velocity and Muscle Strength of Upper Limbs. 臺灣碩博士論文加值系統, 2010.
44.PocketYoga, Mountain Pose. Available from: https://pocketyoga.com/pose/.
45.PocketYoga, Warrior Pose II. Available from: https://pocketyoga.com/pose/.
46.PocketYoga, Tree Pose. Available from: https://pocketyoga.com/pose/.