近年來因應新冠肺炎的影響，全球各地開始限制外出，嚴重地方甚至封城，這導致
許多公共展覽空間，例如：博物館和美術館展覽，紛紛被迫停止或延期。同時在疫情的
影響下，民眾外出參與藝文活動的意願大幅降低，這也使得許多國家的博物館和美術館
開始進行數位轉型，其目的在於避免人與人之間密切的接觸，進而導致疫情擴散，且藉
由數位媒體的傳播，更能吸引到新的客群以及合作，達到知識資產分享之目的。
本研究希望讓使用者在不出門的情況下，能夠自由的在家中進行佈展與參觀，同時
以身邊常見之設備進行混合實境體驗，因此本研究選擇手機作為開發平台。考慮到傳統
頭戴式裝置其採用魚眼鏡片進行變形成像，導致畫面有所變形不真實，於是本研究自行
開發頭戴式裝置，使用者只須將手機放入自製之頭戴式裝置內，便能透過鏡面反射
(Specular Reflection)原理體驗混合實境的觀展效果。
在本研究中，透過 Google ARCore 函式庫，可以得知使用者當前所在位置，與現實
空間中環境樣貌，並使用單點雷射掃描系統對欲替換之現實物體進行掃描，以便實現虛
實物體替換功能，並藉由自動縮放與矯正系統，使虛擬物體能夠覆蓋住現實物體。
為了讓使用者能夠更隨心所欲地進行佈展，本研究另外設計出透過手指位置與即時
語音指令，便可進行佈展的物體建置系統，基於 OpenCV 函式庫和顏色通道轉換演算法，
達到捕捉使用者手部之效果，透過混合實境中的環境感知技術，讓使用者可以憑藉著手
指與語音指令，自由地將虛擬物品建置於現實場景中。
同時為了讓使用者可以與虛擬物體進行互動，本研究基於 Android Studio 的 Google
Translate 函式庫，開發連續性背景是語音對話系統，讓使用者可透過語音的方式，與虛
擬導覽員互動導覽和參觀佈展，藉以到無須出門就能體驗參觀博物館之混合實境體驗。

In recent years, the pandemic of COVID-19 had serious impacts globally. The travel
restrictions leaded to the closure of many museums and art galleries. In this situation, the
intention of visitors to participate these cultural activities was reduced. On the other hand, the
digital transformation of museums and art galleries are introduced to avoid close contact of
visitors.
In order to reach the quality improvement of virtual exhibition, the smartphones are chosen
as the development platform. In order to avoid the weight issue and cyber sickness of traditional
head-mounted device (HMD), the new HMD is implemented in this study.
The Google ARCore library is used to create the mix-reality (MR) exhibition. With the
used of single-point laser scan system, the real objects could be replaced as a mix-reality
collection with the automatic zoom and correction ability. Based on the OpenCV library and
color channel conversion algorithm, the object placement system is implemented with users’
fingers. This system allows users to place virtual objects in real scenes with the voice command
directly.
In addition, based on Android Studio’s Google Translate library. Users can use Speech to
Text (STT) and Text to Speech (TTS) to exhibition visits and interact

摘要.............................................................................................................................................i
Abstract.......................................................................................................................................ii
誌謝...........................................................................................................................................iii
目錄...........................................................................................................................................iv
表目錄.......................................................................................................................................vi
圖目錄......................................................................................................................................vii
第一章 緒論....................................................................................................................1
1.1 研究動機................................................................................................................1
1.2 研究回顧................................................................................................................2
1.2.1 虛實物體替換系統(Virtual Object Replacement System, VORS) ...........2
1.2.2 HSV 顏色辨識（Hue Saturation Value Color Detection）系統及應用 ..2
1.2.3 語音辨識和語音合成(Speech-to-Text、Text-to-Speech).........................2
1.2.4 運動追蹤和視覺地圖建構(MT、vSLAM) ..............................................2
1.2.5 混合實境(Mixed Reality)和相關技術.......................................................3
1.3 研究目的................................................................................................................3
1.4 論文架構說明........................................................................................................3
第二章 相關技術分析與文獻探討................................................................................4
2.1 數位轉型概述........................................................................................................4
2.2 HSV 顏色辨識概述...............................................................................................5
2.3 語音技術概述........................................................................................................6
2.4 語音技術原理........................................................................................................7
2.5 虛擬實境概述........................................................................................................8
2.6 虛擬實境投射原理................................................................................................9
2.7 手機混合實境投射原理及深度感知能力問題概述............................................9
2.8 虛擬實境裝置分析.............................................................................................. 11
2.9 影像標記物辨識定位技術分析..........................................................................15
2.10 相關設備探討......................................................................................................16
第三章 系統設計..........................................................................................................17
3.1 系統功能整體流程..............................................................................................17
3.2 頭戴式裝置設計..................................................................................................19
3.2.1 自製頭戴式裝置.............................................................................................19
3.2.2 頭戴式裝置設計過程與討論.........................................................................20
3.3 虛實物體替換設計..............................................................................................25
3.3.1 現實環境中物體偵測.....................................................................................25
3.3.2 虛擬物體放置.................................................................................................29
3.3.3 虛擬物體校準.................................................................................................31
v
3.4 手動建置虛擬物體於現實環境系統設計..........................................................36
3.4.1 顏色空間通道轉換.........................................................................................36
3.4.2 記憶體暫存滿容量之問題.............................................................................41
3.4.3 手部辨識設計.................................................................................................41
3.4.4 空間映射與校正.............................................................................................44
3.4.5 手動建置之景深設計.....................................................................................46
背景式連續性語音對話系統設計..................................................................................47
3.5 虛擬導覽員..........................................................................................................49
3.6 辨識現實物體外輪廓設計過程與討論..............................................................53
3.7.1 透過 OpenCV 取得特徵點並繪製成表輪廓.................................................53
3.7.2 現實物體與使用者距離.................................................................................55
3.7.3 整合 ARCore...................................................................................................58
3.7.4 虛擬世界與現實空間距離比.........................................................................59
3.7.5 整合 ARCore Depth Camera 並取得現實環境深度值..................................61
第四章 實驗結果..........................................................................................................62
4.1 頭戴式裝置設置.......................................................................................................63
4.2 初始化與開始介面...................................................................................................64
4.3 虛擬導覽員設定與跟隨...........................................................................................64
4.4 自動模式實測(上午).................................................................................................67
4.5 自動模式實測(下午).................................................................................................70
4.6 手動模式實測(白天).................................................................................................73
4.7 手動模式實測(下午).................................................................................................75
4.8 展覽模式實測...........................................................................................................76
4.9 防重複性替換實驗....................................................................................................77
4.10 重新掃描.................................................................................................................78
4.11 黑夜環境與背景白色時實驗 .................................................................................79
4.12 複雜與單純環境下手動模式實驗.........................................................................81
4.13 虛擬物體深度遮擋.................................................................................................83
第五章 結論..................................................................................................................85
5.1 結論............................................................................................................................85
5.2 未來展望....................................................................................................................87
參考文獻..................................................................................................................................88
Extended Abstract.....................................................................................................................90

1. Parliament, U. Talks and events. 2021 [cited 2021; Available from:
https://www.parliament.uk/visiting/visiting-and-tours/online-talks-and-events/.
2. Parliament, U. The British Museum. 2021; Available from:
https://www.google.co.uk/maps/@51.5191409,-
0.1274224,2a,75y,155.25h,92.64t/data=!3m6!1e1!3m4!1sKiaaOuVwgN9QXWcdhDC
giA!2e0!7i13312!8i6656?hl=en.
3. Parliament, U. Houses of Parliament 360° virtual tour. 2021; Available from:
https://www.parliament.uk/visiting/virtualtour/?utm_source=visitlondon&utm_medium=website&utm_campaign=virtually-london.
4. Fuentes-Pacheco, J., J. Ruiz-Ascencio, and J.M. Rendón-Mancha, Visual simultaneous
localization and mapping: a survey. Artificial Intelligence Review, 2015. 43(1): p. 55-
81.
5. Curiosity. Nefertari's Tomb. 2018; Available from:
https://curiositystream.com/video/2147.
6. Demons, B.o. Nefertari: Journey to Eternity - VR in Egypt. 2018; Available from:
https://www.youtube.com/watch?v=XeVtJiHr0Nk.
7. HSV Model. Available from:
https://zh.wikipedia.org/wiki/HSL%E5%92%8CHSV%E8%89%B2%E5%BD%A9%
E7%A9%BA%E9%97%B4.
8. LearnOpenCV. Invisibility Cloak using OpenCV. 2019; Available from:
https://www.youtube.com/watch?v=YLnP2Ge65MU.
9. Zenbo. Available from: https://www.asus.com/tw/Commercial-IntelligentRobot/Zenbo/.
10. Mini, G.N.; Available from: https://www.ruten.com.tw/item/show?21952115993793.
11. Android Activity. Available from: http://tyroandroid.blogspot.com/2009/07/androidactivity.html.
12. VR Box. Available from: https://shopvr.se/produkt/vr-box-2-0/.
13. Google Daydream. Available from: https://arvr.google.com/daydream/.
14. VR-View-Method. Available from:
http://news.eeworld.com.cn/qrs/article_2016030926849.html.
15. 張廷嘉 ZHANG, T.-J., 應用於混合與擴增實境遊戲之穿戴式多節點體感即時偵
測系統，The Wireless Body Sensor Networks with The Augmented Reality and
Mixed Reality Games. 臺灣碩博士論文加值系統, 2021.
16. 李文揚 LI, W.-Y., 基於三維重建技術之混合式虛擬與擴增實境體驗設計與實
作，The Design and Implementation of Mixed Reality and 3D Reconstruction
Technologies. 臺灣碩博士論文加值系統, 2019.
100
17. Virtuix Omni VR concepts. Available from: https://fortune.com/2016/01/08/virtuix-vresports-tournament-ces/.
18. Virtuix Omni Product Introduction. Available from:
https://www.virtuix.com/products/.
19. HTC Vive Product Introduction. Available from:
https://blog.vive.com/tw/2016/02/22/htc-vive-mwc-2016-preorder/.
20. HoloLens2. Available from: https://4experience.co/hololens-2-vs-hololens-1-whatsnew/.
21. VR Bluetooth joystick.
22. ArUco Pose estimation. Available from:
https://answers.opencv.org/question/215377/aruco-orientation-using-the-functionarucoestimateposesinglemarkers/.
23. Foxlin, E. and Y. Altshuler, Motion-tracking. 2006, Google Patents.
24. Single Frame YUV420. Available from:
https://en.wikipedia.org/wiki/YUV?fbclid=IwAR0psZdtg03m61WRGHDJo5D_pAPI
MEY3a5fpHIpvhiun2XwpguS5Tj78UQE#Y%E2%80%B2UV420sp_(NV21)_to_RG
B_conversion_(Android).
25. YUV420 Sampling Chart. Available from:
https://www.shuzhiduo.com/A/amd0wmBq5g/?fbclid=IwAR2-
fb8sHJnulp6stgKEHqI3mjipfybvz_xo28BxpVagQw92yMzk9MzpDtU.
26. There are different results for camera texture in the editor and mobile environment.
Available from: https://github.com/google-ar/arcore-unity-sdk/issues/520.
27. ARCore. Available from: https://developers.google.com/ar.
28. Lab, G.A. Introduction to Depth on Unity targeting Android. Available from:
https://developers.google.com/ar/develop/unity/depth/introduction