臺灣博碩士論文加值系統

摘　要 i
ABSTRACT ii
誌　謝 iv
目　錄 v
表目錄 vii
圖目錄 viii
壹、 緒論 1
1.1 背景與研究動機 1
1.2 研究目的 2
1.3 研究流程 3
貳、 文獻探討 5
2.1 螢幕內容分析 5
2.2 螢幕內容資料集 7
2.3 邊緣運算 8
2.4 應用程式偵測 13
2.5 監考系統 16
參、 研究方法 19
3.1 系統架構與流程 19
3.1.1 系統架構 19
3.1.2 系統流程 20
3.2 物件偵測模型建置 22
3.2.1 資料集結構 22
3.2.2 資料集收集與標註 23
3.2.3 模型訓練 25
3.3 物件偵測子功能 30
3.3.1 影像擷取方法 30
3.3.2 瀏覽器部署物件偵測模型 31
3.4 樣態決策與螢幕截圖子功能 36
3.5 光學字元辨識子功能 38
3.6 審查回放與考試管理子功能 39
肆、 實驗及數據說明 41
4.1 實驗設計 41
4.1.1 實驗環境 41
4.1.2 實驗項目 43
4.2 實驗數據說明 45
4.2.1 實驗一：推論效率實驗 45
4.2.2 實驗二：光學字元辨識花費時間實驗 48
4.2.3 實驗三：物件偵測模型效果評估 50
伍、 結論與未來展望 53
5.1 研究結論 53
5.2 未來展望 53
參　考　文　獻 54

[1]P. Zhu, Y. Li, T. Li, W. Yang, and Y. Xu, “GUI Widget Detection and Intent Generation via Image Understanding,” IEEE Access, vol. 9, pp. 160697–160707, 2021, doi: 10.1109/ACCESS.2021.3131753.
[2]B. Kang, M. Jo, and C.-S. Jeong, “Clickable Object Detection Network for a Wide Range of Mobile Screen Resolutions,” IEEE Access, vol. 10, pp. 115051–115060, 2022, doi: 10.1109/ACCESS.2022.3202222.
[3]F. N. Iandola, S. Han, M. W. Moskewicz, K. Ashraf, W. J. Dally, and K. Keutzer, “SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size,” 2016, doi: 10.48550/ARXIV.1602.07360.
[4]K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778.
[5]J. Cooley and S. Smith, “Privacy-preserving screen capture: Towards closing the loop for health IT usability,” Journal of Biomedical Informatics, vol. 46, no. 4, pp. 721–733, Aug. 2013, doi: 10.1016/j.jbi.2013.05.007.
[6]L. Bao, Z. Xing, X. Xia, D. Lo, M. Wu, and X. Yang, “psc2code: Denoising Code Extraction from Programming Screencasts,” ACM Trans. Softw. Eng. Methodol., vol. 29, no. 3, pp. 1–38, Jul. 2020, doi: 10.1145/3392093.
[7]X. Pournaras and D. A. Koutsomitropoulos, “Deep Learning on the Web: State-of-the-art Object Detection using Web-based Client-side Frameworks,” in 2020 11th International Conference on Information, Intelligence, Systems and Applications (IISA, Piraeus, Greece: IEEE, Jul. 2020, pp. 1–8. doi: 10.1109/IISA50023.2020.9284358.
[8]H. Liu, C. Li, Q. Wu, and Y. J. Lee, “Visual Instruction Tuning,” 2023, doi: 10.48550/ARXIV.2304.08485.
[9]B. Deka, Z. Huang, and R. Kumar, “ERICA: Interaction mining mobile apps,” in Proceedings of the 29th annual symposium on user interface software and technology, 2016, pp. 767–776.
[10]B. Deka et al., “Rico: A mobile app dataset for building data-driven design applications,” in Proceedings of the 30th annual ACM symposium on user interface software and technology, 2017, pp. 845–854.
[11]B. Deka et al., “An Early Rico Retrospective: Three Years of Uses for a Mobile App Dataset,” Artificial Intelligence for Human Computer Interaction: A Modern Approach, pp. 229–256, 2021.
[12]L. A. Leiva, A. Hota, and A. Oulasvirta, “Enrico: A dataset for topic modeling of mobile UI designs,” in 22nd International Conference on Human-Computer Interaction with Mobile Devices and Services, 2020, pp. 1–4.
[13]S. Bunian, K. Li, C. Jemmali, C. Harteveld, Y. Fu, and M. S. Seif El-Nasr, “VINS: Visual Search for Mobile User Interface Design,” in Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, Yokohama Japan: ACM, May 2021, pp. 1–14. doi: 10.1145/3411764.3445762.
[14]G. Li, G. Baechler, M. Tragut, and Y. Li, “Learning to Denoise Raw Mobile UI Layouts for Improving Datasets at Scale,” in CHI Conference on Human Factors in Computing Systems, New Orleans LA USA: ACM, Apr. 2022, pp. 1–13. doi: 10.1145/3491102.3502042.
[15]J. Wu, S. Wang, S. Shen, Y.-H. Peng, J. Nichols, and J. P. Bigham, “WebUI: A Dataset for Enhancing Visual UI Understanding with Web Semantics,” in Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, Hamburg Germany: ACM, Apr. 2023, pp. 1–14. doi: 10.1145/3544548.3581158.
[16]K. Cao, Y. Liu, G. Meng, and Q. Sun, “An Overview on Edge Computing Research,” IEEE Access, vol. 8, pp. 85714–85728, 2020, doi: 10.1109/ACCESS.2020.2991734.
[17]T. Parisi, WebGL: up and running. O’Reilly Media, Inc., 2012.
[18]A. Dakkak, C. Pearson, and W.-M. Hwu, “WebGPU: A Scalable Online Development Platform for GPU Programming Courses,” in 2016 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW), Chicago, IL, USA: IEEE, May 2016, pp. 942–949. doi: 10.1109/IPDPSW.2016.63.
[19]C. Gerard, Practical machine learning in JavaScript: TensorFlow. js for web developers. Springer, 2021.
[20]O. R. developers, “ONNX Runtime.” Jan. 2018. Accessed: Aug. 21, 2023. [Online]. Available: https://github.com/microsoft/onnxruntime
[21]D. Smilkov et al., “TensorFlow.js: Machine Learning for the Web and Beyond,” 2019, doi: 10.48550/ARXIV.1901.05350.
[22]E. N. Li Yulong Wang, Du, “ONNX Runtime Web—running your machine learning model in browser,” Microsoft Open Source Blog, Sep. 02, 2021. https://cloudblogs.microsoft.com/opensource/2021/09/02/onnx-runtime-web-running-your-machine-learning-model-in-browser/ (accessed Aug. 22, 2023).
[23]J. Lee, R. Tang, and J. Lin, “JavaScript Convolutional Neural Networks for Keyword Spotting in the Browser: An Experimental Analysis,” arXiv preprint arXiv:1810.12859, 2018.
[24]A. Karpathy, “ConvNetJS: Deep Learning in your browser,” Accessed: Mar, vol. 23, p. 2018, 2014.
[25]“Home - Keras.js Documentation.” https://transcranial.github.io/keras-js-docs/ (accessed Sep. 04, 2023).
[26]“cazala/synaptic: architecture-free neural network library for node.js and the browser.” https://github.com/cazala/synaptic/tree/master (accessed Aug. 22, 2023).
[27]“WebDNN.” Machine Intelligence Laboratory (The University of Tokyo), Aug. 20, 2023. Accessed: Aug. 22, 2023. [Online]. Available: https://github.com/mil-tokyo/webdnn
[28]“ml5js/ml5-library.” ml5, Aug. 21, 2023. Accessed: Aug. 22, 2023. [Online]. Available: https://github.com/ml5js/ml5-library
[29]J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You Only Look Once: Unified, Real-Time Object Detection.” arXiv, May 09, 2016. Accessed: Jun. 25, 2023. [Online]. Available: http://arxiv.org/abs/1506.02640
[30]Y. Amit, P. Felzenszwalb, and R. Girshick, “Object detection,” Computer Vision: A Reference Guide, pp. 1–9, 2020.
[31]G. Jocher, “YOLOv5 by Ultralytics.” May 2020. doi: 10.5281/zenodo.3908559.
[32]G. Jocher, A. Chaurasia, and J. Qiu, “YOLO by Ultralytics.” Jan. 2023. Accessed: Jun. 28, 2023. [Online]. Available: https://github.com/ultralytics/ultralytics
[33]Z. Zheng, P. Wang, W. Liu, J. Li, R. Ye, and D. Ren, “Distance-IoU loss: Faster and better learning for bounding box regression,” in Proceedings of the AAAI conference on artificial intelligence, 2020, pp. 12993–13000.
[34]X. Li et al., “Generalized focal loss: Learning qualified and distributed bounding boxes for dense object detection,” Advances in Neural Information Processing Systems, vol. 33, pp. 21002–21012, 2020.
[35]廖家瑤, “應用RESTful技術於線上程式考試及監考系統之研究,” 國防大學, 桃園縣, 2017. [Online]. Available: https://hdl.handle.net/11296/y59hs7
[36]杜威德, “線上程式設計考試系統監考機制之研究,” 國防大學, 桃園縣, 2018. [Online]. Available: https://hdl.handle.net/11296/u98dvz
[37]卜邦榮, “線上考試監控系統之設計,” 東吳大學, 台北市, 2010. [Online]. Available: https://hdl.handle.net/11296/9rz6xr
[38]S. Prathish and K. Bijlani, “An intelligent system for online exam monitoring,” in 2016 International Conference on Information Science (ICIS), IEEE, 2016, pp. 138–143.
[39]P. Beust, I. Duchatelle, and V. Cauchard, “Exams taken at the student’s home,” in Online, Open and Flexible Higher Education Conference, EADTU 2018, 2018.
[40]Y. Cheung and Q. Peng, “Eye Gaze Tracking With a Web Camera in a Desktop Environment,” IEEE Trans. Human-Mach. Syst., vol. 45, no. 4, pp. 419–430, Aug. 2015, doi: 10.1109/THMS.2015.2400442.
[41]“Kryterion : Home,” Kryterion. https://www.kryterion.com/ (accessed Aug. 23, 2023).
[42]“The ProctorU Proctoring Platform - Advanced Exam Technology Backed by Human Validation,” ProctorU. https://www.proctoru.com/ (accessed Aug. 23, 2023).
[43]S. Liu et al., “Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection,” 2023, doi: 10.48550/ARXIV.2303.05499.
[44]M. Caron et al., “Emerging Properties in Self-Supervised Vision Transformers,” 2021, doi: 10.48550/ARXIV.2104.14294.
[45]B. Su, H. Zhang, J. Li, and Z. Zhou, “Towards Few-Shot Open-Set Object Detection,” 2022, doi: 10.48550/ARXIV.2210.15996.
[46]J. Deng, W. Dong, R. Socher, L.-J. Li, Kai Li, and Li Fei-Fei, “ImageNet: A large-scale hierarchical image database,” in 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL: IEEE, Jun. 2009, pp. 248–255. doi: 10.1109/CVPR.2009.5206848.
[47]“MediaTrackConstraints: displaySurface property - Web APIs | MDN,” Apr. 07, 2023. https://developer.mozilla.org/en-US/docs/Web/API/MediaTrackConstraints/displaySurface (accessed Jun. 29, 2023).
[48]“MediaStreamTrack: getSettings() method - Web APIs | MDN,” Apr. 07, 2023. https://developer.mozilla.org/en-US/docs/Web/API/MediaStreamTrack/getSettings (accessed Jun. 29, 2023).
[49]W. Setianto, “YOLOv8 with onnxruntime-web.” Aug. 07, 2023. Accessed: Aug. 09, 2023. [Online]. Available: https://github.com/Hyuto/yolov8-onnxruntime-web
[50]W. Setianto, “Object Detection using YOLOv8 and Tensorflow.js.” Aug. 06, 2023. Accessed: Aug. 09, 2023. [Online]. Available: https://github.com/Hyuto/yolov8-tfjs
[51]“Introducing the WebAssembly backend for TensorFlow.js.” https://blog.tensorflow.org/2020/03/introducing-webassembly-backend-for-tensorflow-js.html (accessed Aug. 18, 2023).