傳統的人機互動介面操作方式單一且缺乏觸覺回饋，使用者無法在實體空間以真實與沉浸的方式來參與互動系統。本論文研究主要開發一個可觸握式使用者介面、擴增影像與重量感回饋的整合性互動系統，稱為「力回饋可觸握式擴增互動（Tangible Augmented Interactive, TAI）系統」，並以虛實結合的釣魚互動遊戲為例呈現在實體空間。在論文中，我們也探討了可觸握式使用者介面、擴增影像應用及力回饋系統的相關文獻。
TAI系統，主要內容包含以微電子控制器及數位自造方式製作的一支可觸握式使用者介面(TUI)的實體釣竿；並以雷射光模擬成釣魚線及魚餌，藉由三軸感測器與馬達來控制釣竿角度與釣魚線關係；運用動態影像辨識及空間矩即時計算射魚餌的所在位置；運用槓桿原理及釣竿的重心變化來設計釣到不同魚種時釣竿有著不同的重量感回饋。除此，本研究以MQTT物聯網溝通協定來整合TUI釣竿及電腦中的池塘互動感測系統及MQTT伺服器等物件，透過物件的相互溝通觸發互動過程中的事件。
本研究融合了可觸握式使用者介面、擴增影像及重量感回饋，透過物聯網整合了釣竿和池塘兩個獨立的物件，打造出一個虛實合一的釣魚互動遊戲系統，其中重量感回饋是本研究的最主要特色，這是目前台灣在互動系統開發比較罕見的。本研究成果加強了使用者的互動體驗感受，並將「資訊應用」結合「數位自造」，展現了跨域的精神，希望藉以本研究能啟發未來更多虛實融合的互動系統開發與應用。

Traditional human-computer interactive interfaces are often limited to a single mode of operation and lack of haptic feedback. It restricts users from engaging in physical spaces in a realistic and immersive manner. This paper introduces a study focused on developing an integrated system called “Tangible Augmented Interactive (TAI) System” that incorporates a tangible user interface, augmented imagery, and weight sensation feedback. The system is demonstrated through a mixed reality fishing game example to enhance physical space interaction. This research explores relevant literature on tangible user interfaces, augmented imagery, and force feedback systems.
The TAI system features a tangible user interface (TUI) fishing rod created using microelectronic controllers and digital fabrication methods. Laser beam simulates fishing line and bait, controlled by accelerometer and motor to adjust rod angles and manage fishing line. Dynamic image recognition and spatial moment calculations are employed for real-time detection of bait location. fish bites. Additionally, leveraging and rod center of gravity variations provides weight sensation feedback specific to different fish species caught. Integration via the MQTT protocol connects the TUI rod, pond interactive system, and MQTT server, enabling mutual object communication to trigger events during interactive processes.
By combining tangible user interface, augmented imagery, and weight sensation feedback, this study integrates separate objects—a fishing rod and a fish pond—into a unified mixed reality fishing game system. The emphasis on weight perception feedback distinguishes this research, representing an uncommon development in interactive system design in Taiwan. This achievement enhances user’s interaction experiences, merging information applications with digital fabrication and showcasing interdisciplinary innovation. This study aims to inspire future developments and applications of integrated virtual and physical interaction systems.

[1] P. Langdon, S. Keates, P. Clarkson, and P. Robinson, “Using Haptic Feedback to Enhance Computer Interaction for Motion-Impaired Users,” in 3rd International Conference on Disability, Virtual Reality and Associated Technologies (ICDVRAT), 2000, pp. 25-32.
[2] P. Milgram and F. Kishino, “A Taxonomy of Mixed Reality Visual Displays,” IEICE Trans. Information Systems, vol. vol. E77-D, no. 12, pp. 1321-1329, 12/01 1994.
[3] H. Ishii and B. Ullmer, “Tangible Bits: Towards Seamless Interfaces Between People, Bits and Atoms,” presented at the Proceedings of the ACM SIGCHI Conference on Human factors in computing systems, Atlanta, Georgia, USA, 1997.
[4] 未來創意互動實驗室(FBI Lab)。2018印象清華¬¬¬-物聯網科技藝術節。檢索於2024年3月。檢索自https://techart.nthu.edu.tw/?page_id=68
[5] Graffiti Research Lab. L.A.S.E.R. Tag. Accessed: Aug 2024. [Online]. Available: https://graffitiresearchlab.com/blog/projects/laser-tag/#video
[6] O. Zuckerman and A. Gal-Oz, “To TUI or not to TUI: Evaluating performance and preference in tangible vs. graphical user interfaces,” International Journal of Human-Computer Studies, vol. 71, no. 7-8, pp. 803-820, 2013.
[7] S. Moroi, S. Sasada, and R. Shibata, “The Floating Words for Kids: Interactive Installation for Learning Alphabet,” presented at the ACM SIGGRAPH 2002 conference abstracts and applications, San Antonio, Texas, 2002.
[8] 陳威郡，“平衡與介入”互動裝置藝術創作，碩士論文，國立臺北藝術大學，2018。
[9] D. Merrill, E. Sun, and J. Kalanithi, “Sifteo Cubes,” presented at the CHI '12 Extended Abstracts on Human Factors in Computing Systems, Austin, Texas, USA, 2012.
[10] 曾筱筠，“可觸握方塊”介面在互動設計之創新應用，碩士論文，國立臺北藝術大學，2014。
[11] J. A. Gallud, R. Tesoriero, M. D. Lozano, V. M. R. Penichet, and H. M. Fardoun, “The Use of Tangible User Interfaces in K12 Education Settings: A Systematic Mapping Study,” IEEE Access, vol. 10, pp. 24824-24842, 2022.
[12] N. Yannier, K. R. Koedinger, and S. E. Hudson, “Learning from Mixed-Reality Games: Is Shaking a Tablet as Effective as Physical Observation?,” presented at the Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, Seoul, Republic of Korea, 2015.
[13] C. Sylla, P. Branco, C. Coutinho, and E. Coquet, “TUIs vs. GUIs: Comparing the learning potential with preschoolers,” Personal and Ubiquitous Computing, vol. 16, pp. 421-432, 04/01 2011, doi: 10.1007/s00779-011-0407-z.
[14] Z. B. Simpson. Mine-Control. Accessed: Mar. 2024. [Online]. Available: https://www.mine-control.com/
[15] Mine-Control. Save the Baby? Accessed: Mar. 2024. [Online]. Available: https://www.mine-control.com/work#/save-the-baby/
[16] Mine-Control. Mondriaan. Accessed: Jul. 2024. [Online]. Available: https://www.mine-control.com/work#/mondriaan/
[17] 廖志唯，架構在躍介物聯網系統下的擴增實境創意應用與情感愉悅探討，碩士論文，國立清華大學，2021。
[18] Y. Matsuura and N. Koizumi, “Fairlift: Interaction With Mid-air Images on Water Surface,” presented at the ACM SIGGRAPH 2018 Emerging Technologies, Vancouver, British Columbia, Canada, 2018.
[19] M. Otsubo, “Optical Imaging Apparatus and Optical Imaging Method,” U.S. Patent 8702252B2, 2014.
[20] Y. Suzuki, M. Kobayashi, Y. Shimada, A. Nakayama, and S. Iwaki, “Untethered Force Feedback Interface That Uses Air Jets,” presented at the ACM SIGGRAPH 2004 Emerging technologies, Los Angeles, California, 2004.
[21] Y. Nakano et al., “Invisible: the Shadow Chaser,” presented at the ACM SIGGRAPH 2006 Emerging technologies, Boston, Massachusetts, 2006.
[22] Tomy Company Ltd.. Super Realistic Fishing Simulation Game Virtual Masters Real. Accessed: Aug. 2024. [Online]. Available: https://www.takaratomy.co.jp/english/product_release/pdf/p110126_en.pdf
[23] Bandai Namco Amusement Inc.. Ace Angler. Accessed: Aug. 2024. [Online]. Available: https://trsp.bn-am.net/index.html
[24] J. Noguchi et al., “Powder Screen: a Virtual Materializer,” presented at the ACM SIGGRAPH 2006 Emerging technologies, Boston, Massachusetts, 2006.
[25] D. P. Kingma and M. Welling, “Auto-encoding Variational Bayes,” 2013, arXiv:1312.6114.
[26] Y. Honda and A. Kawamura, “Winding System for Fishing Simulator with Real Rod,” in 2021 20th International Symposium on Communications and Information Technologies (ISCIT), 19-22 Oct. 2021 2021, pp. 61-66, doi: 10.1109/ISCIT52804.2021.9590606.
[27] OpenCV. Opencv-python. Accessed: Mar. 2024. [Online]. Available: https://github.com/opencv/opencv-python
[28] OpenCV. cv::Moments Class Reference. Accessed: Jul. 2024. [Online]. Available: https://docs.opencv.org/4.x/d8/d23/classcv_1_1Moments.html
[29] FBI_LAB。躍界海精靈。檢索於2024年7月。檢索自https://digitalartfestival.tw/daf15/zh/award-4-0204.html。
[30] B. Wenzel. Unique Illustrations of Endangered Animals. Accessed: Jul. 2024. [Online]. Available: https://www.iliketowastemytime.com/2014/03/07/unique-illustrations-of-endangered-animals-33-pics
[31] 淘寶網站。藤編饅頭筐。檢索於2024年7月。檢索自https://world.taobao.com/item/794742096715.htm?spm=a21wu.10013406-tw.taglist-content.8.79bb6da4qZ2YxL
[32] A. Banks and R. Gupta. MQTT Version 3.1.1. Accessed: Feb. 2024. [Online]. Available: http://docs.oasis-open.org/mqtt/mqtt/v3.1.1/os/mqtt-v3.1.1-os.html
[33] 淘寶網站。迷你出煙器。檢索於2024年8月。檢索自https://m.tb.cn/h.glp31nD5T8PvUu7?tk=YJBx3V2Z9Ur