簡易檢索 / 詳目顯示

回結果列表
研究生： 楊馥名
Yong, Seraphina
論文名稱： 虛擬實境雙人合作中指向性聲音提示對人們空間資訊與物體關係的習得之影響
Targeted Sonification Supports Spatial Information Acquisition for Spatial Knowledge-Based Activity and Collaboration in Virtual Reality
指導教授： 朱宏國
Chu, Hung-Kuo
曾元琦
Tseng, Yuan-Chi
口試委員: 王浩全
Wang, Hao-Chuan
楊政達
Yang, Cheng-Ta
詹力韋
Chan, Li-Wei
學位類別： 碩士
Master
系所名稱： 電機資訊學院 - 資訊工程學系
Computer Science
論文出版年： 2019
畢業學年度： 107
語文別： 英文
論文頁數： 64
中文關鍵詞： 虛擬實境 、空間性 、聲音 、知識 、協同工作
外文關鍵詞： VR, spatial, audio, knowledge, collaboration
相關次數： 點閱：2  下載：0 
分享至:
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報

    • 虛擬環境現在用於各種應用，從專業培訓到室內設計。在許多情況下，例如醫學訓練和基於空間性規則的活動（例如團隊運動），獲得的空間知識準確性是一個重要的部分。虛擬環境中出現空間認知的問題是它提供給人類視覺資訊的不足，導致人對環境空間特徵無法有精確判斷。這甚至存在於虛擬實境中，其中距離估計的準確性仍然表現的比實體世界低。在我們調查其他感官對空間認知的支持之後，聽覺刺激似乎是大多數科技最易於整合的選擇，因為它提供距離和方向的信息。我們分別進行了兩個實驗來確認指向性聲音對空間知識獲得之影響，接下來再用針對性的空間音頻線索和聲波化中表現出來，增強人在雙人合作情境下對任務相關信息的接收。實驗結果告知未來的設計該如何整合指向性聲音去支持空間性的虛擬活動，也可以針對每一種情境裡最傾向的感官來增強活動裡的感受。

    Virtual environments are now used in a diverse array of applications, from professional training to interior design. In many cases, such as medical training and rule-based spatial activity (e.g. team sports), accuracy of acquired spatial knowledge is an important component of the activity. An issue that arises for spatial cognition in virtual environments is the inadequacy of visual affordances for humans to form precise judgments about spatial features in the environment. This exists even in virtual reality, where distance estimation still demonstrates declined accuracy compared to in the physical world. After surveying the disposition of other sensory modalities to spatial cognition, auditory stimuli appeared to be the most holistic and easily integrable option for most technologies, due to its provision of distance and orientational information. We performed a set of two lab experiments to confirm, respectively, the impact of spatial auditory cues on spatial knowledge acquisition, and then in a collaborative context to explore the effect of interlocutor-controlled spatial auditory cues on addressee understanding of communicated information. The findings show that targeted spatial audio cues and sonification do enhance reception of task-relevant information, and inform how future design of technology-mediated spatial activity may target perceptual enhancement to accommodate an activity’s sensory predisposition.

    Acknowledgments ii Abstract iii Table of Contents 1 Chapter 1 Introduction 3 Chapter 2 Background 8 2.1 Spatial Perception, the Vergence-Accommodation Conflict, and Audition 8 2.2 Audiovisual Integration of Spatio-temporal and Spatial Relational Processing 9 2.2.1Auditory and Visual Integration in Semantic and Spatio-temporal processing 9 2.2.2 Auditory Processing of Spatial Relational References 11 Chapter 3 — Experiment 1 16 3.1 Method 16 3.1.1 Experiment Design 16 3.1.2 Experiment Materials and Virtual Reality System 17 3.1.3 Participants 19 3.1.4 Procedure 19 3.1.5 Measurement 20 3.2 Results 21 3.3 Discussion 22 Chapter 4 — Experiment 2 23 4.1 Method 24 4.1.1 Experiment Design 24 4.1.2 AuralTrace Prototype: Pointer Trajectory Pitch Sonification 26 4.1.3 Experiment Stages and Tasks 29 4.1.4 Participants 33 4.1.5 Procedure 33 4.1.6 Measurement 35 4.2 Results 39 4.2.1 Post-Communication SRI Task Performance 39 4.2.2 Subjective Evaluation of Pointer Effects 43 4.3 Discussion 48 4.3.1 Effects of Pitch Perception on Understanding Higher-level Spatial Information 48 4.3.2 Perceptual Context for Communication and Collaboration 50 4.3.3 AuralTrace Effects: Explicit or Implicit? 52 4.3.4 Future Work 53 Chapter 5 — General Discussion & Conclusion 56 References 58

    1. Miguel A. Alonso-Arevalo, Simon Shelley, Dik Hermes, Jacqueline Hollowood, Michael Pettitt, Sarah Sharples, and Armin Kohlrausch. 2012. Curve shape and curvature perception through interactive sonification. ACM Transactions on Applied Perception 9, 4: 1–19. https://doi.org/10.1145/2355598.2355600
    2. Giuliano Benelli, Maurizio Caporali, Antonio Rizzo, and Elisa Rubegni. 2001. Design concepts for learning spatial relationships. In Proceedings of the 19th annual international conference on Computer documentation  - SIGDOC ’01, 22. https://doi.org/10.1145/501516.501522
    3. M. Billinghurst and S. Weghorst. The use of sketch maps to measure cognitive maps of virtual environments. In Proceedings Virtual Reality Annual International Symposium ’95, 40–47. https://doi.org/10.1109/VRAIS.1995.512478
    4. Jennifer K Bizley and Yale E Cohen. 2013. The what, where and how of auditory-object perception. Nature reviews. Neuroscience 14, 10: 693–707. https://doi.org/10.1038/nrn3565
    5. Jennifer K. Bizley and Andrew J. King. 2008. Visual–auditory spatial processing in auditory cortical neurons. Brain Research 1242: 24–36. https://doi.org/10.1016/J.BRAINRES.2008.02.087
    6. GEORGE M. BODNER and ROLAND B. GUAY. 1997. The Purdue Visualization of Rotations Test. The Chemical Educator 2, 4: 1–17. https://doi.org/10.1007/s00897970138a
    7. James Broderick, Jim Duggan, and Sam Redfern. 2018. The Importance of Spatial Audio in Modern Games and Virtual Environments. In 2018 IEEE Games, Entertainment, Media Conference (GEM), 1–9. https://doi.org/10.1109/GEM.2018.8516445
    8. Yuan-Chia Chang, Hao-Chuan Wang, Hung-kuo Chu, Shung-Ying Lin, and Shuo-Ping Wang. 2017. AlphaRead. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing - CSCW ’17, 2246–2259. https://doi.org/10.1145/2998181.2998258
    9. Alan R. Cohen, Subash Lohani, Sunil Manjila, Suriya Natsupakpong, Nathan Brown, and M. Cenk Cavusoglu. 2013. Virtual reality simulation: basic concepts and use in endoscopic neurosurgery training. Child’s Nervous System 29, 8: 1235–1244. https://doi.org/10.1007/s00381-013-2139-z
    10. Christopher M. Conway and Morten H. Christiansen. 2005. Modality-Constrained Statistical Learning of Tactile, Visual, and Auditory Sequences. Journal of Experimental Psychology: Learning, Memory, and Cognition 31, 1: 24–39. https://doi.org/10.1037/0278-7393.31.1.24
    11. H. Couclelis, R. G. Golledge, N. Gale, and W. Tobler. 1987. Exploring the anchor-point hypothesis of spatial cognition. Journal of Environmental Psychology 7, 2: 99–122. https://doi.org/10.1016/S0272-4944(87)80020-8
    12. A. Cummings, R. Čeponienė, A. Koyama, A.P. Saygin, J. Townsend, and F. Dick. 2006. Auditory semantic networks for words and natural sounds. Brain Research 1115, 1: 92–107. https://doi.org/10.1016/j.brainres.2006.07.050
    13. Sarah D’Angelo and Darren Gergle. 2016. Gazed and Confused. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems - CHI ’16, 2492–2496. https://doi.org/10.1145/2858036.2858499
    14. H.Q. Dinh, N. Walker, L.F. Hodges, Chang Song, and A. Kobayashi. Evaluating the importance of multi-sensory input on memory and the sense of presence in virtual environments. In Proceedings IEEE Virtual Reality (Cat. No. 99CB36316), 222–228. https://doi.org/10.1109/VR.1999.756955
    15. Zohar Eitan, Asi Schupak, Alex Gotler, and Lawrence E. Marks. 2014. Lower Pitch Is Larger, Yet Falling Pitches Shrink. Experimental Psychology 61, 4: 273–284. https://doi.org/10.1027/1618-3169/a000246
    16. Jing Feng, Ian Spence, and Jay Pratt. 2007. Playing an Action Video Game Reduces Gender Differences in Spatial Cognition. Psychological Science 18, 10: 850–855. https://doi.org/10.1111/j.1467-9280.2007.01990.x
    17. Daniel J. Finnegan, Eamonn O’Neill, and Michael J. Proulx. 2016. Compensating for Distance Compression in Audiovisual Virtual Environments Using Incongruence. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems - CHI ’16, 200–212. https://doi.org/10.1145/2858036.2858065
    18. Susan R. Fussell, Robert E. Kraut, and Jane Siegel. 2000. Coordination of communication. In Proceedings of the 2000 ACM conference on Computer supported cooperative work  - CSCW ’00, 21–30. https://doi.org/10.1145/358916.358947
    19. Darren Gergle, Robert E Kraut, and Susan R Fussell. 2013. Using Visual Information for Grounding and Awareness in Collaborative Tasks Using Visual Inform
    20. Michele Geronazzo, Alberto Bedin, Luca Brayda, Claudio Campus, and Federico Avanzini. 2016. Interactive spatial sonification for non-visual exploration of virtual maps. International Journal of Human-Computer Studies 85: 4–15. https://doi.org/10.1016/J.IJHCS.2015.08.004
    21. Timothy D. Griffiths, Christophe Micheyl, and Tobias Overath. 2012. Auditory Object Analysis. . Springer, New York, NY, 199–223. https://doi.org/10.1007/978-1-4614-2314-0_8
    22. Souta Hidaka, Wataru Teramoto, and Yoichi Sugita. 2015. Spatiotemporal Processing in Crossmodal Interactions for Perception of the External World: A Review. Frontiers in integrative neuroscience 9: 62. https://doi.org/10.3389/fnint.2015.00062
    23. Michael Janner, Karthik Narasimhan, and Regina Barzilay. 2018. Representation Learning for Grounded Spatial Reasoning. Transactions of the Association for Computational Linguistics 6: 49–61. https://doi.org/10.1162/tacl_a_00004
    24. Ute Jekosch. 2005. Assigning Meaning to Sounds — Semiotics in the Context of Product-Sound Design. In Communication Acoustics. Springer-Verlag, Berlin/Heidelberg, 193–221. https://doi.org/10.1007/3-540-27437-5_8
    25. Adam Jones, J. Edward Swan, Gurjot Singh, and Eric Kolstad. 2008. The effects of virtual reality, augmented reality, and motion parallax on egocentric depth perception. In Proceedings - IEEE Virtual Reality, 267–268. https://doi.org/10.1109/VR.2008.4480794
    26. Marcel Adam Just and Patricia A. Carpenter. 1985. Cognitive Coordinate Systems. Accounts of Mental Rotation and Individual Differences in Spatial Ability. Psychological Review 92, 2: 137–172. https://doi.org/10.1037/0033-295X.92.2.137
    27. Andrew J King. 2009. Visual influences on auditory spatial learning. Philosophical transactions of the Royal Society of London. Series B, Biological sciences 364, 1515: 331–9. https://doi.org/10.1098/rstb.2008.0230
    28. Robert Konrad. 2015. What is the vergence-accommodation conflict and how do we fix it? XRDS: Crossroads, The ACM Magazine for Students 22, 1: 52–55. https://doi.org/10.1145/2810048
    29. Maria Kozhevnikov and Andre Garcia. 2011. Visual-Spatial Learning and Training in Collaborative Design in Virtual Environments. In Collaborative Design in Virtual Environments. Springer Netherlands, Dordrecht, 17–26. https://doi.org/10.1007/978-94-007-0605-7_2
    30. Robert E. Kraut, Susan R. Fussell, and Jane Siegel. 2003. Visual Information as a Conversational Resource in Collaborative Physical Tasks. Human–Computer Interaction 18, 1–2: 13–49. https://doi.org/10.1207/S15327051HCI1812_2
    31. Mei-Po Kwan and Jiyeong Lee. 2005. Emergency response after 9/11: the potential of real-time 3D GIS for quick emergency response in micro-spatial environments. Computers, Environment and Urban Systems 29, 2: 93–113. https://doi.org/10.1016/J.COMPENVURBSYS.2003.08.002
    32. Elinda Ai-Lim Lee and Kok Wai Wong. 2014. Learning with desktop virtual reality: Low spatial ability learners are more positively affected. Computers & Education 79: 49–58. https://doi.org/10.1016/J.COMPEDU.2014.07.010
    33. Fumiko Maeda, Ryota Kanai, and Shinsuke Shimojo. 2004. Changing pitch induced visual motion illusion. Current biology : CB 14, 23: R990-1. https://doi.org/10.1016/j.cub.2004.11.018
    34. Lawrence E. Marks. 1974. On Associations of Light and Sound: The Mediation of Brightness, Pitch, and Loudness. The American Journal of Psychology 87, 1/2: 173. https://doi.org/10.2307/1422011
    35. Adrien Merer, Sølvi Ystad, Richard Kronland-Martinet, and Mitsuko Aramaki. 2011. Abstract Sounds and Their Applications in Audio and Perception Research. . Springer, Berlin, Heidelberg, 176–187. https://doi.org/10.1007/978-3-642-23126-1_12
    36. Cecilie Møller, Andreas Højlund, Klaus B. Bærentsen, Niels Chr. Hansen, Joshua C. Skewes, and Peter Vuust. 2018. Visually induced gains in pitch discrimination: Linking audio-visual processing with auditory abilities. Attention, Perception, & Psychophysics 80, 4: 999–1010. https://doi.org/10.3758/s13414-017-1481-8
    37. Cesare V Parise, Katharina Knorre, and Marc O Ernst. 2014. Natural auditory scene statistics shapes human spatial hearing. Proceedings of the National Academy of Sciences of the United States of America 111, 16: 6104–8. https://doi.org/10.1073/pnas.1322705111
    38. Stacey Parrott, Emmanuel Guzman-Martinez, Laura Orte, Marcia Grabowecky, Mark D Huntington, and Satoru Suzuki. 2015. Direction of Auditory Pitch-Change Influences Visual Search for Slope From Graphs. Perception 44, 7: 764–78. https://doi.org/10.1177/0301006615596904
    39. Robert Earl Patterson, Byron J. Pierce, Herbert H. Bell, and Gary Klein. 2010. Implicit Learning, Tacit Knowledge, Expertise Development, and Naturalistic Decision Making. Journal of Cognitive Engineering and Decision Making 4, 4: 289–303. https://doi.org/10.1177/155534341000400403
    40. G. Pingali, A. Opalach, Y. Jean, and I. Carlbom. 2001. Visualization of sports using motion trajectories: providing insights into performance, style, and strategy. In Proceedings Visualization, 2001. VIS ’01., 75–544. https://doi.org/10.1109/VISUAL.2001.964496
    41. C. C. Pratt. 1930. The spatial character of high and low tones. Journal of Experimental Psychology 13, 3: 278–285. https://doi.org/10.1037/h0072651
    42. B. Razavi, W. E. O’Neill, and G. D. Paige. 2007. Auditory Spatial Perception Dynamically Realigns with Changing Eye Position. Journal of Neuroscience 27, 38: 10249–10258. https://doi.org/10.1523/JNEUROSCI.0938-07.2007
    43. Arthur S. Reber. 1989. Implicit learning and tacit knowledge. Journal of Experimental Psychology: General 118, 3: 219–235. https://doi.org/10.1037/0096-3445.118.3.219
    44. Hessam Roodaki, Navid Navab, Abouzar Eslami, Christopher Stapleton, and Nassir Navab. 2017. SonifEye: Sonification of Visual Information Using Physical Modeling Sound Synthesis. IEEE Transactions on Visualization and Computer Graphics 23, 11: 2366–2371. https://doi.org/10.1109/TVCG.2017.2734327
    45. J R Saffran, E K Johnson, R N Aslin, and E L Newport. 1999. Statistical learning of tone sequences by human infants and adults. Cognition 70, 1: 27–52. Retrieved January 14, 2019 from http://www.ncbi.nlm.nih.gov/pubmed/10193055
    46. Alejandro Salgado-Montejo, Fernando Marmolejo-Ramos, Jorge A. Alvarado, Juan Camilo Arboleda, Daniel R. Suarez, and Charles Spence. 2016. Drawing sounds: representing tones and chords spatially. Experimental Brain Research 234, 12: 3509–3522. https://doi.org/10.1007/s00221-016-4747-9
    47. R N Shepard. 1982. Geometrical approximations to the structure of musical pitch. Psychological review 89, 4: 305–33. Retrieved January 14, 2019 from http://www.ncbi.nlm.nih.gov/pubmed/7134331
    48. Jonathan Z Simon. 2015. The encoding of auditory objects in auditory cortex: insights from magnetoencephalography. International journal of psychophysiology : official journal of the International Organization of Psychophysiology 95, 2: 184–90. https://doi.org/10.1016/j.ijpsycho.2014.05.005
    49. Maya Visser, Elizabeth Jefferies, Karl V. Embleton, and Matthew A. Lambon Ralph. 2012. Both the Middle Temporal Gyrus and the Ventral Anterior Temporal Area Are Crucial for Multimodal Semantic Processing: Distortion-corrected fMRI Evidence for a Double Gradient of Information Convergence in the Temporal Lobes. Journal of Cognitive Neuroscience 24, 8: 1766–1778. https://doi.org/10.1162/jocn_a_00244
    50. Ilana B Witten and Eric I Knudsen. 2005. Why seeing is believing: merging auditory and visual worlds. Neuron 48, 3: 489–96. https://doi.org/10.1016/j.neuron.2005.10.020
    51. Seraphina Yong and Hao-Chuan Wang. 2018. Using Spatialized Audio to Improve Human Spatial Knowledge Acquisition in Virtual Reality. In Proceedings of the 23rd International Conference on Intelligent User Interfaces Companion - IUI 18, 1–2. https://doi.org/10.1145/3180308.3180360

    QR CODE