本研究設計互動情境系統，電腦代理人以語音傳遞情境內容，並搭配不同非語言行為表現與受測者溝通。本研究自變項為「電腦代理人非語言行為」，分為無語意組、誤配組、高頻率組與控制組。「無語意」指電腦代理人所表現出之行為與情境溝通內容無關；「誤配」則表示電腦代理人行為與對話內容相反。而高頻率組則展現出大量的非語言行為；控制組幾乎沒有行為表現。實驗應變項包含兩個層面：其一為使用者認知負荷與訊息接收成效，其二為使用者主觀感受、互動滿意度與再互動意願。
本研究的結果證實不同非語言行為表現的效果在電腦代理人的應用上效果類似人際互動並對使用者造成顯著影響。誤配組與高頻率組的受測者認知負荷顯著高於其他組別，代表誤配的電腦代理人非語言行為表現造成使用者注意力分散，對訊息接收成效、使用者主觀感受、互動滿意度與再互動意願有顯著的負面影響。質性資料顯示，電腦代理人表現出高頻率的非語言行為時，除了會增加認知負荷、降低訊息傳遞成效，尚會提升部分使用者主觀感受。

This research explores the external stimuli to human cognition and perception and also discusses the human information reception during the interaction. The experimental system involves embodied computer agents in an interaction scenario. The agents communicate with users via vocal messages and different nonverbal behavior in distinct groups. The independent variable is “nonverbal behavior of computer agents” in four groups: non-semantics, mismatch, hyperactivity, and control. Non-semantics refers to the scenario where the behavior is irrelevant to the communication; mismatch refers to the scenario where the behavior is opposing to the communication. The computer agent in the hyperactivity group exhibits an excessive amount of behaviors. The computer agent in the control group performs no behavior. The dependent variables include “cognitive load, recall and comprehension efficiency, perception, satisfaction and further interaction will”.
The experimental results verify that the effect of nonverbal behaviors on human-computer interaction is similar with interpersonal communication. The mismatch group and the hyperactivity group present a high cognitive load than the non-semantics group and the control group. The semantic-mismatched nonverbal behavior of a computer agent distracts uses’ attention and decreases user perception, satisfaction and the will of further interaction. In particular, according to the qualitative data, nonverbal behavior in high frequency not only leads to cognitive load and decrease the information communication efficiency but also creates positive and lively impression of the computer agent to users. This further improves user satisfaction.

英文部份
Baddeley, A. (2003). Working memory: looking back and looking forward. Nat Rev Neurosci, 4(10), 829-839.
Baylor, A. (2005). The impact of pedagogical agent image on affective outcomes. Paper presented at the International Conference on Intelligent User Interfaces, San Diego, CA.
Baylor, A., & Kim, S. (2008). The Effects of Agent Nonverbal Communication on Procedural and Attitudinal Learning Outcomes Intelligent Virtual Agents (pp. 208-214).
Baylor, A. L., & Kim, S. (2009). Designing nonverbal communication for pedagogical agents: When less is more. Computers in Human Behavior, 25(2), 450-457.
Baylor, A. L., & Ryu, J. (2003). The effects of image and animatio in enhancing pedagogical agent persona, Journal of Educational Computing Research: Baywood Publishing Company, Inc.
Baylor, A. L., & Yanghee, K. (2005). Simulating Instructional Roles through Pedagogical Agents International Journal of Artificial Intelligence in Education, 15(2), 95-115.
Berry, D. C., Butler, L. T., & de Rosis, F. (2005). Evaluating a realistic agent in an advice-giving task. International Journal of Human-Computer Studies, 63(3), 304-327.
Bickmore, T., & Schulman, D. (2006). The comforting presence of relational agents. Paper presented at the CHI '06 extended abstracts on Human factors in computing systems.
Brave, S., Nass, C., & Hutchinson, K. (2005). Computers that are care: investigating the effects of orientation of emotion exhibited by an embodied computer agent. [Article]. International Journal of Human-Computer Studies, 62(2), 161-178.
Buisine, S., & Martin, J.-C. (2007). The effects of speech-gesture cooperation in animated agents' behavior in multimedia presentations. Interacting with Computers, 19(4), 484-493.
Carli, L. L., LaFleur, S. J., & Loeber, C. C. (1995). Nonverbal behavior, gender, and influence. Journal of Personality and Social Psychology, 68(6), 1030-1041.
Cassell, J. (2000a). Embodied conversational agents / edited by Justine Cassell ... [et al.]: MIT Press.
Cassell, J. (2000b). Embodied conversational interface agents. Commun. ACM, 43(4), 70-78.
Cassell, J., McNeill, D., & McCullough, K. (1999). Speech-gesture mismatches: Evidence for one underlying representation of linguistic and nonlinguistic information. Pragmatics and Cognition, 7, 1-34.
Cassell, J., Pelachaud, C., Badler, N., Steedman, M., Achorn, B., Becket, T., et al. (1994). Animated conversation: rule-based generation of facial expression, gesture & spoken intonation for multiple conversational agents. Paper presented at the Proceedings of the 21st annual conference on Computer graphics and interactive techniques.
Cohen, S., & Spacapan, S. (1978). The aftereffects of stress: An attentional interpretation. [10.1007/BF01114531]. Journal of Nonverbal Behavior, 3(1), 43-57.
Creed, C., & Beale, R. (2008). Psychological responses to simulated displays of mismatched emotional expressions. Interacting with Computers, 20(2), 225-239.
Daft, R., & Lengel, R. (1986). Organizational information requirements, media richness and structural design. Management science, 32(5), 554-571.
Ekman, P. (1964). Body position, facial expression, and verbal behavior during interviews. The Journal of Abnormal and Social Psychology, 68(3), 295-301.
Ekman, P. (1993). Facial expression and emotion. American Psychologist, 48(4), 384-392.
Ekman, P. (1999). Basic emotions. Handbook of cognition and emotion, 45-60.
Ekman, P., Freisen, W. V., & Ancoli, S. (1980). Facial signs of emotional experience. Journal of Personality and Social Psychology, 39(6), 1125-1134.
Ekman, P., Friesen, W., & Hager, J. (1978). Facial action coding system: Consulting Psychologists Press Palo Alto, CA.
Ekman, P., & Friesen, W. V. (1975). Unmasking the face. A guide to recognizing emotions from facial clues. New Jersey: Englewood Cliffs.
Fogg, B. J. (2002). Persuasive technology: using computers to change what we think and do. Ubiquity, 2002(December), 2.
Fogg, B. J., & Nass, C. (1997). Silicon sycophants: the effects of computers that flatter. International Journal of Human-Computer Studies, 46(5), 551-561.
Forlizzi, J., Zimmerman, J., Mancuso, V., & Kwak, S. (2007). How interface agents affect interaction between humans and computers. Paper presented at the Proceedings of the 2007 conference on Designing pleasurable products and interfaces.
Friesen, W. V., Ekman, P., & Wallbott, H. (1979). Measuring hand movements. Journal of Nonverbal Behavior, 4(2), 97-112.
Gallaher, P. E. (1992). Individual Differences in Nonverbal Behavior: Dimensions of Style. Journal of Personality and Social Psychology, 63(1), 133-145.
Gifford, R. (1991). Mapping Nonverbal Behavior on the Interpersonal Circle. Journal of Personality and Social Psychology, 61(2), 279-288.
Gong, L. (2008). How social is social responses to computers? The function of the degree of anthropomorphism in computer representations. Comput. Hum. Behav., 24(4), 1494-1509.
Heerink, M., Krose, B., Evers, V., & Wielinga, B. (2008). The influence of social presence on enjoyment and intention to use of a robot and screen agent by elderly users. Paper presented at the Robot and Human Interactive Communication, 2008. RO-MAN 2008. The 17th IEEE International Symposium on.
Heeter, C. (1992). Being there: the subjective experience of presence. Presence: Teleoper. Virtual Environ., 1(2), 262-271.
Helmut, P., & Mitsuru, I. (2005). Human Physiology as a Basis for Designing and Evaluating Affective Communication with Life-Like Characters. IEICE Transactions on Information and Systems, E88-D(11), 2453.
Hickson, M., & Stacks, D. (1985). NVC, nonverbal communication: studies and applications: WC Brown Publishers.
Hinde, R. A. (1972). Non-verbal communication / edited by R. A. Hinde: University Press.
Hollands, J., & Wickens, C. (1999). Engineering Psychology and Human Performance: Prentice Hall.
Hone, K. (2006). Empathic agents to reduce user frustration: The effects of varying agent characteristics. Interacting with Computers, 18(2), 227-245.
Isbister, K., & Nass, C. (2000). Consistency of personality in interactive characters: verbal cues, non-verbal cues, and user characteristics. International Journal of Human-Computer Studies, 53(2), 251-267.
Johnson, D., Gardner, J., & Wiles, J. (2004). Experience as a moderator of the media equation: the impact of flattery and praise. International Journal of Human-Computer Studies, 61(3), 237-258.
Kalyuga, S. (2000). When using sound with a text or picture is not beneficial for learning. Australian Journal of Educational Technology, 16(2), 161-172.
Kalyuga, S., Chandler, P., & Sweller, J. (1999). Managing split-attention and redundancy in multimedia instruction. Applied Cognitive Psychology, 13(4), 351-371.
Katsyri, J., & Sams, M. (2008). The effect of dynamics on identifying basic emotions from synthetic and natural faces. International Journal of Human-Computer Studies, 66(4), 233-242.
Kiesler, S. (2005). Fostering common ground in human-robot interaction. Paper presented at the Robot and Human Interactive Communication, 2005. Roman 2005. IEEE International Workshop on.
Kim, Y., Baylor, A. L., & Shen, E. (2007). Pedagogical agents as learning companions: the impact of agent emotion and gender [Article], Journal of Computer Assisted Learning: Blackwell Publishing Limited.
Knapp, M., & Miller, G. (1985). Handbook of interpersonal communication: Sage Thousand Oaks, CA.
Knapp, M. L., & Hall, J. A. (2002). Nonverbal communication in human interaction / Ma: Wadsworth/Thomson Learning.
Krämer, N., Simons, N., & Kopp, S. (2007). The Effects of an Embodied Conversational Agent’s Nonverbal Behavior on User’s Evaluation and Behavioral Mimicry Intelligent Virtual Agents, 238-251.
Lee, E.-J. (2003). Effects of "gender" of the computer on informational social influence: the moderating role of task type. International Journal of Human-Computer Studies, 58(4), 347-362.
Lee, J., & Marsella, S. (2006). Nonverbal Behavior Generator for Embodied Conversational Agents Intelligent Virtual Agents, 243-255.
Lee, K., Peng, W., Jin, S., & Yan, C. (2006). Can Robots Manifest Personality?: An Empirical Test of Personality Recognition, Social Responses, and Social Presence in Human-Robot Interaction. Journal of Communication, 56(4), 754.
Lester, J. C., & Stone, B. A. (1997). Increasing believability in animated pedagogical agents. Paper presented at the Proceedings of the first international conference on Autonomous agents.
Lindgaard, G. (2007). Aesthetics, visual appeal, usability, and user satisfaction: What do the user’s eyes tell the user’s brain. Australian journal of emerging technologies and society, 5(1), 2007, 1-14.
Mao, X., Li, Z., & Xue, Y. (2009). Emotional gaze behavior generation in human-agent interaction. Paper presented at the Proceedings of the 27th international conference extended abstracts on Human factors in computing systems.
Marynelle, D.-C. (1988). The Effects of Reference Librarians' Nonverbal Communications on the Patrons' Perceptions of the Library, Librarians, and Themselves. [Journal Articles; Reports - Research; Tests/Questionnaires]. Library and Information Science Research, 10(4), 389-400.
Mayer, R., Heiser, J., & Lonn, S. (2001). Cognitive constraints on multimedia learning: When presenting more material results in less understanding. Journal of Educational Psychology, 93(1), 187-198.
McNeill, D. (1992). Hand and mind: University of Chicago Press.
Mori, J., Prendinger, H., & Ishizuka, M. (2003). Evaluation of an embodied conversational agent with affective behavior.
Nass, C., Moon, Y., Fogg, B. J., Reeves, B., & Dryer, C. (1995). Can computer personalities be human personalities? Paper presented at the Conference companion on Human factors in computing systems.
Nijholt, A. (2002). Issues in multimodal nonverbal communication and emotion in embodied （conversational） agents. Paper presented at the 6th World Multiconference on Systemics, Cybernetics and Informatics.
Nijholt, A. (2003). Disappearing computers, social actors and embodied agents. Paper presented at the Cyberworlds, 2003. Proceedings. 2003 International Conference on.
Nijholt, A. (2004). Where computers disappear, virtual humans appear. Computers & Graphics, 28(4), 467-476.
Nowak, K., & Biocca, F. (2003). The effect of the agency and anthropomorphism on users' sense of telepresence, copresence, and social presence in virtual environments. Presence: Teleoperators & Virtual Environments, 12(5), 481-494.
Paas, F., & Van Merriënboer, J. (1994). Instructional control of cognitive load in the training of complex cognitive tasks. Educational Psychology Review, 6(4), 351-371.
Patterson, M. (1990). Functions of non-verbal behavior in social interaction. Handbook of language and social psychology, 101-120.
Pelachaud, C. (2009). Studies on gesture expressivity for a virtual agent. Speech Communication, 51(7), 630-639.
Perera, N., Kennedy, G., & Pearce, J. (2008). Are you bored?: Maybe an interface agent can help! Paper presented at the Proceedings of the 20th Australasian Conference on Computer-Human Interaction: Designing for Habitus and Habitat.
Pratt, J. A., Hauser, K., Ugray, Z., & Patterson, O. (2007). Looking at human-computer interface design: Effects of ethnicity in computer agents. Interacting with Computers, 19(4), 512-523.
Prendinger, H., Ma, C., Mori, J., & Ishizuka, M. (2005). Does non-verbal behavior of an embodied agent matter? Paper presented at the Active Media Technology, 2005. (AMT 2005). Proceedings of the 2005 International Conference on.
Prendinger, H., Mori, J., & Ishizuka, M. (2005). Using human physiology to evaluate subtle expressivity of a virtual quizmaster in a mathematical game. International Journal of Human-Computer Studies, 62(2), 231-245.
Reeves, B., & Nass, C. I. (1998). The media equation : how people treat computers, and places / Byron Reeves & Clifford Nass: CSLI Publications ; Cambridge University Press.
Rosenberg-Kima, R. B., Baylor, A. L., Plant, E. A., & Doerr, C. E. (2008). Interface agents as social models for female students: The effects of agent visual presence and appearance on female students' attitudes and beliefs. Computers in Human Behavior, 24(6), 2741-2756.
Rosenberg, B. G., & Langer, J. (1965). A study of postural-gestural communication. Journal of Personality and Social Psychology, 2(4), 593-597.
Russell, J. (1997). Reading emotions from and into faces: Resurrecting a dimensional-contextual perspective. The psychology of facial expression, 295–320.
Sebeok, T. A., Umiker-Sebeok, D. J., & Kendon, A. (1981). Nonverbal communication, interaction, and gesture editors Thomas A. Sebeok and Jean Umiker-Sebeok ; volum: Mouton Publishers.
Shearer, J., Olivier, P., Heslop, P., & de Boni, M. (2006). Requirements of non-verbal communication in believable synthetic agents. Paper presented at the Proceedings of the AISB Symposium on Narrative AI and Games.
Singer, J. (1980). The power and limitations of television: A cognitive-affective analysis. The entertainment functions of television, 31-65.
Sweller, J. (1994). Cognitive load theory, learning difficulty, and instructional design. Learning and Instruction, 4(4), 295-312.
Sweller, J., van Merrienboer, J., & Paas, F. (1998). Cognitive Architecture and Instructional Design. Educational Psychology Review, 10(3), 251-296.
Tourangeau, R., Couper, M. P., & Steiger, D. M. (2003). Humanizing self-administered surveys: experiments on social presence in web and IVR surveys. Computers in Human Behavior, 19(1), 1-24.
van Vugt, H. C., Hoorn, J. F., Konijn, E. A., & de Bie Dimitriadou, A. (2006). Affective affordances: Improving interface character engagement through interaction. International Journal of Human-Computer Studies, 64(9), 874-888.
Vargas, M. (1986). Louder than words: An introduction to nonverbal communication: Wiley-Blackwell.
Vargas, M. F. (1986). Louder than words : an introduction to nonverbal communication / Marjorie Fink Vargas: Iowa State University Press.
Walker, J. H., Sproull, L., & Subramani, R. (1994). Using a human face in an interface. Paper presented at the Proceedings of the SIGCHI conference on Human factors in computing systems: celebrating interdependence.
 

中文部份
Argyle, M. （1995）。人際行為心理學。台北市：巨流。
Argyle, M. （1998）。肢體溝通。台北市：巨流。
方永安（2008）。電腦介面角色之外表美醜、個性內外向對不同性別使用者在角色吸引、社交表徵及滿意度上之影響。國立清華大學資訊系統與應用研究所碩士論文，未出版，新竹。
李宗益（2007）。設計能使學習者感受關懷之可情緒互動虛擬人物。國立中央大學資訊工程研究所碩士論文，未出版，桃園。
徐易稜（2000）。多媒體呈現方式對學習者認知負荷與學習成效之影響研究。國立中央大學資訊管理研究所碩士論文，未出版，桃園。
張春興（1989）。教育心理學。台北市：東華書局。
陳佩真（2008）。電視字幕對語言理解的影響 : 以「形系」和「音系」文字的差異為切入點。臺北市：秀威資訊科技。
詹竹莘（1997）。表演技術與表演教程：書林。
劉說芳（1992）。電視字幕對視覺及劇情理解度影響之研究。國立成功大學工業設計研究所碩士論文，未出版，臺南。
謝財旺（2005）。適性化學習環境中學習者訊息處理能力與內容媒體型態的適配性對學習成效的影響。國立中山大學資訊管理研究所碩士論文，未出版，高雄。