異質性資料在人類行為其信息交互過程之探討:疼痛程度評估和多媒體暴力檢測

簡易檢索 / 詳目顯示

回結果列表

研究生：	洪彗庭 Hong, Hui-Ting
論文名稱：	異質性資料在人類行為其信息交互過程之探討:疼痛程度評估和多媒體暴力檢測 Investigate Communication Process of Human Behavior from Heterogeneous Data: Pain Level Assessment and Multimedia Violence Prediction
指導教授：	李祈均 Lee, Chi-Chun
口試委員:	曹昱 Tsao, Yu 賴穎暉 Lai, Ying-Hui 陳冠宇 Chen, Kuan-Yu
學位類別：	碩士 Master
系所名稱：	電機資訊學院 - 電機工程學系 Department of Electrical Engineering
論文出版年：	2020
畢業學年度：	108
語文別：	英文
論文頁數：	74
中文關鍵詞：	人類行為訊號處理、疼痛程度辨識、急診檢傷分類、傳媒暴力程度預測
外文關鍵詞：	Behavioral signal processing, pain level, triage, violence prediction
相關次數：	點閱：2 下載：0
分享至:	分享至facebook 分享至twitter

查詢本校圖書館目錄查詢臺灣博碩士論文知識加值系統勘誤回報

人類在現今依存的世界裡透過多種不同的方式進行溝通與交流，從概念的生成、送訊者的編碼、傳達的媒介一直到收訊者的解碼，是一連串複雜而多層次的程序，其中亦可能因噪音(訊息之扭曲、干擾)或是回饋的產生，而為其增添更多未知之可能性。本研究基於人類行為本質的複雜性，著重探討編碼與解碼的過程以及行為者與現象發生之關聯性，並實際驗證於兩個任務:ㄧ)疼痛程度評估以及二)多媒體暴力檢測。在醫療臨床疼痛程度診斷的應用上，部分臨床紀錄之資訊在工程研究中尚未列入考慮，本次運用多模態機器學習之模型設計技術，提出軟層排序任務特定(Task Specific Encoder with Soft Layer Order- ing, TSEN-SLO)架構，透過參閱疼痛部位的臨床紀錄，提升在診斷疼痛程度的準確率，而為理解各個特徵在不同族群的重要程度及其影響，該研究亦透過多變量變異數分析，了解語音特徵在多個獨立變項(年齡、性別、疼痛程度、疼痛部位)影響下的顯著性差異。而在多媒體內容檢索的應用上，過去研究多著重彌合語義(semantic)資訊、擷取影視內容之摘要片段等等，而較少探討創作者與受眾之間的關係脈絡，本次使用英國影集—黑鏡之觀者與劇本撰寫者之辭彙特徵，運用外類別(extra-genre multimodal embedding)模型架構，初步驗證受眾回饋具有評估多媒體暴力程度的可行性。

Living world nurtures diverse ways on how people communicate with one another. Its concepts could be structured from encoding, message, medium to decoding. This communication process is a series of complicate and multilayered procedures which also include the generation of noise (distortion and interference) or feedback. Base on the multidimensionality of human behavior, this research focused on investigating the pro- cess from encoding to decoding, and the relationship between the human beings and their corresponding phenomenon. We verified the results on two tasks: 1) automatic pain level recognition and 2) multimedia violence prediction. In the first task, triage pain assessment, clinical parameter like pain-site, is yet not considered in previous engineer studies. We utilized multi-modal machine learning technique and proposed the Task Specific Encoder with Soft Layer Ordering (TSEN-SLO) structure with the auxiliary information from pain-site and demonstrates improvement in the pain-level recognition. Moreover, to further reveal the variability of voice quality conditioned on clinical parameters (age, gender, pain-level, pain-site), the multivariate analysis of variance was conducted to understand the significant differences of acoustic features with respect to multiple independent variables. In the second task, multimedia violence pre- diction, previous lines of work aim to bridge the semantic gap and cut down the content of multimedia to its essential parts. We, however, conducted the experiment on the British TV series – Black Mirror, revealing the relationship between screenplay writers and audiences. The extra-genre multimodal embedding structure is proposed in this task and preliminarily verify the feasibility of assessing multimedia violence extent by considering audience feedback.

摘要································································································ I ABSTRACT ···················································································· II
誌謝······························································································ IV
CHAPTER 1 INTRODUCTION·························································3
CHAPTER 2 TASK1: AUTOMATIC PAIN LEVEL RECOGNITION ········6
1 Introduction·················································································6
2 Database ·····················································································9
2.1 Data Acquisition········································································ 11
2.2 Processed Data········································································ 13
3 Research Methodology·································································· 14
3.1 Discriminative Features Extractions ········································· 15
3.1.1 Visual Features ································································ 16
3.1.2 Acoustic Features ··························································· 20
3.2 Session-Level Behavior Encoding ··········································· 22
3.2.1 Gaussian Mixture Models Fisher Vector Encoding (GMMs-FV) ······ 22 2.3.2.2 Functional Encoding······································································ 25 2.3.3 Model Architecture ·········································································· 25 2.3.3.1 Multi-task Learning (MTL)························································· 26 2.3.3.2 Soft Layer Ordering [63] ·························································· 28 2.3.3.3 Task Specific Encoder with Soft Layer Ordering (TSEN- SLO)························ 31
4 Experiments and Results ······························································· 33
4.1 Experimental Settings······························································ 33
4.2 Recognition Results································································· 36
5 Statistical Analysis······································································· 38
5.1 Variability of Facial Expressions ··············································· 39
5.2 Variability of Acoustic Expressions ·········································· 41
5.2.1 Clinical Parameters: Age, Gender, Pain-Site························ 42
5.2.2 Voice Quality······································································ 43
5.2.3 MANOVA Settings······························································ 45
5.2.4 Result (I): Main / Interaction Effect ···································· 47
5.2.5 Result (II): Univariate Analysis and Pairwise Comparison ···· 51
CHAPTER 3 TASK2: MULTIMEDIA VIOLENCE PREDICTION ·········· 56
1 Introduction··············································································· 56
2 Database ··················································································· 56
3 Research Methodology·································································· 59
3.1 Features Extractions ································································ 59
3.1.1 Linguistic Inquiry and Word Count (LIWC) ························· 59
3.1.2 BERT ················································································ 62
3.2 Model Architecture ·································································· 63
4 Experiments and Results ···························································· 64 CHAPTER 4 CONCLUSION ·························································· 67 REFERENCE ················································································· 69
                                

[1] Popescu, Gabriela. "Human behavior, from psychology to a transdisciplinary insight." Procedia- Social and Behavioral Sciences 128 (2014): 442-446.
[2] Giddens, Anthony, et al. Introduction to sociology. New York: Norton, 1991.
[3] Rose, Arnold M. Human behavior and social processes: An interactionist approach. Routledge,
2013.
[4] Segall, Marshall H., et al. Human behavior in global perspective: An introduction to cross-cul-
tural psychology. Pergamon Press, 1990.
[5] Zimbardo, Philip G., and Floyd L. Ruch. "Psychology and life." (1975).
[6] Rosenzweig, Mark R., S. Marc Breedlove, and Arnold L. Leiman. Biological psychology: An in-
troduction to behavioral, cognitive, and clinical neuroscience. Sinauer Associates, 2002.
[7] Alland, Alex. Evolution and human behaviour: an introduction to Darwinian anthropology.
Routledge, 2012.
[8] Cronk, Lee. "Human behavioral ecology." Annual Review of Anthropology 20.1 (1991): 25-53.
[9] Lepri, Bruno, et al. "Human behavior understanding for inducing behavioral change: Social and
theoretical aspects." International Joint Conference on Ambient Intelligence. Springer, Berlin,
Heidelberg, 2011.
[10] Deleanu,I.,(1983),Biologieşidrept,DaciaPublishing-House,Cluj-Napoca,p.53
[11] Gastil,RaymondD."Thedeterminantsofhumanbehavior."AmericanAnthropologist63.6
(1961): 1281-1291.
[12] Narayanan,Shrikanth,andPanayiotisG.Georgiou."Behavioralsignalprocessing:Derivinghu-
man behavioral informatics from speech and language." Proceedings of the IEEE 101.5 (2013):
1203-1233.
[13] Lee,Chi-ChunJeremy.ModelingHumanBehaviorsinPsychologyUsingEngineeringMethods.
River Publishers, 2014.
[14] Lee,ChulMin,andShrikanthS.Narayanan."Towarddetectingemotionsinspokendia-
logs." IEEE transactions on speech and audio processing 13.2 (2005): 293-303.
[15] Narayanan,Shrikanth,andPanayiotisG.Georgiou."Behavioralsignalprocessing:Derivinghu-
man behavioral informatics from speech and language." Proceedings of the IEEE 101.5 (2013):
1203-1233.
[16] Bieri,Daiva,etal."TheFacesPainScalefortheself-assessmentoftheseverityofpainexperi-
enced by children: development, initial validation, and preliminary investigation for ratio scale
properties." Pain 41.2 (1990): 139-150.
[17] Hicks,CarrieL.,etal."TheFacesPainScale–Revised:towardacommonmetricinpediatricpain
measurement." Pain 93.2 (2001): 173-183.
[18] Carlsson,AnnaMaria."Assessmentofchronicpain.I.Aspectsofthereliabilityandvalidityof the visual analogue scale." Pain 16.1 (1983): 87-101.
[19] Bond,M.R.,andI.Pilowsky."Subjectiveassessmentofpainanditsrelationshiptothe
A., Alison R. Snyder, and Brian G. Leggin. "Responsiveness of the numeric pain rating scale in pa-
tients with shoulder pain and the effect of surgical status." Journal of sport rehabilitation 20.1
(2011): 115-128.
[23] Farrar,JohnT.,etal."Clinicalimportanceofchangesinchronicpainintensitymeasuredonan
11-point administration of analgesics in patients with advanced cancer." Journal of psychoso-
matic research (1966).
[20] Shields,B.J.,etal."Predictorsofachild'sabilitytouseavisualanaloguescale."Child:care,
health and development 29.4 (2003): 281-290.
[21] Bijur,PollyE.,ClarkeT.Latimer,andE.JohnGallagher."Validationofaverballyadministered
numerical rating scale of acute pain for use in the emergency department." Academic Emergency
Medicine 10.4 (2003): 390-392.
[22] Michener,Lorinumericalpainratingscale."Pain94.2(2001):149-158.
[24] Weideman,AnnMarie,etal."Meta-analysisoftheage-dependentefficacyofmultiplesclerosis treatments." Frontiers in neurology 8 (2017): 577.
[25] Edwards,RobertR.,etal."Paintoleranceasapredictorofoutcomefollowingmultidisciplinary treatment for chronic pain: differential effects as a function of sex." Pain 106.3 (2003): 419-426.
[26] Prkachin,KennethM."Theconsistencyoffacialexpressionsofpain."PaulEkmanErikaL.Ros- enberg (1997): 198.
[27] Kaltwang,Sebastian,OgnjenRudovic,andMajaPantic."Continuouspainintensityestimation from facial expressions." International Symposium on Visual Computing. Springer, Berlin, Hei- delberg, 2012.
[28] Ashraf,AhmedBilal,etal."Thepainfulface–painexpressionrecognitionusingactiveappear- ance models." Image and vision computing 27.12 (2009): 1788-1796.
[29] Olugbade,TemitayoA.,etal."Howcanaffectbedetectedandrepresentedintechnologicalsup- port for physical rehabilitation?." ACM Transactions on Computer-Human Interaction (TO- CHI) 26.1 (2019): 1-29.
[30] Rivas,JesúsJoel,etal."Unobtrusiveinferenceofaffectivestatesinvirtualrehabilitationfrom upper limb motions: A feasibility study." IEEE transactions on affective computing (2018).
[31] Tsai,Fu-Sheng,etal."EmbeddingstackedbottleneckvocalfeaturesinaLSTMarchitecturefor automatic pain level classification during emergency triage." 2017 Seventh International Confer- ence on Affective Computing and Intelligent Interaction (ACII). IEEE, 2017.
[32] Li,Jeng-Lin,etal."LearningConditionalAcousticLatentRepresentationwithGenderandAge Attributes for Automatic Pain Level Recognition." Interspeech. 2018.
[33] Cervero,Fernando."Visceralversussomaticpain:similaritiesanddifferences."DigestiveDis- eases 27.Suppl. 1 (2009): 3-10.
[34] Kischner,S.,andR.C.McMyne."Dermatomesanatomy."URL:http://emedicine.medscape. com/article/1878388-overview[accessed June 1, 2017] (2015).
[35] Mantyh,PatrickW."Theneurobiologyofskeletalpain."EuropeanjournalofNeuroscience39.3 (2014): 508-519.
[36] AndrewRice,Clinicalpainmanagement,HodderArnold,London,2008.
[37] Tsai,Fu-Sheng,etal."TowardDevelopmentandEvaluationofPainLevel-RatingScalefor
Emergency Triage based on Vocal Characteristics and Facial Expressions." Interspeech. 2016.
[38] Eriksson,Kerstin,etal."Numericratingscale:patients'perceptionsofitsuseinpostoperative
pain assessments." Applied nursing research 27.1 (2014): 41-46.
[39] Perronnin,Florent,JorgeSánchez,andThomasMensink."Improvingthefisherkernelforlarge-
scale image classification." European conference on computer vision. Springer, Berlin, Heidel-
berg, 2010.
[40] Prkachin,KennethM.,andKennethD.Craig."Expressingpain:thecommunicationandinterpre-
tation of facial pain signals." Journal of Nonverbal Behavior 19.4 (1995): 191-205.
[41] Craig,KennethD."ThefacialexpressionofpainBetterthanathousandwords?."APSJour-
nal 1.3 (1992): 153-162.
[42] Fuller,BarbaraF."Acousticdiscriminationofthreetypesofinfantcries."NursingRe-
search (1991).
[43] Oshrat,Yaniv,etal."Speechprosodyasabiosignalforphysicalpaindetection."ConfProc8th
Speech Prosody. 2016.
[44] Ren,Zhao,etal."Evaluationofthepainlevelfromspeech:Introducinganovelpaindatabaseand
benchmarks." Speech Communication; 13th ITG-Symposium. VDE, 2018.
[45] Baltrusaitis,Tadas,PeterRobinson,andLouis-PhilippeMorency."Constrainedlocalneuralfields
for robust facial landmark detection in the wild." Proceedings of the IEEE international confer-
ence on computer vision workshops. 2013.
[46] Xing,Junliang,etal."Towardsmulti-viewandpartially-occludedfacealignment."Proceedings
of the IEEE Conference on Computer Vision and Pattern Recognition. 2014.
[47] Peng,Jian,LiefengBo,andJinboXu."Conditionalneuralfields."Advancesinneuralinfor-
mation processing systems. 2009.
[48] Hjortsjö,Carl-Herman.Man'sfaceandmimiclanguage.Studenlitteratur,1969.
[49] Ekman,Paul,WallaceV.Freisen,andSoniaAncoli."Facialsignsofemotionalexperience."Jour-
nal of personality and social psychology 39.6 (1980): 1125.
[50] McIntyre,Gordon,etal."Anapproachforautomaticallymeasuringfacialactivityindepressed subjects." 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops. IEEE, 2009.
[51] Cohn,JeffreyF.,etal."Detectingdepressionfromfacialactionsandvocalprosody."20093rd International Conference on Affective Computing and Intelligent Interaction and Workshops. IEEE, 2009.
[52] LeResche,Linda."Facialexpressioninpain:astudyofcandidphotographs."JournalofNonver- bal Behavior 7.1 (1982): 46-56.
[53] Lucey,Patrick,etal."Automaticallydetectingpainusingfacialactions."20093rdInternational Conference on Affective Computing and Intelligent Interaction and Workshops. IEEE, 2009.
[54] Rathee,Neeru,andDineshGanotra."Anovelapproachforpainintensitydetectionbasedonfa- cial feature deformations." Journal of Visual Communication and Image Representation 33 (2015): 247-254.
[55] https://www.cs.cmu.edu/~face/facs.htm
[56] 徐雅玲. "利用多模態模型混合CNN 和LSTM 影音特徵以自動化偵測急診病患疼痛程度." (2018): 1-44.
[57] Schuller,Björn,etal."Theinterspeech2012speakertraitchallenge."ThirteenthAnnualConfer- ence of the International Speech Communication Association. 2012.
[58] Reynolds,DouglasA."GaussianMixtureModels."Encyclopediaofbiometrics741(2009).
[59] Moon,ToddK."Theexpectation-maximizationalgorithm."IEEESignalprocessingmaga-
zine 13.6 (1996): 47-60.
[60] Kaya,Heysem,AlexeyA.Karpov,andAlbertAliSalah."Fishervectorswithcascadednormali-
zation for paralinguistic analysis." Sixteenth Annual Conference of the International Speech
Communication Association. 2015.
[61] Zhang,Yu,andQiangYang."Asurveyonmulti-tasklearning."arXivpreprint
arXiv:1707.08114 (2017).
[62] Ruder,Sebastian."Anoverviewofmulti-tasklearningindeepneuralnetworks."arXivpreprint
arXiv:1706.05098 (2017).
[63] Meyerson,Elliot,andRistoMiikkulainen."Beyondsharedhierarchies:Deepmultitasklearning
through soft layer ordering." arXiv preprint arXiv:1711.00108 (2017).
[64] Nair,Vinod,andGeoffreyE.Hinton."Rectifiedlinearunitsimproverestrictedboltzmannma-
chines." Proceedings of the 27th international conference on machine learning (ICML-10). 2010.
[65] Sherman,R.O.G.E.R."Abdominalpain."(1990).
[66] Sluka,K.A."Stimulationofdeepsomatictissuewithcapsaicinproduceslong-lastingmechanical
allodynia and heat hypoalgesia that depends on early activation of the cAMP pathway." Journal of Neuroscience 22.13 (2002): 5687-5693.
[67] Taverner,Deryck."Musclespasmasacauseofsomaticpain."Annalsoftherheumaticdis- eases 13.4 (1954): 331.
[68] Sikandar,Shafaq,andAnthonyH.Dickenson."Visceralpain–theinsandouts,theupsand downs." Current opinion in supportive and palliative care 6.1 (2012): 17.
[69] https://bodyinmind.org/pain-communication-body-posture/
[70] Walsh,Joseph,ChristopherEccleston,andEdmundKeogh."Paincommunicationthroughbody posture: The development and validation of a stimulus set." PAIN® 155.11 (2014): 2282-2290.
[71] Blondell,RichardD.,MohammadrezaAzadfard,andAngelaM.Wisniewski."Pharmacologic therapy for acute pain." American family physician 87.11 (2013): 766-772.
[72] Nicholson,Bruce."Differentialdiagnosis:nociceptiveandneuropathicpain."TheAmericanjour- nal of managed care 12.9 Suppl (2006): S256-62.
[73] Teixeira,JoãoPaulo,CarlaOliveira,andCarlaLopes."Vocalacousticanalysis-jitter,shimmer and HNR parameters." (2013).
[74] Fusaroli,Riccardo,etal."IsvoiceamarkerforAutismspectrumdisorder?Asystematicreview and meta- analysis." Autism Research 10.3 (2017): 384-407.
[75] Vaiciukynas,Evaldas,etal."DetectingParkinson’sdiseasefromsustainedphonationandspeech signals." PloS one 12.10 (2017).
[76] Silva,DarcioG.,LuísC.Oliveira,andMarioAndrea."Jitterestimationalgorithmsfordetection of pathological voices." EURASIP Journal on Advances in Signal Processing 2009 (2009): 1-9.
[77] Biemans,Monique.Gendervariationinvoicequality.NetherlandsGraduateSchoolofLinguis-
tics, 2000.
[78] Lortie,CatherineL.,etal."Effectsofageontheamplitude,frequencyandperceivedqualityof
voice." Age 37.6 (2015): 117.
[79] Mendoza,Elvira,etal."Differencesinvoicequalitybetweenmenandwomen:Useofthelong-
term average spectrum (LTAS)." Journal of voice 10.1 (1996): 59-66.
[80] Weninger,Felix,etal."Ontheacousticsofemotioninaudio:whatspeech,music,andsound
have in common." Frontiers in psychology 4 (2013): 292.
[81] Teixeira,JoãoPaulo,andPaulaOdeteFernandes."Jitter,ShimmerandHNRclassificationwithin
gender, tones and vowels in healthy voices." Procedia technology 16 (2014): 1228-1237.
[82] Devlin,Jacob,etal."Bert:Pre-trainingofdeepbidirectionaltransformersforlanguageunder-
standing." arXiv preprint arXiv:1810.04805 (2018).
[83] Ko,Ming-Ya,Jeng-LinLi,andChi-ChunLee."LearningMinimalExtra-genreMultimodalEm-
bedding from Trailer Content and Reactor Expressions for Box Office Prediction." 2019 IEEE
International Conference on Multimedia and Expo (ICME). IEEE, 2019.
[84] Zhang,Ying,andHuchuanLu."Deepcross-modalprojectionlearningforimage-textmatch-
ing." Proceedings of the European Conference on Computer Vision (ECCV). 2018.
[85] Sudhakaran,Swathikiran,andOswaldLanz."Learningtodetectviolentvideosusingconvolu- tional long short-term memory." 2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS). IEEE, 2017.
[86] AlfredLKroeberandTalcottParsons.1958.Theconceptsofcultureandofsocialsystem.Americansocio- logical review 23, 5 (1958), 582–583.

簡易檢索 / 詳目顯示

相關論文