阿茲海默症為一種慢性神經退化疾病，好發於六十五歲以上的年長者，短期記憶喪失是其初期常見的症狀，隨著時間進而逐漸出現認知與運動功能退化的現象，如步態異常、平衡問題、閱讀困難等。目前的醫療診斷大多由醫師藉由神經心理學評估或過往病史進行診斷，針對病人的注意力、記憶、空間等功能的退化評估該是否與腦病變有所關連，常用的評估檢測技術如簡短智能測驗(Mini-Mental State Examination, MMSE)與臨床失智量表(Clinical Dementia Rating, CDR)，亦或是藉由電腦斷層掃瞄與核磁共振造影觀察腦部變化，然而這些臨床診斷方法昂貴耗時且需要專業醫療人員協助診斷。此外，許多潛在的阿茲海默症患者將這些認知或行動能力下降的症狀視為老化的象徵，往往直到這些症狀嚴重影響日常生活時才尋求臨床評估，為了及早診斷發現，我們提出了基於電腦視覺的方法，分別藉由Time Up and Go (TUG)測試以及圖片描述任務分析受試者的步態運動與臉部頭部變化以評估病人是否罹患阿茲海默症，並使用多個指標：準確率(Accuracy)、敏感性(Sensitivity)、特異性(Specificity)、精確率(Precision)、 F1-score與AUC (Area under receiver operating
characteristic (ROC) curve)評估此作法的有效性。

在這項研究中，我們收集99位受試者的實驗資料，分別從步態影片中提取受試者臀部、膝蓋、腳踝的位置，並細分為步行(Walking)、起立坐下(Sit-Stand)與轉身(Turning)三個子任務，圖片描述(Describing)子任務則從圖片描述的影片中提取其左眼與右眼的位置。進而在每個子任務中利用提取的二維軌跡資料訓練二元分類機器學習模型，分析阿茲海默症患者與非阿茲海默症患者之間的差異。我們的方法同時分析了步態與臉部特徵的綜合結果，並達到準確率93.21\%、敏感性87.60\%、特異性95.71\%、精確率90.47\%、F1-score為0.89的分類能力表現，這凸顯了使用基於影像和機器學習模型進行阿茲海默症檢測的潛力，並為醫療專業人員提供了額外的參考資訊。此外，我們應用模型可解釋方法：Gradient-weighted Class Activation Mapping (Grad-CAM)與SHapley Additive exPlanations (SHAP)計算特徵的重要性進而解釋模型進行預測時的背後原因，使我們能夠更直觀地了解模型隨時間而變化的決策，並提高對於模型預測結果的可信度。

Alzheimer's disease (AD), an irreversible progressive neurodegenerative disorder, is characterized by several cognitive and functional symptoms such as memory loss, balance problems, and difficulty reading. Current diagnosis relies on a combination of medical evaluations and tests, such as brain imaging (i.e., Magnetic Resonance Imaging (MRI)) and laboratory tests (i.e., blood tests) which are time-consuming and expensive. Moreover, many AD patients treat these kinds of lower cognitive or functional abilities as the symptoms of aging and do not seek clinical assessments till these symptoms affect their daily lives. To address this, we develop a computer vision-based method to detect AD and Non-AD from both walking and head movement assessment, which are the Time Up and Go test (TUG) and Cookie Theft (CT) picture description task respectively. We then assessed the efficacy of our approach with multiple metrics, including accuracy, sensitivity, specificity, precision, and F1-score.

In this work, we collected body joint position data for the gait analysis when they performed the TUG test in front of the camera. We pre-processed this signal data and separated it into the Walking (W), Sit-Stand (S), and Turning (T) subtasks for the following analysis. For the picture description task, named Describing (D) subtask, we extracted left and right eye position data to evaluate the difference between AD and Non-AD subjects. In each subtask, we utilized a 2D convolutional neural network (CNN) or support vector machine (SVM) classifier to classify the two groups. With comprehensive analysis of both gait and facial aspects, our method achieved the performance with an average accuracy of 93.21\%, sensitivity of 87.60\%, specificity of 95.71\%, precision of 90.47\%, and F1-score of 0.89. This highlights the potential of using video-based analysis and machine-learning methods for AD detection and providing additional references to medical professionals.

To understand the decision-making process intuitively, we applied the model interpretable method, Grad-CAM, on multivariate time-series data to estimate the importance of features on the time dimension instead of the spatial dimension. In addition, we utilized SHAP to measure the importance of each feature in order to understand which features have a greater impact on the model's predictions. These model interpretation approaches allowed us to gain insights into the strengths and weaknesses of our model and improve the credibility of predicted results.

[1] CG Lyketsos, O Lopez, B Jones, AL Fitzpatrick, J Breitner, and S DeKosky. Prevalence of neuropsychiatric symptoms in dementia and mild cognitive impairment: results from the cardiovascular health study. Thorax, 288:1475–83, 2002.

[2] ME Peters, S Schwartz, D Han, PV Rabins, M Steinberg, JT Tschanz, and CG Lyketsos. Neuropsychiatric symptoms as predictors of progression to severe Alzheimer’s dementia and death: the cache county dementia progression study. Am J Psychiatry, 172:460–5, 2015.

[3] E Nichols, CEI Szoeke, SE Vollset, et al. Global, regional, and national burden of alzheimer’s disease and other dementias, 1990-2016: a systematic analysis for the global burden of disease study 2016. Lancet Neurol, 18:88–106, 2018.

[4] E Nichols, JD Steinmetz, SE Vollset, et al. Estimation of the global prevalence of dementia in 2019 and forecasted prevalence in 2050: an analysis for the global burden of disease study 2019. Lancet Public Health, 7:e105–e125, 2022.

[5] J Rasmussen and H Langerman. Alzheimer’s disease – why we need early diagnosis. Degenerative Neurological and Neuromuscular Disease, 9:123–130, 2019.

[6] JT Becker, F Boiler, OL Lopez, et al. The natural history of alzheimer’s disease: description of study cohort and accuracy of diagnosis. arch neurol. Archives of Neurology, 51(6):585–594, 1994.

[7] ST O’keeffe, H Kazeem, RM Philpott, JR Playfer, M Gosney, and M Lye. Gait disturbance in alzheimer’s disease: A clinical study. Age Ageing, 25:313–316, 1996.

[8] J Dumurgier, F Artaud, C Touraine, et al. Gait speed and decline in gait speed as predictors of incident dementia. The Journals of Gerontology, 72(5):655– 661, 2017.

[9] O Beauchet, G Allali, G Berrut, C Hommet, V Dubost, and F Assal. Gait analysis in demented subjects: Interests and perspectives. Neuropsychiatric Disease and Treatment, 4(1):155, 2008.

[10] LM Allan, CG Ballard, DJ Burn, and RA Kenny. Prevalence and severity of gait disorders in alzheimer’s and non-alzheimer’s dementias. Journal of the American Geriatrics Society, 53(10):1681–1687, 2005.

[11] VS Thomas, EV Vandenberg, and JF Potter. Non-neurological factors are implicated in impairments in gait and mobility among patients in a clinical dementia referral population. International journal of geriatric psychiatry, 17(2):128–133, 2002.

[12] X Xu, RW McGorry, LS Chou, J Lin, and C Chang. Accuracy of the microsoft kinectTM for measuring gait parameters during treadmill walking. Gait Posture, 39:1062–1068, 2015.

[13] J Stenum, C Rossi, and RT Roemmich. Two-dimensional video-based analysis of human gait using pose estimation. PLOS Computational Biology, 17:e1008935, 2021.

[14] M Seifallahi, AH Mehraban, JE Galvin, et al. Alzheimer’s disease detection using comprehensive analysis of timed up and go test via kinect v.2 camera and machine learning. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 30:1589–1600, 2022.

[15] A Ladas, C Frantzidis, P Bamidis, and AB Vivas. Eye blink rate as a biological marker of mild cognitive impairment. International Journal of Psychophysiology, 93(1):12–16, 2014.

[16] T Fukui, T Yamazaki, and R Kinno. Can the “head-turning sign” be a clinical marker of alzheimer’s disease. Dementia and Geriatric Cognitive Disorders Extra, 1(1):310–317, 2011.

[17] J Dura ̃es, M Ta ́buas-Pereira, and R Arau ́jo. The head turning sign in dementia and mild cognitive impairment: Its relationship to cognition, behavior, and cerebrospinal fluid biomarkers. Dementia and Geriatric Cognitive Disorders, 46(1-2):42–49, 2018.

[18] MF Folstein, SE Folstein, and PR McHugh. “mini-mental state”: a practical method for grading the cognitive state of patients for the clinician. Psychiatric Research, 12(3):189–198, 1975.

[19] JC Morris. The clinical dementia rating (cdr) current version and scoring rules. Neurology, 43:2412–2414, 1993.

[20] A Alberdi, A Aztiria, and A Basarab. On the early diagnosis of alzheimer’s disease from multimodal signals: A survey. Artificial intelligence in medicine, 71:1–29, 2016.

[21] RR Selvaraju, M Cogswell, A Das, et al. Grad-cam: Visual explanations from deep networks via gradient-based localization. Proceedings of the IEEE International Conference on Computer Vision (ICCV), pages 618–626, 2017.

[22] Scott M. Lundberg and Su-In Lee. A unified approach to interpreting model predictions. Proceedings of the 31st International Conference on Neural Information Processing Systems, 2017.

[23] J Verghese, C Wang, RB Lipton, et al. Quantitative gait dysfunction and risk of cognitive decline and dementia. Journal of Neurology, Neurosurgery & Psychiatry, 2007.

[24] MM Mielke, RO Roberts, R Savica, et al. Assessing the temporal relationship between cognition and gait: Slow gait predicts cognitive decline in the mayo clinic study of aging. Journals of Gerontology Series A: Biomedical Sciences and Medical Sciences, 68(8):929–937, 2013.

[25] ML Callisaya, CL Blizzard, AG Wood, et al. Longitudinal relationships be- tween cognitive decline and gait slowing: the tasmanian study of cognition and gait. Journals of Gerontology Series A: Biomedical Sciences and Medical Sciences, 70(10):1226–1232, 2015.

[26] NB Alexander, JM Mollo, B Giordani, et al. Maintenance of balance, gait patterns, and obstacle clearance in alzheimer’s disease. Neurology, 45:908–914, 1995.

[27] J Hannink, T Kautz, CF Pasluosta, et al. Sensor-based gait parameter extraction with deep convolutional neural networks. IEEE Journal of Biomedical and Health Informatics, 21:85–93, 2016.

[28] Z Cao, T Simon, SE Wei, et al. Realtime multi-person 2d pose estimation using part affinity fields. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017.

[29] D Maji, S Nagori, M Mathew, et al. Yolo-pose: Enhancing yolo for multi person pose estimation using object keypoint similarity loss. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2022.

[30] L Kidzin ́ski, B Yang, JL Hicks, et al. Deep neural networks enable quantita- tive movement analysis using single-camera videos. Nature Communications, 11:4054, 2020.

[31] RA Armstrong. Alzheimer’s disease and the eye. Elsevier, 2:103–111, 2009.

[32] DD Salvucci and JH Goldberg. Identifying fixations and saccades in eye-tracking protocols. In Proceedings of the Symposium on Eye Tracking Research and Applications, 2000.

[33] S Tokushige, H Matsumoto, S Matsuda, et al. Early detection of cognitive decline in alzheimer’s disease using eye tracking. Frontiers in Aging Neuro- science, 15, 2023.

[34] G Ferna ́ndez, P Mandolesi, NP Rotstein, et al. Eye movement alterations during reading in patients with early alzheimer disease. Investigative Ophthalmology & Visual Science, 54:8345–8352, 2013.

[35] H Tanaka, H Adachi, H Kazui, et al. Detecting dementia from face in human-agent interaction. International Conference on Multimodal Interaction, 5, 2019.

[36] T Baltruˇsaitis, P Robinson, LP Morency, et al. Openface: an open source facial behavior analysis toolkit. IEEE Workshop on Applications of Computer Vision (WACV), 2016.

[37] C Zheng, M Bouazizi, T Ohtsuki, et al. Detecting dementia from face-related features with automated computational methods. Bioengineering, 10(7), 2023.

[38] M Ghadiri-Sani and AJ Larner. Head turning sign. Journal of the Royal College of Physicians of Edinburgh, 49:323–326, 2019.

[39] E Tjoa and C Guan. A survey on explainable artificial intelligence (xai): Toward medical xai. IEEE Transactions on Neural Networks and Learning Systems, 32(11):4793–4813, 2020.

[40] A Das and P Rad. Opportunities and challenges in explainable artificial intelligence (xai): A survey. arXiv:2006.11371, 2020.

[41] W Saeed and C Omlin. Explainable ai (xai): A systematic meta-survey of current challenges and future opportunities. Knowledge-Based Systems, 263, 2023.

[42] S Pereira, R Meier, V Alves, et al. Automatic brain tumor grading from mri data using convolutional neural networks and quality assessment. Un- derstanding and Interpreting Machine Learning in Medical Image Computing Applications, Springer, pages 106–114, 2018.

[43] H Panwar, PK Gupta, MK Siddiqui, et al. A deep learning and grad-cam based color visualization approach for fast detection of covid-19 cases using chest x-ray and ct-scan images. Chaos, Solitons & Fractals, 140, 2020.

[44] H Alshazly, C Linse, E Barth, et al. Explainable covid-19 detection using chest ct scans and deep learning. Sensors, 21(2), 2021.

[45] W Zhao, W Jiang, and X Qiu. Deep learning for covid-19 detection based on ct images. Scientific Reports, 11(1):1–12, 2021.

[46] M Umair, MS Khan, F Ahmed, et al. Detection of covid-19 using transfer learning and grad-cam visualization on indigenously collected x-ray dataset. Sensors, 21(17):5713, 2021.

[47] R Assaf and A Schumann. Explainable deep neural networks for multivariate time series predictions. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence (IJCAI), pages 6488–6490, 2019.

[48] K Fauvel, T Lin, V Masson, et al. Xcm: An explainable convolutional neural network for multivariate time series classification. Mathematics, 9, 2021.

[49] Y Li, H Yang, J Li, et al. Eeg-based intention recognition with deep recurrent- convolution neural network: Performance and channel selection by grad-cam. Neurocomputing, 415:225–233, 2020.

[50] V Jahmunah, EYK Ng, RS Tan, et al. Explainable detection of myocardial infarction using deep learning models with grad-cam technique on ecg signals. Computers in Biology and Medicine, 146, 2022.

[51] S El-Sappagh, JM Alonso, SMR Islam, et al. A multilayer multimodal de-detection and prediction model based on explainable artificial intelligence for alzheimer’s disease. Scientific Reports, 11(1):2660, 2021.

[52] Y Du, AR Rafferty, FM McAuliffe, et al. An explainable machine learning-based clinical decision support system for prediction of gestational diabetes mellitus. Scientific Reports, 12(1):1170, 2022.

[53] C Duckworth, FP Chmiel, DK Burns, et al. Using explainable machine learning to characterise data drift and detect emergent health risks for emergency department admissions during covid-19. Scientific Reports, 11(1):23017, 2021.