隨著人工智慧的快速發展，我們對於建立高準確度預測模型的追求同時引
起了對公平性的關注，尤其是在醫療領域。確保模型能夠準確協助醫生進行診
斷成為眾多研究學者的共同目標。特別是在處理胸腔放射圖(Chest Radiograph,
CXR) 等醫學影像時，這些影像可能包含有關患者種族、性別或年齡等信息。如
果模型在診斷過程中存在偏見，將被視為捷徑而不是全面學習，這將對患者產
生不公平的影響。
本研究使用了COVID-19 射線照相資料集和MIMIC-CXR 資料集，並深入探
討了可能存在的捷徑，包括圖片特徵類的捷徑(feature-related shortcut)、資料來
源的捷徑(source-related shortcut) 以及醫學統計資訊的捷徑(demography-related
shortcut)。同時，我們整合了現有的自監督學習(self-supervised learning, SSL)
方法，以建立更穩健的模型主幹。此外，我們進行了消融式分析，透過定義四
種組合(CJB、JB、CB、CJ)，其中”C”代表裁剪，”J”代表顏色抖動，”B”代表
模糊，來觀察捷徑與增強方法之間的關係。
最後，我們不僅使用了梯度加權類激活圖(Gradient-weighted Class Activation
Mapping, GradCAM) 評估模型在圖像上的關注位置，還使用了t-隨機鄰近嵌
入法(t-distributed stochastic neighbor embedding, t-SNE) 來觀察自監督模型擷
取出的特徵的分布。此外，我們提出了三種捷徑類型的公平性計算方法算
法，在統計數據偏差的捷徑的公平性/表現評估中，表現最佳的自監督學習方
法不僅取得了更高的接收者操作特徵曲線面積(AUROC score, AUC)，比基準
模型多0.0133，也改善了公平性，對於性別偏差減少0.0026，對於年齡偏差減
少0.0823，種族偏差減少0.0209。通過這些實驗，我們成功在CXR數據集中消除
了特徵類的捷徑、資料來源的捷徑以及醫學統計資訊的捷徑。
這些算法有助於評估現有的自監督訓練方法對於存在捷徑的資料集的適用
性，以確保醫學影像分析的公平性和準確性。
I

With the rapid advancement of artificial intelligence, our pursuit of highaccuracy predictive models has also brought attention to the issue of fairness, particularly in the field of healthcare. Ensuring that models can accurately assist doctors in diagnosis has become a shared goal among many researchers. This is especially crucial when dealing with medical images such as Chest Radiographs (CXR), which may contain information related to patient ethnicity, gender, or age. If models exhibit biases during the diagnosis process, they are seen as taking shortcuts rather than achieving comprehensive learning, which can result in unfair implications for patients.
In this study, we employed the COVID-19 Radiography dataset and the MIMICCXR dataset to explore potential shortcuts, including feature-related shortcuts, source-related shortcuts, and demography-related shortcuts. When it comes to addressing shortcut learning, various methods are employed to circumvent shortcuts and enhance the generalization of neural networks such as adversarial attacks, meta-learning, domain adaptation, and increasing domain-specific prior
knowledge. In this work, we integrated existing self-supervised learning (SSL) methods to establish a more robust model backbone. Additionally, we conducted ablation analysis to investigate the relationship between shortcuts and augmentation methods by defining augmentations as four combinations (CJB, JB, CB, CJ), where ”C” represents cropping, ”J” represents color jittering, and ”B”
represents blurring.
Finally, our evaluation extended beyond assessing model attention on images
using Gradient-weighted Class Activation Mapping (GradCAM). We also
employed t-distributed stochastic neighbor embedding (t-SNE) to observe the distribution of features extracted by self-supervised models. Furthermore,we proposed three fairness definition calculation methods for different shortcut types. During the fairness/performance evaluation involving the demographyrelated shortcut, the top-performing SSL methods not only achieved a higher AUROC score (AUC), surpassing the baseline model by 0.013, but also exhibited improved fairness metrics. Specifically, they demonstrated a 0.003 reduction
AUC in gender bias, a 0.082 reduction AUC in age bias, and a 0.021 reduction AUC in race bias. Through these experiments, we successfully eliminate featurerelative shortcuts, source-relative shortcuts and demography-relative shortcuts in CXR datasets.
These methods aid in evaluating the applicability of existing self-supervised training methods to datasets with shortcuts, ensuring both fairness and accuracy in medical image analysis.

[1] Hangbo Bao, Li Dong, Songhao Piao, and Furu Wei. Beit: Bert pre-training of image transformers. arXiv preprint arXiv:2106.08254, 2021.
[2] Adrien Bardes, Jean Ponce, and Yann LeCun. Vicreg: Variance-invariancecovariance regularization for self-supervised learning. arXiv preprint arXiv:2105.04906, 2021.
[3] Solon Barocas, Moritz Hardt, and Arvind Narayanan. Fairness and Machine Learning: Limitations and Opportunities. fairmlbook.org, 2019. http://www.fairmlbook.org.
[4] Alexander Brown, Nenad Tomasev, Jan Freyberg, Yuan Liu, Alan Karthikesalingam, and Jessica Schrouff. Detecting shortcut learning for fair medical ai using shortcut testing. Nature Communications, 14(1):4314, Jul 2023.
[5] Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, and Armand Joulin. Unsupervised learning of visual features by contrasting cluster assignments. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 9912–9924. Curran Associates, Inc., 2020.
[6] Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In Proceedings of the 37th International Conference on Machine Learning, ICML’20. JMLR.org, 2020.
[7] Xinlei Chen and Kaiming He. Exploring simple siamese representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 15750–15758, June 2021.
[8] Prateek Chhikara, Prabhjot Singh, Prakhar Gupta, and Tarunpreet Bhatia. Deep convolutional neural network with transfer learning for detecting pneumonia on chest x-rays. In Lakhmi C. Jain, Maria Virvou, Vincenzo Piuri, and Valentina E. Balas, editors, Advances in Bioinformatics, Multimedia, and Electronics Circuits and Signals, pages 155–168, Singapore, 2020. Springer Singapore.
[9] Muhammad E. H. Chowdhury, Tawsifur Rahman, Amith Khandakar, Rashid
Mazhar, Muhammad Abdul Kadir, Zaid Bin Mahbub, Khandakar Reajul
Islam, Muhammad Salman Khan, Atif Iqbal, Nasser Al Emadi, Mamun
Bin Ibne Reaz, and Mohammad Tariqul Islam. Can AI help in screening
viral and COVID-19 pneumonia? IEEE Access, 8:132665–132676, 2020.
[10] Alex J. DeGrave, Joseph D. Janizek, and Su-In Lee. AI for radiographic COVID-19 detection selects shortcuts over signal. Nature Machine Intelligence, 3(7):610–619, Jul 2021.
[11] Robert Geirhos, Jorn-Henrik Jacobsen, Claudio Michaelis, Richard Zemel, Wieland Brendel, Matthias Bethge, and Felix A. Wichmann. Shortcut learning in deep neural networks. Nature Machine Intelligence, 2(11):665–673, Nov 2020.
[12] Spyros Gidaris, Praveer Singh, and Nikos Komodakis. Unsupervised representation learning by predicting image rotations, 2018.
[13] Ary L. Goldberger, Luis A. N. Amaral, Leon Glass, Jeffrey M. Hausdorff, Plamen Ch. Ivanov, Roger G. Mark, Joseph E. Mietus, George B. Moody, Chung-Kang Peng, and H. Eugene Stanley. Physiobank, Physiotoolkit, and Physionet. Circulation, 101(23):e215–e220, 2000.
[14] Jean-Bastien Grill, Florian Strub, Florent Altch´e, Corentin Tallec, Pierre H.Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Daniel Guo, Mohammad Gheshlaghi Azar, Bilal Piot, Koray Kavukcuoglu, R´emi Munos, and Michal Valko. Bootstrap your own latent a new approach to self-supervised learning. In Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS’20, Red Hook, NY, USA, 2020. Curran Associates Inc.
[15] Jie Gui, Tuo Chen, Qiong Cao, Zhenan Sun, Hao Luo, and Dacheng Tao. A survey of self-supervised learning from multiple perspectives: Algorithms, theory, applications and future trends, 01 2023.
[16] Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Doll´ar, and Ross Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16000–16009, 2022.
[17] Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations, 2019.
[18] Alistair E. W. Johnson, Tom J. Pollard, Seth J. Berkowitz, Nathaniel R. Greenbaum, Matthew P. Lungren, Chih-ying Deng, Roger G. Mark, and Steven Horng. MIMIC-CXR, a de-identified publicly available database of chest radiographs with free-text reports. Scientific Data, 6(1):317, Dec 2019.
[19] Alistair E. W. Johnson, Tom J. Pollard, Seth J. Berkowitz, Roger G. Mark, and Steven Horng. MIMIC-CXR database(version2.0.0). PhysioNet, 2019. https://doi.org/10.13026/C2JT1Q.
[20] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
[21] Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278–2324, Nov 1998.
[22] Timothy P. Lillicrap, Jonathan J. Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra. Continuous control with deep reinforcement learning. In Yoshua Bengio and Yann LeCun, editors, 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Conference Track Proceedings, 2016.
[23] Matthias Minderer, Olivier Bachem, Neil Houlsby, and Michael Tschannen. Automatic shortcut removal for self-supervised representation learning. In Hal Daum´e III and Aarti Singh, editors, Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pages 6927–6937. PMLR, 13–18 Jul 2020.
[24] Amith Khandakar Rashid Mazhar Muhammad Abdul Kadir Zaid Bin Mahbub Khandakar R. Islam Muhammad Salman Khan Prof. Atif Iqbal Nasser
Al-Emadi Prof. Mamun Bin Ibne Reaz Muhammad E. H. Chowdhury, Tawsifur Rahman. COVID-19 radiography database. https://www.kaggle.com/
datasets/tawsifurrahman/covid19-radiography-database.
[25] Junhyun Nam, Hyuntak Cha, Sungsoo Ahn, Jaeho Lee, and Jinwoo Shin.
Learning from failure: De-biasing classifier from biased classifier. Advances
in Neural Information Processing Systems, 33:20673–20684, 2020.
[26] Mehdi Noroozi and Paolo Favaro. Unsupervised learning of visual representations by solving jigsaw puzzles, 2017.
[27] Tom J Pollard, Alistair E W Johnson, Jesse D Raffa, and Roger G Mark.
tableone: An open source Python package for producing summary statistics for research papers. JAMIA Open, 1(1):26–31, 05 2018.
[28] Tawsifur Rahman, Amith Khandakar, Yazan Qiblawey, Anas Tahir, Serkan Kiranyaz, Saad Bin Abul Kashem, Mohammad Tariqul Islam, Somaya
Al Maadeed, Susu M. Zughaier, Muhammad Salman Khan, and Muhammad E.H. Chowdhury. Exploring the effect of image enhancement techniques
on COVID-19 detection using chest X-ray images. Computers in Biology and Medicine, 132:104319, 2021.
[29] Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based localization. In ICCV, pages 618–626. IEEE Computer Society, 2017.
[30] Enzo Tartaglione, Carlo Alberto Barbano, Claudio Berzovini, Marco Calandri, and Marco Grangetto. Unveiling covid-19 from chest x-ray with deep learning: A hurdles race with small data. International Journal of Environmental Research and Public Health, 17(18), 2020.
[31] Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of Machine Learning Research, 9(86):2579–2605, 2008.
[32] Jure Zbontar, Li Jing, Ishan Misra, Yann LeCun, and St´ephane Deny. Barlow twins: Self-supervised learning via redundancy reduction. In International Conference on Machine Learning, pages 12310–12320. PMLR, 2021.
[33] Zhilu Zhang and Mert R. Sabuncu. Generalized cross entropy loss for training deep neural networks with noisy labels. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, NIPS’18, page 8792–8802, Red Hook, NY, USA, 2018. Curran Associates Inc.
[34] Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A. Efros. Unpaired image-to-image translation using cycle-consistent adversarial networks. In 2017 IEEE International Conference on Computer Vision (ICCV), pages 2242–2251, 2017.