. Author manuscript; available in PMC: 2023 Jun 1.

Published in final edited form as: Proc IEEE Int Conf Acoust Speech Signal Process. 2023 May 5;2023:10.1109/icassp49357.2023.10097126. doi: 10.1109/icassp49357.2023.10097126

GLACIER: GLASS-BOX TRANSFORMER FOR INTERPRETABLE DYNAMIC NEUROIMAGING

Usman Mahmood ^*,^†,^◦, Zening Fu ^*,^†, Vince Calhoun ^*,^‡, Sergey Plis ^*,^†

PMCID: PMC10231935 NIHMSID: NIHMS1889297 PMID: 37266485

Abstract

Deep learning models can perform as well or better than humans in many tasks, especially vision related. Almost exclusively, these models are used to perform classification or prediction. However, deep learning models are usually of black-box nature, and it is often difficult to interpret the model or the features. The lack of interpretability causes a restrain from applying deep learning to fields such as neuroimaging, where the results must be transparent, and interpretable. Therefore, we present a ’glass-box’ deep learning model and apply it to the field of neuroimaging. Our model mixes spatial and temporal dimensions in succession to estimate dynamic connectivity between the brain’s intrinsic networks. The interpretable connectivity matrices produced by our model result in beating state-of-the-art models on many tasks using multiple functional MRI datasets. More importantly, our model estimates task-based flexible connectivity matrices, unlike static methods such as Pearson’s correlation coefficients.

Keywords: Interpretable DL, neuroimaging, fMRI

1. INTRODUCTION

Deep learning (DL) has been successfully applied to a variety of tasks across different fields. As of late, DL models in some cases exceeded human-level performance. However, DL is still perceived as a black-box method as it is hard to interpret the results generated by a deep neural network. The lack of interpretability limits the possible application of DL as it cannot be used for explainability tasks and even cannot be used independently for simple classification tasks in fields that require a high level of transparency, e.g, neuroimaging.

Studies like [1, 2, 3, 4] show that disruptions in the dynamics of the brain’s intrinsic network connectivity are related the multiple brain disorders. Functional (network) connectivity (F(N)C) matrices are used to represent the brain’s connectivity and are computed by measuring some form of correlation such as Pearson’s correlation coefficients (PCC). These hand-crafted FNC matrices are used with classical machine learning methods (ML) such as logistic regression (LR) to achieve high classification performance on brain disorders. Though interpretable the computed FNC matrices are only data dependent and can be seen as proxy-features that are stripped of many important dynamics present in the actual data.

Many studies [5, 6, 7, 8] have used DL in the field of neuroimaging for disorder or gender classification. Because of the lack of interpretability of DL, in almost all of these studies, an interpretable statistical method is incorporated in some way to compute the FNC or is pre-computed. The DL models use the computed FNCs to create embeddings, which are then used for downstream classification. This approach gives high classification performance which was already achieved by ML methods and makes us think about the benefit of these models.

To solve the interpretability problem of DL, methods like saliency maps [9, 10] have been built that try to increase the explainability of the data and the model. While the output maps of such methods improve interpretability, the values of bivariate correlation coefficients are still easier to communicate across. Also, methods like saliency maps have limitations and are post hoc, and do not specifically encourage the model to produce interpretable representations.

We hypothesize that DL can uncover task-dependent dynamics present in the neuroimaging datasets which are missed by both existing machine learning and DL models. Another issue with existing methods is that they are only data-dependant and provide the same FNC matrices irrespective of the downstream task. DL models that compute FNC try to mitigate this problem by learning task-dependent embeddings but their method of computing FNC still remains only data-dependent.

In this paper, we present a glass-box transformer model Glacier that provides interpretability by producing task-dependent connectivity matrices. Our model shows that, unlike the hybrid models, DL can be successfully applied to neuroimaging without incorporating another method. Directed (network) connectivity (D(N)C) matrices estimated by our model are downstream task-dependent and uncover crucial spatial and temporal biomarkers. We also show that using the estimated DNCs our model beats state-of-the-art models on brain disorder classification (schizophrenia, dementia, and autism), and gender classification.

2. METHOD

Traditionally, deep neural networks are used to create embeddings from the data. These embeddings are used for downstream tasks but are often difficult to interpret. To solve this problem, we present Glacier as a DL model to estimate the dynamic DC graph between the nodes present in the dataset. Instead of the embeddings, we use only the learned graph structure for the downstream tasks. By not using the embeddings, we force the model to learn distinct graph structures for the groups present in the data (e.g., healthy controls (HC) and Patients). The idea of the model is to use self-attention to uncover task-dependent spatio-temporal dependencies while performing downstream classification. These dependencies are used to interpret the results and highlight important biomarkers. We notice that interpreting distinct graphs are much easier than interpreting embeddings that are often in high-dimension space. The directed edges of our learned graph represent the directed connectivity score estimated via self-attention. We perform temporal attention to create a final static graph based on which classification is performed. We explain the working of our model in this section. Refer to 1 for the architectural details.

2.1. Temporal Mixing

Important information is present in the temporal dimension of datasets which are of the sequential form (time-series). For example in speech datasets, the location of the words in a sentence have great significance, and changing the order can result in a different meaning. Similarly, the temporal dimension of medical imaging data is also of critical nature, where indicator(s) of the interested problem is seen at specific time-points and have a significant effect on other time-points. To capture these time-sensitive dependencies we use transformer encoder [11]. Using the dependencies the encoder mixes the temporal dimension and outputs new embeddings which have more influence from the time-points that are identified as important by the model. We use our model to capture directed dependencies between every time-point. As the dependencies can be different for the nodes in the dataset, we input time-series of each node individually but share the weights of the model. To convert the scalar value $x_{t}^{i}$ representing the value of $i^{t h}$ node at time-point $t$ into a vector we first use a feed-forward neural network whose output is passed to the transformer encoder which outputs the vector $c_{t}^{i}$ .

2.2. Spatial Mixing

Nodes of a system are dependent and affect each other through time. The rate of change in these dependencies is reliant on the underlying system. Activity in the human brain is extremely dynamic and can change at any moment. To capture instantaneous directed connectivity between nodes of the system, we use self-attention between the nodes at each time-point. At any time-point $t$ the self-attention module receives $N$ embeddings represented by vector $c$ and outputs a dependency matrix $W$ . We represent this $W$ matrix as the directed connectivity between the nodes.

2.3. Temporal Attention and Classification

To create a single weight matrix $W^{f}$ for the downstream task we take the weighted average of $W_{1 - T}$ matrices. For the downstream classification task, not all the time-points are equally important, hence it is crucial to incorporate a temporal attention module that assigns weight to each $W_{t}$ and calculates a weighted average of all the weight matrices. We use the GTA proposed by [12]. GTA first takes an average of all the $T$ DC matrices to create $W^{g l o b a l}$ which represents the global view. Then the similarity of each local $W_{t}$ with the global view is compared and used to create the temporal attention vector < is the Hadamard product.

W^{g l o b a l} = \frac{1}{T} \sum_{t = 1}^{T} W_{t} {\tilde{W}}_{t} = W_{t} ⊙ W^{g l o b a l} α = (\oplus_{t = 1}^{T} ((((flat ({\tilde{W}}_{t})) W^{M L P_{l 1}}) W^{M L P_{l 2}}) W^{M L P_{l 3}}) W^{f} = \sum_{t = 1}^{T} W_{t} α_{t}

(1)

3. DATASETS AND TRAINING

We apply our model to neuroimaging datasets because of two major reasons. Firstly, neuroimaging is a field where results need to be transparent and interpretable, and mere classification performance is not enough to trust the results. Secondly, as brain disorders are linked with dysconnectivity in the connectivity between the brain’s intrinsic network, we feel that our model that uses only the estimated graph between nodes for classification is an ideal fit for such fields and datasets. We test our model on four neuroimaging datasets. These datasets represent the functional activity of the brain captured via resting-state functional magnetic resonance imaging (fMRI) scans. Four datasets used in this study include phase III of FBIRN (Function Biomedical Informatics Research Network [13] project, release 1.0 of ABIDE (Autism Brain Imaging Data Exchang)e [14] and release 3.0 of OASIS (Open Access Series of Imaging Studies) [15] to predict schizophrenia, autism, and dementia respectively. Subjects from the ABCD [16] datasets are used for gender prediction.

3.1. Preprocessing

Instead of using the voxel-level data, for disorder classification we use the independent component analysis (ICA) as the brain parcellation method. We first preprocess the fMRI data using statistical parametric mapping (SPM12, http://www.fil.ion.ucl.ac.uk/spm/) within MATLAB 2020. Subjects were included in the analysis if the subjects have head motion ≤ 3° and ≤ 3 mm, and with functional data providing near full brain successful normalization [17]. For each subject, 100 ICA components are estimated using the Neuromark template and used in experiments following the same procedure described in [17].

3.2. Training

We ran our experiments using RTX 2080 using PyTorch. The hidden dimensions for the embedder were 64 and for both self-attention layers, it was set to 48. The dimensions of the MLP layer for calculating the temporal attention vector were $γ * l e n (f l a t (W_{t}))$ and 1 with $γ = 0.25$ . We used batch normalization after the first MLP layer. ReLU activation was used in our model between the MLP layers. A final two-layer MLP was used to get logits for binary classification problems with $W_{f}$ as input with dimensions 64 and 2. We used cross-entropy loss between the binary targets $y$ and the predictions $\hat{y}$ with Adam optimizer. L1 regularization of the parameters was added to the loss with regularization weight set to $1 e^{- 7}$ and the learning rate ( $η$ ) was $1 e^{- 4}$ . For each dataset, we perform experiments on ‘n’ test folds (10 except FBIRN (18)), depending on the size of the data with 10 randomly seeded trials for each fold. We report the mean area under the curve - receiver operating characteristic (AUC-ROC) and other metrics to show classification performance.

4. CLASSIFICATION RESULTS

We compare classification performance with state-of-the-art DL model BNT [18] and with ML model LR. BNT beats existing methods like Graphormer [19], BrainNetCNN [20], BrainNetGNN [21] thus we do not directly compare with those methods. Figure 2 shows that BNT classification performance is similar to LR model using hand-crafted features PCC matrices. This puts a question on the advantage of using a DL model as one of the purposes of DL was to avoid using hand-crafted features. On the other hand, our model works directly on time-courses (TC) and significantly performs better than LR model with TC as input. Also, our model performs equally well as BNT trained on PCC matrices and proves that a DL model can be used in neuroimaging without hand-crafted features.

Fig. 2: — AUC comparison of Glacier with BNT and LR. Our model works on ICA time courses (TC) and not hand-crafted features - PCC. BNT performs similarly to LR using hand-crafted features as input but our model significantly outperforms LR on TC data and does not use hand-crafted features.

5. INTERPRETATION

Figure 3 shows the DNC estimated by our model for HC and SZ patients using FBIRN dataset. We notice that HC shows hyper-connectivity as compared to SZ patients. The internetwork dysconnectivity for SZ patients is reported in existing studies. Furthermore, HC show high connectivity between visual (VI) and sensorimotor (SM) networks, and connectivity of the subcortical (SC) network is shown with the cerebellum (CB), default-mode (DM), and cognitive control (CC) networks. Whereas, SZ patients do not show such patterns but rather show hyper-connectivity in the default-mode (DM) network as compared to the HC. The finding of dysconnectivity and/or lower connectivity scores for VI and SM networks for SZ patients is not surprising as there exists ample evidence in prior studies of schizophrenia leading to multiple abnormalities related to visual and motor functions such as perception of contrast and motion, detection of visual contours, and control of eye movements. These abnormalities certainly affect motor skills which we feel is a reason for the low connectivity for SM and VI networks captured by our model for SZ patients.

Fig. 3: — We compare the estimated DNC of HC and SZ patients. The axis shows the components divided into 7 networks. Connectivity patterns differ for HC and SZ patients across multiple networks, especially VI, SM, and DM.

5.1. Flexible Connectivity Estimation

One of the most significant advantages of our model is that the model extracts useful features (connectivity matrices) and does not rely on inflexible hand-crafted features. The flexibility allows our model to produce task-specific (e.g., brain dis order) connectivity matrix and produce subgraphs for a brain disorder. Thus highlights crucial brain network connectivity patterns. In Figure 4 we show how our model focuses on SM network for dementia prediction and DM network for gender prediction of the same subjects. Static PCC-based FC is inflexible and produces the same result irrespective of the task.

Fig. 4: — Glacier estimates flexible DNC structures based on the ground-truth signal. We train our model for different classification tasks and use the same test subjects to compare the estimated DNC for the subjects. 4a and 4b are the connectivity matrices estimated by our model when trained to predict dementia and gender respectively of the same subjects. 4c is the FNC computed using PCC. The FNC is independent of the task and would remain fixed (inflexible). Notice how the average value of SM in 4a is higher than 4b and 4c. Whereas average value of 4b is higher than 4a and 4c. This shows that our model gives more attention to disorder-specific networks. The importance of SM for dementia prediction and DM for gender prediction is shown in existing studies [22, 23].

6. CONCLUSION

Our work shows the importance of using flexible and interpretable DL models. We used our interpretable model to estimate directed connectivity between the brain’s intrinsic networks. The estimated DNCs not only provided high classification scores but more importantly captured the disorder-dependent differences in brain network connectivity between HC and patients. We also show that deep-learning models can be successfully applied to fields like neuroimaging for classification and interpretability without incorporating other statistical or ML methods. For future work, we would like to see the differences in connectivity for multiple brain disorders and capture interpretable temporal bio-markers.

Fig. 1: — Our model is comprised of two self-attention modules for temporal and spatial mixing. Temporal attention on top generates a single matrix which is used by a Multi-layer perceptron (MLP) to make the downstream classification.

7. REFERENCES

[1].Heuvel Martijn and Pol Hilleke, “Exploring the brain network: A review on resting-state fmri functional connectivity,” European neuropsychopharmacology : the journal of the European College of Neuropsychopharmacology, vol. 20, pp. 519–34, August 2010. [DOI] [PubMed] [Google Scholar]
[2].Culbreth Adam, Wu Qiong, Chen Shuo, Adhikari Bhim, et al. , “Temporal-thalamic and cingulo-opercular connectivity in people with schizophrenia,” NeuroImage: Clinical, vol. 29, pp. 102531, January 2021. [DOI] [PMC free article] [PubMed] [Google Scholar]
[3].Yu Qingbao, Sui Jing, Rachakonda Srinivas, He Hao, et al. , “Altered small-world brain networks in temporal lobe in patients with schizophrenia performing an auditory oddball task,” Frontiers in systems neuroscience, vol. 5, pp. 7, February 2011. [DOI] [PMC free article] [PubMed] [Google Scholar]
[4].Zhang Yuan, Dai Zhongxiang, et al. , “Altered intra- and inter-hemispheric functional dysconnectivity in schizophrenia,” Brain Imaging and Behavior, vol. 13, 10 2019. [DOI] [PubMed] [Google Scholar]
[5].Kazi Anees, Farghadani Soroush, and Navab Nassir, “Ia-gcn: Interpretable attention based graph convolutional network for disease prediction,” 2021. [DOI] [PMC free article] [PubMed]
[6].Kim Byung-Hoon and Ye Jong Chul, “Understanding graph isomorphism network for rs-fmri functional connectivity analysis,” Frontiers in Neuroscience, vol. 14, pp. 630, 2020. [DOI] [PMC free article] [PubMed] [Google Scholar]
[7].Sofia Ira Ktena Sarah Parisot, Ferrante Enzo, Rajchl Martin, et al. , “Metric learning with spectral graph convolutions on brain connectivity networks,” NeuroImage, vol. 169, pp. 431–442, 2018. [DOI] [PubMed] [Google Scholar]
[8].Ma Guixiang, Ahmed Nesreen K., Willke Ted, Sengupta Dipanjan, et al. , “Similarity learning with higher-order graph convolutions for brain network analysis,” 2019.
[9].Simonyan Karen, Vedaldi Andrea, and Zisserman Andrew, “Deep inside convolutional networks: Visualising image classification models and saliency maps,” 2014.
[10].Angelov Plamen P., Soares Eduardo A., Jiang Richard, Arnold Nicholas I., et al. , “Explainable artificial intelligence: an analytical review,” WIREs Data Mining and Knowledge Discovery, vol. 11, no. 5, pp. e1424, 2021. [Google Scholar]
[11].Vaswani Ashish, Shazeer Noam, Parmar Niki, Uszkoreit Jakob, et al. , “Attention is all you need,” in Proceedings of the 31st International Conference on Neural Information Processing Systems, Red Hook, NY, USA, 2017, NIPS’17, p. 6000–6010, Curran Associates Inc. [Google Scholar]
[12].Mahmood Usman, Fu Zening, Calhoun Vince, and Plis Sergey, “Deep dynamic effective connectivity estimation from multivariate time series,” in 2022. International Joint Conference on Neural Networks (IJCNN), 2022, pp. 1–10. [Google Scholar]
[13].Keator David B, Erp Theo GM van, Turner Jessica A, Glover Gary H, et al. , “The function biomedical informatics research network data repository,” Neuroimage, vol. 124, pp. 1074–1079, 2016. [DOI] [PMC free article] [PubMed] [Google Scholar]
[14].Adriana Di Martino Chao-Gan Yan, Li Qingyang, Denio Erin, et al. , “The autism brain imaging data exchange: towards a large-scale evaluation of the intrinsic brain architecture in autism,” Molecular psychiatry, vol. 19, no. 6, pp. 659, 2014. [DOI] [PMC free article] [PubMed] [Google Scholar]
[15].Eugene H Rubin Martha Storandt, Miller J Philip, et al. , “A prospective study of cognitive function and onset of dementia in cognitively healthy elders,” Archives of neurology, vol. 55, no. 3, pp. 395–401, 1998. [DOI] [PubMed] [Google Scholar]
[16].Casey BJ, Cannonier Tariq, et al. , “The adolescent brain cognitive development (abcd) study: Imaging acquisition across 21 sites,” Developmental Cognitive Neuroscience, vol. 32, pp. 43–54, 2018. [DOI] [PMC free article] [PubMed] [Google Scholar]
[17].Fu Zening, Caprihan Arvind, Chen Jiayu, Du Yuhui, et al. , “Altered static and dynamic functional network connectivity in alzheimer’s disease and subcortical ischemic vascular disease: shared and specific brain connectivity abnormalities,” Human Brain Mapping, 2019. [DOI] [PMC free article] [PubMed]
[18].Kan Xuan, Dai Wei, Cui Hejie, Zhang Zilong, et al. , “Brain network transformer,” ArXiv, vol. abs/2210.06681, 2022.
[19].Ying Chengxuan, Cai Tianle, Luo Shengjie, Zheng Shuxin, et al. , “Do transformers really perform bad for graph representation?,” in NeurIPS, 2021.
[20].Kawahara Jeremy, Brown Colin J., et al. , “Brainnetcnn: Convolutional neural networks for brain networks; towards predicting neurodevelopment,” NeuroImage, vol. 146, pp. 1038–1049, 2017. [DOI] [PubMed] [Google Scholar]
[21].Mahmood U, Zening Fu, Calhoun Vince D., and Plis S, “A deep learning model for data-driven discovery of functional connectivity,” ArXiv, vol. abs/2112.04013, 2021. [Google Scholar]
[22].Albers Mark W, Gilmore Grover C, Jeffrey Kaye, Claire Murphy, et al. , “At the interface of sensory and motor dysfunctions and alzheimer’s disease,” Alzheimer’s & Dementia, vol. 11, no. 1, pp. 70–98, 2015. [DOI] [PMC free article] [PubMed] [Google Scholar]
[23].Kim Byung-Hoon, Jong Chul Ye, and Jae-Jin Kim, “Learning dynamic graph representation of brain connectome with spatio-temporal attention,” 2021.

PERMALINK

GLACIER: GLASS-BOX TRANSFORMER FOR INTERPRETABLE DYNAMIC NEUROIMAGING

Usman Mahmood

Zening Fu

Vince Calhoun

Sergey Plis

Abstract

1. INTRODUCTION