Assessing Biological and Methodological Aspects of Brain Volume Loss in Multiple Sclerosis

Study Population

In this study, the first 147 consecutive patients with MS (with conditions defined according to the McDonald criteria^17,18) who were enrolled into the prospective MS-VisualPath cohort¹⁹ at the Hospital Clinic–University of Barcelona were evaluated for eligibility. We excluded 5 patients who had not completed any MRI follow-up visits, 1 patient who had MRI scans of insufficient quality, and 1 patient with a disease duration longer than 30 years. As a result, 140 patients with MS were included in this study. Each patient underwent annual MRI scans (median number per patient, 4; interquartile range [IQR], 3-4) with a median interscan period of 364 days (IQR, 351-379 days). In addition, we included 2 different cohorts of healthy volunteers: a cohort of 34 healthy volunteers who underwent 2 MRI scans within a median interscan period of 24.5 days, and a second cohort of 20 healthy volunteers who underwent 2 MRIs within a median interscan period of 384.5 days.

The institutional review board of the Hospital Clinic of Barcelona approved the study, and all participants provided their written informed consent. Moreover, the article follows the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) guidelines. Further details can be found in eMethods in the Supplement.

Images Acquisition and Processing

All MRI studies were performed with a 3-T Magnetom Trio scanner (Siemens), using a 32-channel phased-array head coil.¹⁹ In this study, 3-dimensional (3-D) structural T1-weighted isometric voxel magnetization-prepared rapid gradient echo (T1-MPRAGE) images with a voxel size of 0.9 × 0.9 × 0.9 mm³ were used to estimate all the volumes, and 3-D T2-FLAIR with the same voxel size was used to make the lesion masks.

First, T2-FLAIR images were registered to T1-MPRAGE scans using a rigid transformation, with 6 df (FSL). Second, a trained neurologist manually created T1 lesion masks using ITK-SNAP, version 2.4 (Penn Image Computing and Science Laboratory, University of Pennsylvania, and the Scientific Computing and Imaging Institute, University of Utah).²⁰ Third, lesion in-painting was performed on T1-MPRAGE scans using lesion mask, a state-of-the-art method for volumetric analysis that avoids pixel misclassification.²¹ Finally, we used the JI¹³ and FSL methods^14,15,16 to quantify the change in volume in the whole brain, gray and white matter, thalamus, caudate nucleus, putamen, and hippocampus. Healthy volunteers were evaluated using the same protocol, scanner, and image processing pipeline, excluding the lesion in-painting.

Structural Image Evaluation, Using Normalisation, of Atrophy (SIENA) is a registration-based technology used to quantify changes in whole-brain volume between 2 time points. By contrast, SIENA-X and FMRIB's Integrated Registration and Segmentation Tool (FIRST) are segmentation-based techniques that are useful for cross-sectional studies but less reliable for longitudinal studies. The SIENA-X and FIRST tools segment the voxels in the region of interest (ROI) (eg, gray matter) in a single 3-D T-1 scan, measuring the volume as the number of segmented voxels in the ROI multiplied by the voxel volume of the 3-D T-1 scan. The difference between 2 points can then be calculated by subtracting the volumes.

The JI method computes a nonrigid registration between the two 3-D T-1 scans, producing a Jacobian matrix for each voxel, the determinant of which reflects the expansion and contraction of that voxel. The integration of all determinants in the segmented region (eg, gray matter) gives an estimate of the change in volume.²² We used a custom JI pipeline defined previously¹³ that is essentially no different than commercial JI solutions (such as MSmetrix).²³

Statistical Analyses

We used the median and IQRs to describe the quantitative variables, absolute frequencies, and the proportions of the qualitative variables. We used mixed-effects regression to model the rate of change in brain volume, accounting for intraparticipant correlation.²⁴ We compared third-order and second-order B-spline and linear models based on Akaike Information Criterion.²⁵ With disease duration (time from MS onset) as a main fixed-effect predictor, these models were used to fit the rates of brain volume changes between visits using the JI and FSL methods. We used the pseudoR² defined by Nakagawa and Schielzoth²⁶ to estimate the goodness of fit of the models. To evaluate the potential influence of sex, age, DMT use, and use of steroids, we included these variables as fixed effect predictors in mixed-effects models. We included age at MS onset instead of age at study inclusion to avoid collinearity with disease duration. For variables with a significant association, we evaluated whether the effect on the rate of brain volume change might differ during disease progression (effect modification) using an interaction term between MS duration and covariate.

Short-term test-retest measurement errors were estimated using the formula of 100 * (V2 − V1) / mean(V1,V2), where V1 and V2 were absolute values measured at the 2 time points for the segmentation technologies (SIENA-X and FIRST), while the absolute percentage change in volume between 2 points was used for registration-based technologies (SIENA and JI). We presented the 50th and 75th percentiles with their 95th CIs estimated using a bias-corrected accelerated bootstrap to summarize the test-retest measurement errors. We also calculated the same difference after coregistering both MPRAGE (which is not a standard step for SIENA-X).

To assess if the JI and FSL estimated brain volumes are comparable, R² was evaluated as a measure of the goodness of fit of the simple linear regression models for pairwise brain regions quantified with JI and FSL. Additionally, we evaluated the performance of the annual rate of whole and regional brain volume changes to classify 20 healthy volunteers and 100 age and sex-matched patients with MS. We used a Hosmer-Lemeshow goodness-of-fit test for calibration and receiver operating characteristics curve analyses for discrimination. We also obtained the area under the curve, sensitivity, and specificity of the best cutoffs according to the Youden J statistic.²⁷

We performed 2 sensitivity analyses: complete case analyses to assess the influence of missing data and analyses excluding participants with more than a 10-fold interscan change (which would evaluate the influence of extreme values on results). Two-tailed P values <.05 were considered statistically significant, and all statistical analyses were performed using R language (R version 3.3.3; R Foundation for Statistical Computing). Data analysis was completed from January 2017 to May 2017.

Study Population Features

The study population included 140 patients with MS (95 women [67.8%]; median [interquartile range] age, 40.7 [33.6-48.1] years) with a median (IQR) of 7.0 (3.3-12.6) years of disease duration at inclusion (Table 1). Most patients had relapsing-remitting MS, were mildly disabled (median [IQR] score on the Expanded Disability Status Scale, 1.5 [1-2] points), and were receiving DMTs at inclusion.

Patients were allowed to change therapy during the follow-up period, and 7 patients (5.0%) started a DMT (low to intermediate potency); 11 patients (7.9%) changed to another DMT within the same treatment group (low to intermediate potency or high potency); 8 patients (5.7%) changed from natalizumab to fingolimod because of a positive John Cunningham virus response; and 1 patient (0.7%) changed from low-to-intermediate–potency to high-potency treatment because of a lack of efficacy of the first regimen. Additionally, 22 of 138 patients (15.9%) received steroids during the first interscan period, 13 of 118 patients (11.0%) received them during the second, and 9 of 107 (8.4%) received them during the third interscan period.

Rate of Brain Volume Loss by MS Stages

Third-order B-spline mixed-effects models best fit the distribution of the annual changes in whole-brain and gray matter volume quantified with JI according to the Akaike Information Criterion²⁵ (eTable 1 in the Supplement). The annual whole-brain and gray matter volume changed faster during the first 5 years of disease (Figure 1) than in later years, with Nakagawa conditional R² values of 0.65 and 0.52, respectively. The JI method gave a similar dynamic pattern of annual changes in the thalamus, putamen, and hippocampus, but not the caudate nucleus (eFigure 1 in the Supplement). No significant model was found for the annual rate of changes in white matter volume. Similar annual rates of change in whole-brain volume were obtained with SIENA (eFigure 2 in the Supplement), yet no suitable model was found for the annual rate of gray matter volume change using SIENA-X (eFigure 2 in the Supplement) or for the deep gray matter volume changes using FIRST (eFigure 3 in the Supplement).

Demographic-Associated and MS–Associated Variables and Brain Volume Change Dynamics

We did not find a significant association with sex (β = −0.091; P = .21), age at inclusion (β = 0.001; P = .92), DMT (low-to-intermediate–potency drugs: β = 0.105; P = .12; high-potency drugs: β = 0.061; P = .63), or steroid use (β = 0.073; P = .19) in the model assessing whole-brain volume changes. Similarly, there was no significant coefficient for disease duration in the model of white-matter volume change conditioning on sex, age at MS onset, DMT use, and steroid use. In the model of changes in gray matter volume, no significant effect of sex (β = 0.051; P = .47) or age at MS onset (β = −0.008; P = .06) was found.

However, the near significance of the age at MS onset prompted exploration of the interaction of age at onset with other factors. The interaction between age at MS onset and MS duration was marginally significant (modeled as a binary variable with a cutoff in 5 years or >5 years: β = 0.015; P = .047). Controlling for sex, DMT, and steroid use, participants with an MS duration of 5 years or less had 0.018% per year lower change in gray matter volume for each year of later MS onset (age: β = −0.003; interaction: β = −0.015), whereas the effect for those with an MS duration longer than 5 years was a decrease of 0.003% per year (age: β = −0.003).

We did not find a significant association with DMT (low-to-intermediate–potency drugs: β = 0.015; P = .83; high-potency drugs: β = −0.088; P = .52) or steroid use (β = 0.110; P = .09) in an assessment with no interaction with MS duration (β = 0.164; P value = .23). Results from patients with MS who had a disease duration of less than 5 years, stratified by DMT and steroid use, are in eTable 2 in the Supplement.

For deep gray matter structures, we found a significant association with steroids in models assessing changes in thalamic volume (β = 0.280; P = .02), without interaction with disease duration (β = 0.145; P = .56). Moreover, we found a significant association with age at MS onset (β = 0.019; P = .01) in the dynamics of hippocampal volume change, yet no interaction with disease duration (β = −0.002; P = .86). We did not find any other significant associations for the other brain regions tested.

Limited Reproducibility of Brain Volume Estimates and Use in Individuals

As registration-based methods, FSL and JI produced similar short-term test-retest errors for whole-brain volume changes (FSL: 75th percentile, 0.27; JI: 75th percentile, 0.36). The estimates with JI were more precise (with a smaller IQR) and less biased (with a median closer to 0) than those with FSL for all the brain regions except the putamen, for which JI was more precise but FIRST was less biased. The error in assessing the gray matter volume was nearly 5-fold higher for SIENA-X (75th percentile, 1.78) than for JI (75th percentile, 0.38) (Table 2 and Figure 2). Coregistration of two 3D-T1 scans before running SIENA-X did not improve the results (eTable 3 in the Supplement).

Comparison of Algorithms for Brain Volume Change Assessments

An evaluation of the correlations between the changes in brain volume quantified with JI and FSL yielded a high correlation for the whole-brain volume changes (R² = 0.92; P < .001). By contrast, the correlations for different brain regions were low, with a maximum R² = 0.23 for the thalamus (eFigure 4 in the Supplement).

Brain Volume Cutoffs per Estimation Method

Finally, we evaluated the performance of the annual rates of brain volume changes estimated with FSL and JI to correctly classify 20 healthy volunteers (with a median [IQR] age of 33.0 [28.7-39.2] years; 10 female [50%]) and 100 patients with MS (median [IQR] age = 35.5 [32.6-41.3] years; 69 female [69%]) from the 140 patients with MS who were matched by age (P = .22) and sex (P = .17). Although the area under curve for whole-brain volume changes was similar for both methods, the optimal thresholds differed (Table 3).

Finally, we found different sensitivity and specificity for the cutoffs described previously for annual whole-brain volume changes²⁸ using the FSL (SIENA) and JI methods when quantifying brain volume changes in the same study population. A Hosmer-Lemeshow goodness-of-fit test produced nonsignificant P values (favoring acceptable calibration) for all models except the gray matter volume change using the JI method (area under curve, 0.550; P = .04; Table 3).

Sensitivity Analyses

We included 107 participants (76.4%) with 4 MRI scans, 21 participants (15%) with 3 MRI scans, and 12 participants (8.6%) with 2 MRI scans. We found similar results from the complete case analyses (eFigure 5 in the Supplement) and the results were similar in the models after excluding participants with extreme changes (eFigure 6 in the Supplement).

Limitations

Our study also has some limitations, not least of all that our sample size was insufficient to analyze relapsing vs progressive MS separately and model changes after more than 25 to 30 years of disease duration. Moreover, we did not find a model for the white matter volume change, which may indicate that the sample size is too small for such a complex model. In this context, the acquisition of additional time points or of 2 T1s (with repositioning) at each scan session may have been useful. We also did not assess repeatability in patients with MS, although similar test-retest error measurements have already been reported for whole-brain volume changes in patients with MS assessed with JI (MS metrics) and FSL (SIENA).²³ In addition, we did not evaluate other methods such as Freesurfer,³⁷ Statistical Parametric Mapping,³⁸ or the new unreleased version of SIENA-XL (a longitudinal version of SIENA-X),³⁹ sequences other than MPRAGE, or different scanners and strengths. Similar errors were found with JI (MSmetrics) and FSL (SIENA) using 1.5-T and 3-T scanners,²³ yet the impact of using scanners with different field strength on the error in BVL estimation was an order of magnitude lower for JI than for FSL (SIENA). Finally, we used a short-term rescan approach to assess the repeatability of the brain volume estimations, while a long-term rescan method would have been preferable to account for changes in brain shape or other long-term changes that were not addressed by our approach.