Travelling Heads – Measuring Reproducibility and Repeatability of Magnetic Resonance Imaging in Dementia

Catherine Morgan, Reece Roberts, Lynette Tippett and the Dementia Prevention Research Clinic Staff and Research Group.
School of Psychology and Centre for Brain Research, The University of Auckland.

The need to standardise MRI in multicentre studies

Multicentre studies, such as the DPRCs allow for larger, and more representative cohorts to be recruited for research trials. However, inter-site measurement variability needs to be quantifiable to interpret the pooled data. For imaging markers to be adopted widely and incorporated into clinical practice, inter-site measurement reliability needs to be established to determine confidence in diagnostic metrics. Reproducibility can be defined as variability due to measurements being collected under different conditions, e.g., at different sites with different hardware or processed with different software.

In addition, knowledge of imaging biomarker variability over time due to measurement uncertainty is essential when monitoring longitudinal cognitive changes as a function of normal ageing, disease, or treatment. Without this crucial information, it cannot be determined whether subsequent changes in the imaging markers are due to underlying physiological changes, or simply measurement variability. Repeatability refers to variability in a measurement collected under the same conditions multiple times.

Method

We assessed reproducibility and repeatability in a suite of imaging parameters used in the DPRC MRI protocol to study cognitive decline. We recruited six “travelling heads” (THs), the same participants who travelled to the three imaging centres in the DPRCs, Auckland, Christchurch, and Dunedin. They were scanned at repeated time-points with the same DPRC MRI scans as used for clinic participants, enabling assessment of the reproducibility and repeatability of quantitative MRI (qMRI) markers for dementia. Images were processed centrally, with the same software used for all data.

Results

MRI images from a person scanned at the three different sites within seven days are shown in Figure 1, images collected at the different sites show excellent visual agreement with similar signal distribution, contrast, and lack of obvious artefacts. As seen in Figure 2, group means for most metrics appear to be in good agreement between sites, overall group means are reproducible. As one example, group mean hippocampal (HC) volumes were 4.40, 4.38 and 4.44cm3 for Auckland, Christchurch, and Dunedin respectively. Considering individual participant data, inter-site variability would be low compared to inter-subject variation if lines connecting the same participants data at each site do not cross, which appears to be the case for grey matter (GM), white matter (WM), and HC volume. Conversely, the resting state functional MRI (rsfMRI) metrics, (Q, DMN, and DAN connectivity) and ASL metrics (grey and white matter perfusion) have crossing lines, suggesting inter-site variability is greater than inter-subject variability. Repeated scan metrics are plotted relative to baseline values in Figure 3. Volumetric measures derived from T1w images are highly repeatable (top row), as indicated by the flat regression lines and very narrow confidence intervals. Much larger confidence intervals are seen for the functional rsfMRI metrics.

Figure 1. Representative data for a single participant collected at three different sites within seven days (columns 1 to 3) and a list of the 15 derived metrics (column 4). From the top, row 1 shows the acquired T1w MPRAGE scan in participant’s native space, row 2 shows DMN connectivity maps (precuneus seed) processed in from resting state functional MRI data, row 3 shows calculated CBF maps, row 4 shows white matter FOD images, row 5 shows acquired FLAIR images, row 6 shows the acquired T2w images, and row 7 the SWI images. An oil capsule affixed to the left side of the head is seen in some images as an hyperintense circle on the right side of the head (images are presented in radiological orientation). Abbreviations: ASL – arterial spin labelling, CS – cross section, CBF – cerebral blood flow, CSF – cerebrospinal fluid, DAN – dorsal attention network, DMN – default mode network, DWI – diffusion weighted imaging, FLAIR – fluid attenuation inversion recovery, FOD – fibre orientation density, GM – grey matter, HC – hippocampal, MNI – Montreal Neurological Institute, rsfMRI – resting state functional magnetic resonance imaging, SWI – susceptibility weighted imaging, T1w – T1-weighted, T2w – T2-weighted, WM – white matter.

 Figure 3. Repeatability of metrics of interest measured at one site (Auckland). Circles represent relative values compared to baseline, measured in individual THs. Colours represent each participant. Grey lines indicate regression lines and shaded areas are 95% confidence intervals. Abbreviations: CS – cross section, CSF – cerebrospinal fluid, DAN – dorsal attention network, DMN – default mode network, FLAIR – fluid attenuation inversion recovery, GM – grey matter, HC – hippocampal SWI – susceptibility weighted imaging, WM – white matter.

Centre for eResearch support

All DPRC MRI data (from the hundreds of study participants and the six travelling heads described in this work) are stored on a Linux virtual machine (VM) hosted by the Centre for eResearch (CeR). Storing and analysing the data on the VM is useful for multiple reasons. 1) Multisite data can be stored in one location and processed centrally. This removes any extra variability in results from data collected between sites and over time (that we are trying to minimise) due to different image processing toolboxes used, 2) The DPRC research team is large and growing, as new members and students join they can easily be added to the VM users list, 3) Some MRI processing software is complicated to install and use. The CeR staff have been critical in helping install imaging software, including Docker packages and Python environments. 4) Some MRI processing software is compute and data intensive. CeR were able to find solutions for us, for example mounting different research drives to the VM and short term use of GPUs for specific data processing tasks. 5) MRI raw data sets and the processed data sets are large. CeR helped us with solutions to archive and backup the data.

References

1. Asken BM, DeKosky, S.T., Clugston, J.R. et al. Diffusion tensor imaging (DTI) findings in adult civilian, military, and sport-related mild traumatic brain injury (mTBI): a systematic critical review. Brain Imaging and Behavior 12, 585–612 (2018).
2. Woolrich MW, Jbabdi S, Patenaude B, Chappell M, Makni S, Behrens T, et al. Bayesian analysis of neuroimaging data in FSL. NeuroImage. 2009;45(1 Suppl):S173-86.
3. Smith SM, Jenkinson M, Woolrich MW, Beckmann CF, Behrens TE, Johansen-Berg H, et al. Advances in functional and structural MR image analysis and implementation as FSL. NeuroImage. 2004;23 Suppl 1:S208-19.
4. Jenkinson M, Beckmann CF, Behrens TE, Woolrich MW, Smith SM. FSL. NeuroImage. 2012;62(2):782-90.
5. Behrens TE, Woolrich MW, Jenkinson M, Johansen-Berg H, Nunes RG, Clare S, et al. Characterization and propagation of uncertainty in diffusion-weighted MR imaging. Magn Reson Med. 2003;50(5):1077- 88.
6. Andersson JL, Skare S, Ashburner J. How to correct susceptibility distortions in spin-echo echoplanar images: application to diffusion tensor imaging. NeuroImage. 2003;20(2):870-88.
7. Andersson JLR, Sotiropoulos SN. An integrated approach to correction for off-resonance effects and subject movement in diffusion MR imaging. NeuroImage. 2016;125:1063-78.
8. Jenkinson M, Bannister P, Brady M, Smith S. Improved optimization for the robust and accurate linear registration and motion correction of brain images. NeuroImage. 2002;17(2):825-41.
9. Jesper L. R. Andersson MJaSS. Non-linear registration aka Spatial normalisation. Oxford, United Kingdom: FMRIB Centre; 2007
10. Wakana S, Caprihan A, Panzenboeck MM, Fallon JH, Perry M, Gollub RL, et al. Reproducibility of quantitative tractography methods applied to cerebral white matter. NeuroImage. 2007;36(3):630-44.
11. Hua K, Zhang J, Wakana S, Jiang H, Li X, Reich DS, et al. Tract probability maps in stereotaxic spaces: analyses of white matter anatomy and tract-speciYc quantification. NeuroImage. 2008;39(1):336-47.
12. Mori S, Oishi K, Faria A V, M ZPC. MRI Atlas of Human White Matter. Amsterdam, The Netherlands: Elsevier; 2005.
13. Shim VB, Tayebi M, Kwon E, Guild S, Scadeng M, Dubowitz D, Mcbryde F, Rosset S, Wang A, Fernandez JW, Li S, Holdsworth S. Combining advanced magnetic resonance imaging (MRI) with Ynite element (FE) analysis for characterising subject-specific injury patterns in the brain after traumatic brain injury. Engineering with Computers. 2022.
14. Shim VB, Holdsworth S, Champagne AA, Coverdale NS, Cook DJ, Lee TR, et al. Rapid Prediction of Brain Injury Pattern in mTBI by Combining FE Analysis With a Machine-Learning Based Approach. IEEE Access. 2020;8:179457-65.
15. Shim VB, Fernandez JW, Gamage PB, Regnery C, Smith DW, Gardiner BS, et al. Subject-specific finite element analysis to characterize the influence of geometry and material properties in Achilles tendon rupture. J Biomech. 2014;47(15):3598- 604.
16. Munro JT, Millar JS, Fernandez JW, Walker CG, Howie DW, Shim VB. Risk analysis of patients with an osteolytic acetabular defect after total hip arthroplasty using subject-specific finite-element modelling. Bone Joint J. 2018;100-b(11):1455-62.
17. Shim VB, Besier TF, Lloyd DG, Mithraratne K, Fernandez JF. The influence and biomechanical role of cartilage split line pattern on tibiofemoral cartilage stress distribution during the stance phase of gait. Biomech Model Mechanobiol. 2016;15(1):195-204.

See more case study projects