Voxel-Wise Fusion of Resting fMRI Networks and Gray Matter Volume for Alzheimer's Disease Classification Using Deep Multimodal Learning

Abstract

Alzheimer's disease (AD) is a prevalent neurodegenerative disorder requiring accurate and early diagnosis to support clinical decision-making and future intervention strategies. Resting-state functional magnetic resonance imaging (rs-fMRI) and gray matter volume analysis from structural MRI (sMRI) are promising tools for identifying AD-related brain alterations. In this study, we propose a novel deep learning framework that fuses spatial maps of rs-fMRI functional networks—via a new voxelwise intensity projection (iVIP) method—and gray matter segmentations to classify subjects from the Alzheimer’s Disease Neuroimaging Initiative (ADNI). Our multi-channel 3D convolutional neural network (CNN), inspired by the AlexNet architecture, integrates anatomical and functional modalities to perform both two-way (AD vs. CN) and three-way (AD vs. MCI vs. CN) classification tasks. While the two-way classification achieves high performance (94.12% accuracy, 97.79 AUC), the three-way classification offers clinically relevant utility, achieving 62.68% accuracy and 68.31 AUC, and enables finer differentiation among diagnostic subtypes. Comparative analysis shows that our iVIP representation outperforms traditional rs-fMRI measures such as ALFF and fALFF, and the fusion of iVIP with sMRI consistently outperforms unimodal models. Saliency map visualizations further reveal discriminative regions—including the hippocampus, amygdala, caudate nucleus, and thalamus—aligned with AD pathology. Our findings highlight the potential of deep multimodal fusion to enhance early diagnosis and subtype differentiation in Alzheimer’s disease.

Note:
Funding declaration: This work was supported by NIH (Grant number: RF1AG063153) and NSF (Grant number: 2112455). Data collection and sharing for this project was funded by the Alzheimer's Disease Neuroimaging Initiative (ADNI) (National Institutes of Health Grant U01 AG024904) and DOD ADNI (Department of Defense award number W81XWH-12-2-0012). ADNI is funded by the National Institute on Aging, the National Institute of Biomedical Imaging and Bioengineering, and through generous contributions from the following: AbbVie, Alzheimer’s Association; Alzheimer’s Drug Discovery Foundation; Araclon Biotech; BioClinica, Inc.; Biogen; Bristol-Myers Squibb Company; CereSpir, Inc.; Cogstate; Eisai Inc.; Elan Pharmaceuticals, Inc.; Eli Lilly and Company; EuroImmun; F. Hoffmann-La Roche Ltd and its affiliated company Genentech, Inc.; Fujirebio; GE Healthcare; IXICO Ltd.; Janssen Alzheimer Immunotherapy Research & Development, LLC.; Johnson & Johnson Pharmaceutical Research & Development LLC.; Lumosity; Lundbeck; Merck & Co., Inc.; Meso Scale Diagnostics, LLC.; NeuroRx Research; Neurotrack Technologies; Novartis Pharmaceuticals Corporation; Pfizer Inc.; Piramal Imaging; Servier; Takeda Pharmaceutical Company; and Transition Therapeutics. The Canadian Institutes of Health Research is providing funds to support ADNI clinical sites in Canada. Private sector contributions are facilitated by the Foundation for the National Institutes of Health (www.fnih.org). The grantee organization is the Northern California Institute for Research and Education, and the study is coordinated by the Alzheimer’s Therapeutic Research Institute at the University of Southern California. ADNI data are disseminated by the Laboratory for Neuro Imaging at the University of Southern California.

Conflict of Interests: The authors have no relevant financial or non-financial interests to disclose.

Ethical Approval: The research protocols of the ADNI study were subject to approval by the institutional review boards at all participating centers, and this approval process is meticulously detailed in the official document, which can be accessed via this link: ADNI Approval Documentation.

Keywords: Independent component analysis, resting-state fMRI, deep learning, sMRI, Alzheimer's disease, classification