Multimodal deep learning models for early detection of Alzheimer's disease stage

Topic: Brain imaging, connectivity and assessment

Most current Alzheimer's disease (AD) and mild cognitive disorders (MCI) studies use single data modality to make predictions such as AD stages. The fusion of multiple data modalities can provide a holistic view of AD staging analysis. Thus, we use deep learning (DL) to integrally analyze imaging (magnetic resonance imaging (MRI)), genetic (single nucleotide polymorphisms (SNPs)), and clinical test data to classify patients into AD, MCI, and controls (CN). We use stacked denoising auto-encoders to extract features from clinical and genetic data, and use 3D-convolutional neural networks (CNNs) for imaging data. We also develop a novel data interpretation method to identify top-performing features learned by the deep-models with clustering and perturbation analysis. Using Alzheimer's disease neuroimaging initiative (ADNI) dataset, we demonstrate that deep models outperform shallow models, including support vector machines, decision trees, random forests, and k-nearest neighbors. In addition, we demonstrate that integrating multi-modality data outperforms single modality models in terms of accuracy, precision, recall, and meanF1 scores. Our models have identified hippocampus, amygdala brain areas, and the Rey Auditory Verbal Learning Test (RAVLT) as top distinguished features, which are consistent with the known AD literature.

Key findings

• We use stacked denoising auto-encoders to extract features from clinical and genetic data, and use 3D-convolutional neural networks (CNNs) for imaging data.
• We also develop a novel data interpretation method to identify top-performing features learned by the deep-models with clustering and perturbation analysis.
• Using Alzheimer's disease neuroimaging initiative (ADNI) dataset, we demonstrate that deep models outperform shallow models, including support vector machines, decision trees, random forests, and k-nearest neighbors.
• In addition, we demonstrate that integrating multi-modality data outperforms single modality models in terms of accuracy, precision, recall, and meanF1 scores.
• Our models have identified hippocampus, amygdala brain areas, and the Rey Auditory Verbal Learning Test (RAVLT) as top distinguished features, which are consistent with the known AD literature.

New to this subject? Start with our plain-English guide: Digital health →

Study details

• Evidence strengthLow

Read the original

• DOI: 10.1038/s41598-020-74399-w
• PubMed: 33547343
• Original source →

Free copy unknown: no free-copy check has an answer for this record yet. That does not mean it is paywalled.

@article{venugopalan2021-33547343,
  author  = {Venugopalan, J. and Tong, L. and Hassanzadeh, H. R. and Wang, M. D.},
  title   = {{Multimodal deep learning models for early detection of Alzheimer's disease stage}},
  journal = {Scientific reports},
  year    = {2021},
  doi     = {10.1038/s41598-020-74399-w},
  pmid    = {33547343},
  url     = {https://doi.org/10.1038/s41598-020-74399-w}
}

Related studies in Digital health tools

• Early-Stage Alzheimer's Disease Prediction Using Machine Learning Models Frontiers in public health · 2022-03-03 · usefulness score 53/100
• Multimodal and Multiscale Deep Neural Networks for the Early Diagnosis of Alzheimer's Disease using structural MR and FDG-PET images Scientific reports · 2018-04-09 · usefulness score 45/100
• PaliGemX: An Interpretable Multimodal Transformer for Early Alzheimer’s Detection Research Square · 2026-10-14 · usefulness score 16/100
• A machine learning framework for early-stage dementia-friendly architectural design evaluation using visual access metrics Architectural Intelligence · 2025-07-10 · usefulness score 29/100
• Multimodal deep learning for Alzheimer's disease dementia assessment Nature communications · 2022-06-20 · usefulness score 47/100
• Multimodal Neuroimaging Feature Learning With Multimodal Stacked Deep Polynomial Networks for Diagnosis of Alzheimer's Disease IEEE journal of biomedical and health informatics · 2017-01-19 · usefulness score 38/100

Also tagged “convolutional”

205 studies in the library carry the topic tag “convolutional”; these are this study's neighbours in a fixed, alphabetical-by-hash tour of them, not the closest in content. All 205 tagged studies.

• Frequency-Resolved Connectivity Graphs for EEG-Based Classification of Alzheimer and Frontotemporal Dementia bioRxiv · 2026-10-22 · usefulness score 16/100 · Lewy body dementia
• DEMNET: A Deep Learning Model for Early Diagnosis of Alzheimer Diseases and Dementia From MR Images IEEE Access · 2021-01-01 · usefulness score 45/100 · Long-term cohort studies

Similar studies

• Deep learning to detect Alzheimer's disease from neuroimaging: A systematic literature review 2020 · Digital health tools
• Deep Learning in Alzheimer's Disease: Diagnostic Classification and Prognostic Prediction Using Neuroimaging Data 2019 · Digital health tools
• Automated classification of Alzheimer's disease and mild cognitive impairment using a single MRI and deep neural networks 2019 · Digital health tools
• Brain MRI analysis for Alzheimer's disease diagnosis using an ensemble system of deep convolutional neural networks 2018 · Digital health tools
• Deep learning-based feature representation for AD/MCI classification 2013 · Digital health tools
• Multimodal neuroimaging feature learning for multiclass diagnosis of Alzheimer's disease 2015 · Digital health tools

More on Digital health tools

• A review that covers this ground Deep learning-based speech analysis for Alzheimer's disease detection: a literature review Alzheimer's research & therapy · 2022-12-14 · usefulness score 43/100
• Newest in this subject Speaker-Wise Dementia Screening fromNaturalistic Speech with Lexical Entropy, Semantic Drift, and Multimodal Fusion Research Square · 2026-10-21 · Preprint — not yet peer-reviewed · usefulness score 16/100
• Newest in this subject Integrating Multisource Target Evidence with Machine-Learning Potency Prediction for Alzheimer’s Disease Drug Discovery bioRxiv · 2026-10-20 · Preprint — not yet peer-reviewed · usefulness score 16/100

See all studies in Digital health tools →