PaliGemX: An Interpretable Multimodal Transformer for Early Alzheimer’s Detection

Topic: Brain imaging, connectivity and assessment

Alzheimer disease (AD) is a disorder of the nervous system that mostly impacts the brain cells and causes the slow destruction of the brain. Previously, majority of the methods to identify this condition were based on a single kind of data and either conventional machine learning or deep learning models. Nonetheless, the recent progress in artificial intelligence has allowed incorporating multimodal information on numerous forms of input, which have played a major role in predicting and cognizing the disease Alzheimer. This study presents a multi-modal vision transformer architecture that integrates co-attention techniques and Pali Gamma, which achieves best performance in Alzheimer’s diseasePali Gemma can be applied to aid more informed clinical decision-making and improve the results of patients with Alzheimer disease in terms of diagnosing and treating the disease through continuous fusion of visual and text inputs. The framework emphasizes interpretability through relevance-based attention processes and incorporates loss functions that improve accuracy, scalability, resilience, and computing economy. Experimental findings indicate the model's enhanced classification accuracy and resilience across various datasets, representing a crucial advancement towards comprehensive, interpretable, and scalable solutions for Alzheimer's disease diagnosis. The proposed model provides a framework for incorporating other modalities, resolving ethical issues, and improving empirical validation in various clinical environments.

Key findings

• This study presents a multi-modal vision transformer architecture that integrates co-attention techniques and Pali Gamma, which achieves best performance in Alzheimer’s diseasePali Gemma can be applied to aid more informed clinical decision-making and improve the results of patients with Alzheimer disease in terms of diagnosing and treating the disease through continuous fusion of visual and text inputs.

New to this subject? Start with our plain-English guide: Digital health →

Study details

• Evidence strengthLow

Read the original

• DOI: 10.21203/rs.3.rs-11122884/v1
• Original source →

Free copy unknown: no free-copy check has an answer for this record yet. That does not mean it is paywalled.

@article{suganya2026-3d051a9b,
  author  = {Suganya, S. and Prasath, A. R.},
  title   = {{PaliGemX: An Interpretable Multimodal Transformer for Early Alzheimer’s Detection}},
  journal = {Research Square},
  year    = {2026},
  doi     = {10.21203/rs.3.rs-11122884/v1},
  url     = {https://doi.org/10.21203/rs.3.rs-11122884/v1}
}

Related studies in Digital health tools

• Multimodal deep learning models for early detection of Alzheimer's disease stage Scientific reports · 2021-02-05 · usefulness score 43/100
• DICE-Net: A Novel Convolution-Transformer Architecture for Alzheimer Detection in EEG Signals IEEE Access · 2023-01-01 · usefulness score 45/100
• A multilayer multimodal detection and prediction model based on explainable artificial intelligence for Alzheimer's disease Scientific reports · 2021-01-29 · usefulness score 76/100
• Multimodal Detection of Agitation in People With Dementia in Clinical Settings: Observational Pilot Study JMIR Aging · 2025-07-15 · usefulness score 41/100
• Multimodal and Multiscale Deep Neural Networks for the Early Diagnosis of Alzheimer's Disease using structural MR and FDG-PET images Scientific reports · 2018-04-09 · usefulness score 45/100
• Combined with multimodal medical imaging and artificial intelligence for early diagnosis of Parkinson's disease Frontiers in Neurology · 2025-10-01 · usefulness score 40/100

Also tagged “convolutional”

205 studies in the library carry the topic tag “convolutional”; these are this study's neighbours in a fixed, alphabetical-by-hash tour of them, not the closest in content. All 205 tagged studies.

• Automatic Segmentation of Retinal Layers in Multiple Neurodegenerative Disorder Scenarios IEEE journal of biomedical and health informatics · 2023-11-07 · usefulness score 41/100 · Brain scans and imaging
• Using Support Vector Machine to identify imaging biomarkers of neurological and psychiatric disease: a critical review Neuroscience and biobehavioral reviews · 2012-01-28 · usefulness score 36/100 · Risk factors
• Unlocking Sensitive Data with SPHERE in the Age of AI bioRxiv · 2026-09-05 · usefulness score 16/100 · Long-term cohort studies

Similar studies

• Multimodal deep learning for Alzheimer's disease dementia assessment 2022 · Digital health tools
• Multimodal neuroimaging feature learning for multiclass diagnosis of Alzheimer's disease 2015 · Digital health tools
• Multimodal Transformer-Based Fusion of Blood-Based and Digital Biomarkers for Explainable (XAI) Ultra-Early Alzheimer's Disease Detection 2026 · Brain scans and imaging
• Artificial intelligence in dementia 2022 · Brain scans and imaging
• Convolutional neural network to identify symptomatic Alzheimer's disease using multimodal retinal imaging 2022 · Brain scans and imaging
• Early detection of dementia through retinal imaging and trustworthy AI 2024 · Brain scans and imaging

More on Digital health tools

• A review that covers this ground Digital biomarkers for early agitation detection in dementia: a scoping review of emerging wearable and smart technologies for personalized care Frontiers in Neurology · 2026-02-10 · usefulness score 56/100
• Newest in this subject Speaker-Wise Dementia Screening fromNaturalistic Speech with Lexical Entropy, Semantic Drift, and Multimodal Fusion Research Square · 2026-10-21 · Preprint — not yet peer-reviewed · usefulness score 16/100
• Newest in this subject Integrating Multisource Target Evidence with Machine-Learning Potency Prediction for Alzheimer’s Disease Drug Discovery bioRxiv · 2026-10-20 · Preprint — not yet peer-reviewed · usefulness score 16/100

See all studies in Digital health tools →