Neuro-JEPA: Building and Evaluating Health-System Scale Neuroimaging Foundation Models with Mixture of Experts

Topic: Brain imaging, connectivity and assessment

Magnetic resonance imaging (MRI) of the brain routinely combines complementary contrasts—including T1-weighted (T1w), T2-weighted (T2w) and FLAIR sequences—to capture anatomy and pathology. Although foundation models trained on large neuroimaging datasets have shown promise, it remains unclear whether they effectively integrate these complementary contrasts across diverse clinical settings. Here we present Neuro-JEPA, a multimodal foundation model that learns unified representations across structural brain MRIs. Neuro-JEPA combines latent predictive learning with a sparse Mixture-of-Experts architecture to encode multiple MRI contrasts within a single model. The model is pretrained on 1.55 million scans from 428,647 clinical imaging studies spanning T1w, T2w and FLAIR sequences. The model performance is evaluated on 47 downstream tasks across three independent health systems and 12 public datasets, encompassing single-contrast, multimodal and cross-domain applications. Across these evaluations, Neuro-JEPA achieved the strongest overall performance and generalized robustly across heterogeneous cohorts, outperforming existing neuroimaging foundation models and strong conventional baselines in the majority of comparisons despite using less than one-third of the pretraining data of leading models. Performance gains were particularly pronounced in data-scarce settings, reaching up to 15% over prior foundation models for clinically important pathologies including gliosis and multiple sclerosis. Neuro-JEPA showed clinically relevant capabilities across domains, including prediction of amyloid status and progression from mild cognitive impairment to Alzheimer’s disease dementia, stroke identification and classification, and prediction of post-stroke functional outcome and length of stay. Together, these results show that data scale, curation and principled model design can act synergistically to produce broadly transferable neuroimaging representations, and establish Neuro-JEPA as a scalable framework toward general-purpose multimodal foundation models for neurological imaging.

Key findings

• Across these evaluations, Neuro-JEPA achieved the strongest overall performance and generalized robustly across heterogeneous cohorts, outperforming existing neuroimaging foundation models and strong conventional baselines in the majority of comparisons despite using less than one-third of the pretraining data of leading models.
• Together, these results show that data scale, curation and principled model design can act synergistically to produce broadly transferable neuroimaging representations, and establish Neuro-JEPA as a scalable framework toward general-purpose multimodal foundation models for neurological imaging.

Study details

• Evidence strengthLow

Read the original

• DOI: 10.21203/rs.3.rs-11130462/v1
• Original source →

Free copy unknown: no free-copy check has an answer for this record yet. That does not mean it is paywalled.

@article{huang2026-e8078740,
  author  = {Huang, H. and Chen, L. and Chen, J. and Hyun, J. and Loftus, J. and Melmed, K. and Orringer, D. and Frontera, J. and others},
  title   = {{Neuro-JEPA: Building and Evaluating Health-System Scale Neuroimaging Foundation Models with Mixture of Experts}},
  journal = {Research Square},
  year    = {2026},
  doi     = {10.21203/rs.3.rs-11130462/v1},
  url     = {https://doi.org/10.21203/rs.3.rs-11130462/v1}
}

Related studies in Mild cognitive impairment

• Assessing the Preparedness of the Health Care System Infrastructure in Six European Countries for an Alzheimer's Treatment Rand health quarterly · 2019-05-16 · usefulness score 43/100
• Evaluating the effects of liraglutide, empagliflozin and linagliptin on mild cognitive impairment remission in patients with type 2 diabetes (LIGHT-MCI): study protocol for a multicentre, randomised controlled trial with an extension phase BMJ Open · 2025-08-01 · usefulness score 46/100
• Evaluating Patient Representation Strategies for Early Prediction of Alzheimer’s Disease and Related Dementias from Longitudinal EHRs Research Square · 2026-10-01 · usefulness score 16/100
• The cholinergic system in aging and neuronal degeneration Behavioural brain research · 2010-12-09 · usefulness score 51/100
• Diabetic retinopathy as the primary predictor of mild cognitive impairment in type 2 diabetes: Insights from machine learning models PLoS ONE · 2025-09-26 · usefulness score 51/100
• An integrative multi-omics approach reveals new central nervous system pathway alterations in Alzheimer's disease Alzheimer's research & therapy · 2021-04-01 · usefulness score 50/100

Also tagged “convolutional”

205 studies in the library carry the topic tag “convolutional”; these are this study's neighbours in a fixed, alphabetical-by-hash tour of them, not the closest in content. All 205 tagged studies.

• Deep representation learning of electronic health records to unlock patient stratification at scale NPJ digital medicine · 2020-07-17 · usefulness score 57/100 · Digital health tools
• Deep Learning Analysis of Retinal Structures and Risk Factors of Alzheimer's Disease Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual International Conference · 2024-07 · usefulness score 43/100 · Brain scans and imaging
• A 2-Level Stacking Ensemble Framework for Multi-Class Alzheimer’s Disease Diagnosis Using CNN Feature Extractors and SVM-XGBoost Classifiers Research Square · 2026-09-07 · usefulness score 16/100 · Long-term cohort studies

Similar studies

• Multimodal deep learning models for early detection of Alzheimer's disease stage 2021 · Digital health tools
• Multimodal neuroimaging feature learning for multiclass diagnosis of Alzheimer's disease 2015 · Digital health tools
• Brain MRI analysis for Alzheimer's disease diagnosis using an ensemble system of deep convolutional neural networks 2018 · Digital health tools
• Multimodal deep learning for Alzheimer's disease dementia assessment 2022 · Digital health tools
• Investigating the Amyloid-Tau-Neurodegeneration Framework in Alzheimer’s Disease Using Semi-Supervised Multimodal Imaging Data Fusion 2025 · Brain scans and imaging
• Hierarchical feature representation and multimodal fusion with deep learning for AD/MCI diagnosis 2014 · Brain scans and imaging

More on Mild cognitive impairment

• A review that covers this ground Oxidatively modified proteins in Alzheimer's disease (AD), mild cognitive impairment and animal models of AD: role of Abeta in pathogenesis Acta neuropathologica · 2009-03-14 · usefulness score 48/100
• Newest in this subject Paths of Aging Minds: Health Journeys of Older Adults with Mild Cognitive Impairment Research Square · 2026-11-01 · Preprint — not yet peer-reviewed · usefulness score 16/100
• Newest in this subject Neuroinflammation is an early pathological process in a mouse model of prodromal Parkinsons disease bioRxiv · 2026-10-30 · Preprint — not yet peer-reviewed · usefulness score 16/100

See all studies in Mild cognitive impairment →