Explainable machine learning for predicting longitudinal dementia status: Establishing a leakage-free benchmark

Topic: Risk, prevention and care in population studies

Dementia research often suffers from methodological pitfalls such as label-information and subject-information leakages. Leveraging the longitudinal OASIS-2 cohort, this study identifies and addresses three critical gaps in prior research: (i) inclusion of Clinical Dementia Rating (CDR) in predictor sets, risking target leakage and inflated performance estimates; (ii) inappropriate splitting strategies for longitudinal data, risking information leakage; and (iii) inadequate incorporation of temporal dynamics in classification models. To address these, CDR was excluded from input features to ensure that model performance reflects genuine predictive signals from demographic and neuroimaging data. A group-aware data-splitting strategy was implemented to maintain longitudinal data integrity and prevent leakage across training and test sets. Several engineered temporal features were also introduced to evaluate their impact on classification performance. To develop classifiers, a kernel-based classifier (support vector classifier, SVC), boosting ensembles (CatBoost, LightGBM), and bagging ensembles (Random Forest, Extra Trees), were employed. Furthermore, Explainable AI (XAI) techniques, including permutation importance and SHapley Additive exPlanations (SHAP), were utilized to interpret predictions of the best models and identify feature contributions. SVC and LightGBM (LGBM), trained on a combined feature set of original and engineered features, outperformed others. SVC achieved higher precision (73.3%) and discriminative power (ROC AUC 89.1%), while LightGBM provided better recall (69.7%) and accuracy (69.7%). XAI analyses revealed that (i) Mini-Mental State Examination and atlas-scaling-factor dynamics dominated SVC decisions, while (ii) education, age, and estimated total intracranial volume were critical for LGBM predictions. A sex-based analysis showed that both SVC and LGBM models performed better among females, which may indicate potential biases in the models. Collectively, this study establishes a transparent benchmark for leakage-free dementia status classification, demonstrating that realistic predictive performance is markedly lower once leakage is eliminated. This benchmark, together with interpretable analyses, provides a reference point for developing more robust longitudinal dementia models.

Key findings

• XAI analyses revealed that (i) Mini-Mental State Examination and atlas-scaling-factor dynamics dominated SVC decisions, while (ii) education, age, and estimated total intracranial volume were critical for LGBM predictions.
• A sex-based analysis showed that both SVC and LGBM models performed better among females, which may indicate potential biases in the models.

Why it matters

This work sets up a leakage-free benchmark for predicting dementia status over time using demographic and neuroimaging data, after excluding the Clinical Dementia Rating from the predictors. It is a methods-focused study and does not test any treatment, prevention or cure.

New to this subject? Start with our plain-English guide: Digital health →

Study details

• Study designlongitudinal study
• Effect size73.3%; AUC 89.1; 69.7%
• Evidence strengthModerate

Read the original

• DOI: 10.1371/journal.pdig.0001409
• PubMed: 42149898
• Original source →
• Free full text: PLOS Digital Health →

Full text: free to read — OpenAlex lists this copy as free to read: the link above (checked 6 Feb 1020).

@article{ghiasi2026-42149898,
  author  = {Ghiasi, Mohammad Mahdi and Falck, Ryan S. and Liu-Ambrose, Teresa Y L and Tam, Roger C},
  title   = {{Explainable machine learning for predicting longitudinal dementia status: Establishing a leakage-free benchmark}},
  journal = {PLOS Digital Health},
  year    = {2026},
  doi     = {10.1371/journal.pdig.0001409},
  pmid    = {42149898},
  url     = {https://doi.org/10.1371/journal.pdig.0001409}
}

Studies this one cites

Referenced by this study and also in the alzhub library, so you can read them here.

• Predicting Alzheimer's disease progression using deep recurrent neural networks NeuroImage · 2020-08-04 · usefulness score 65/100 · Long-term cohort studies
• Early-Stage Alzheimer's Disease Prediction Using Machine Learning Models Frontiers in public health · 2022-03-03 · usefulness score 53/100 · Digital health tools

Related studies in Digital health tools

• Machine learning model for predicting Amyloid-β positivity and cognitive status using early-phase 18F-Florbetaben PET and clinical features Scientific Reports · 2025-07-01 · usefulness score 50/100
• The AHEPA EEG benchmark: setting the standard for machine learning in dementia diagnosis, a scoping review Cognitive Neurodynamics · 2026-05-18 · usefulness score 36/100
• Imputation-Based Harmonization Mitigates Site Effects Without Data Leakage in Machine Learning Studies bioRxiv · 2026-10-05 · usefulness score 16/100
• Resource-stratified machine learning framework for cognitive status classification and mild cognitive impairment to dementia progression prediction Alzheimer s Research & Therapy · 2026-03-07 · usefulness score 46/100
• Predicting explainable dementia types with LLM-aided feature engineering Bioinformatics · 2025-03-29 · usefulness score 45/100
• An explainable machine learning approach for Alzheimer's disease classification Scientific reports · 2024-02-01 · usefulness score 56/100

Similar studies

• Leveraging Explainable AI for Dementia Classification: A Machine Learning Approach 2025 · Digital health tools
• Machine learning diagnosis of cognitive impairment and dementia in harmonized older adult cohorts 2025 · Digital health tools
• Explainable Machine Learning-Based Classification of Clinical Dementia Ratings Using a Tablet-Based Cognitive Assessment 2025 · Getting a diagnosis
• Leveraging explainable artificial intelligence with ensemble of deep learning model for dementia prediction to enhance clinical decision support systems 2025 · Digital health tools
• Machine learning methods for predicting progression from mild cognitive impairment to Alzheimer's disease dementia: a systematic review 2021 · Digital health tools
• Machine Learning for Dementia Prediction: A Systematic Review and Future Research Directions 2023 · Digital health tools

More on Digital health tools

• A review that covers this ground Predicting the progression of mild cognitive impairment using machine learning: A systematic, quantitative and critical review Medical image analysis · 2020-10-06 · usefulness score 70/100
• Newest in this subject Speaker-Wise Dementia Screening fromNaturalistic Speech with Lexical Entropy, Semantic Drift, and Multimodal Fusion Research Square · 2026-10-21 · Preprint — not yet peer-reviewed · usefulness score 16/100
• Newest in this subject Integrating Multisource Target Evidence with Machine-Learning Potency Prediction for Alzheimer’s Disease Drug Discovery bioRxiv · 2026-10-20 · Preprint — not yet peer-reviewed · usefulness score 16/100

See all studies in Digital health tools →