AD-BERT: Using pre-trained language model to predict the progression from mild cognitive impairment to Alzheimer's disease

Topic: Brain imaging, connectivity and assessment

Objective We develop a deep learning framework based on the pre-trained Bidirectional Encoder Representations from Transformers (BERT) model using unstructured clinical notes from electronic health records (EHRs) to predict the risk of disease progression from Mild Cognitive Impairment (MCI) to Alzheimer's Disease (AD). Methods We identified 3657 patients diagnosed with MCI together with their progress notes from Northwestern Medicine Enterprise Data Warehouse (NMEDW) between 2000 and 2020. The progress notes no later than the first MCI diagnosis were used for the prediction. We first preprocessed the notes by deidentification, cleaning and splitting into sections, and then pre-trained a BERT model for AD (named AD-BERT) based on the publicly available Bio+Clinical BERT on the preprocessed notes. All sections of a patient were embedded into a vector representation by AD-BERT and then combined by global MaxPooling and a fully connected network to compute the probability of MCI-to-AD progression. For validation, we conducted a similar set of experiments on 2563 MCI patients identified at Weill Cornell Medicine (WCM) during the same timeframe. Results Compared with the 7 baseline models, the AD-BERT model achieved the best performance on both datasets, with Area Under receiver operating characteristic Curve (AUC) of 0.849 and F1 score of 0.440 on NMEDW dataset, and AUC of 0.883 and F1 score of 0.680 on WCM dataset. Conclusion The use of EHRs for AD-related research is promising, and AD-BERT shows superior predictive performance in modeling MCI-to-AD progression prediction. Our study demonstrates the utility of pre-trained language models and clinical notes in predicting MCI-to-AD progression, which could have important implications for improving early detection and intervention for AD.

Key findings

• Results Compared with the 7 baseline models, the AD-BERT model achieved the best performance on both datasets, with Area Under receiver operating characteristic Curve (AUC) of 0.849 and F1 score of 0.440 on NMEDW dataset, and AUC of 0.883 and F1 score of 0.680 on WCM dataset.

New to this subject? Start with our plain-English guide: Digital health →

Study details

• Participants3,657
• Evidence strengthLow

Read the original

• DOI: 10.1016/j.jbi.2023.104442
• PubMed: 37429512
• Original source →

Free copy unknown: no free-copy check has an answer for this record yet. That does not mean it is paywalled.

@article{mao2023-37429512,
  author  = {Mao, C. and Xu, J. and Rasmussen, L. and Li, Y. and Adekkanattu, P. and Pacheco, J. and Bonakdarpour, B. and Vassar, R. and others},
  title   = {{AD-BERT: Using pre-trained language model to predict the progression from mild cognitive impairment to Alzheimer's disease}},
  journal = {Journal of biomedical informatics},
  year    = {2023},
  doi     = {10.1016/j.jbi.2023.104442},
  pmid    = {37429512},
  url     = {https://doi.org/10.1016/j.jbi.2023.104442}
}

Related studies in Digital health tools

• Cortical Thickness from MRI to Predict Conversion from Mild Cognitive Impairment to Dementia in Parkinson Disease: A Machine Learning-based Model Radiology · 2021-05-25 · usefulness score 46/100
• Alzheimer's Disease Detection via Large Pre-Trained Speech and Language Models: Taxonomy, Progress, and Open Challenges PsyArXiv · 2026-09-03 · usefulness score 16/100
• A Novel Ensemble-Based Machine Learning Algorithm to Predict the Conversion From Mild Cognitive Impairment to Alzheimer's Disease Using Socio-Demographic Characteristics, Clinical Information, and Neuropsychological Measures Frontiers in neurology · 2019-07-16 · usefulness score 51/100
• Predicting the progression of mild cognitive impairment using machine learning: A systematic, quantitative and critical review Medical image analysis · 2020-10-06 · usefulness score 70/100
• Developing a machine learning model for detecting depression, anxiety, and apathy in older adults with mild cognitive impairment using speech and facial expressions: A cross-sectional observational study International journal of nursing studies · 2023-07-07 · usefulness score 52/100
• Machine learning methods for predicting progression from mild cognitive impairment to Alzheimer's disease dementia: a systematic review Alzheimer's research & therapy · 2021-09-28 · usefulness score 65/100

Similar studies

• Transformer-based deep neural network language models for Alzheimer's disease risk assessment from targeted speech 2021 · Digital health tools
• Evaluating Patient Representation Strategies for Early Prediction of Alzheimer’s Disease and Related Dementias from Longitudinal EHRs 2026 · Mild cognitive impairment
• Predicting Cognitive Function Using Transformer-Derived Speech Representations and Longitudinal Coherence Features 2026 · Cognitive tests and assessment
• Deep representation learning of electronic health records to unlock patient stratification at scale 2020 · Digital health tools
• A retrieval-augmented generation large language model framework for accurate dementia identification from electronic health records 2026 · Digital health tools
• Small-Sized Reasoning Language Models for Linguistic Screening of Alzheimer’s Disease 2026 · Digital health tools

More on Digital health tools

• A review that covers this ground Connected speech and language in mild cognitive impairment and Alzheimer's disease: A review of picture description tasks Journal of clinical and experimental neuropsychology · 2018-04-19 · usefulness score 53/100
• Newest in this subject Speaker-Wise Dementia Screening fromNaturalistic Speech with Lexical Entropy, Semantic Drift, and Multimodal Fusion Research Square · 2026-10-21 · Preprint — not yet peer-reviewed · usefulness score 16/100
• Newest in this subject Integrating Multisource Target Evidence with Machine-Learning Potency Prediction for Alzheimer’s Disease Drug Discovery bioRxiv · 2026-10-20 · Preprint — not yet peer-reviewed · usefulness score 16/100

See all studies in Digital health tools →