Transformer-based deep neural network language models for Alzheimer's disease risk assessment from targeted speech

Topic: Brain imaging, connectivity and assessment

Background We developed transformer-based deep learning models based on natural language processing for early risk assessment of Alzheimer's disease from the picture description test. Methods The lack of large datasets poses the most important limitation for using complex models that do not require feature engineering. Transformer-based pre-trained deep language models have recently made a large leap in NLP research and application. These models are pre-trained on available large datasets to understand natural language texts appropriately, and are shown to subsequently perform well on classification tasks with small training sets. The overall classification model is a simple classifier on top of the pre-trained deep language model. Results The models are evaluated on picture description test transcripts of the Pitt corpus, which contains data of 170 AD patients with 257 interviews and 99 healthy controls with 243 interviews. The large bidirectional encoder representations from transformers (BERT Large ) embedding with logistic regression classifier achieves classification accuracy of 88.08%, which improves the state-of-the-art by 2.48%. Conclusions Using pre-trained language models can improve AD prediction. This not only solves the problem of lack of sufficiently large datasets, but also reduces the need for expert-defined features.

@article{roshanzamir2021-33750385,
  author  = {Roshanzamir, A. and Aghajan, H. and Soleymani Baghshah, M.},
  title   = {{Transformer-based deep neural network language models for Alzheimer's disease risk assessment from targeted speech}},
  journal = {BMC medical informatics and decision making},
  year    = {2021},
  doi     = {10.1186/s12911-021-01456-3},
  pmid    = {33750385},
  url     = {https://doi.org/10.1186/s12911-021-01456-3}
}

Key findings

• The overall classification model is a simple classifier on top of the pre-trained deep language model.
• Results The models are evaluated on picture description test transcripts of the Pitt corpus, which contains data of 170 AD patients with 257 interviews and 99 healthy controls with 243 interviews.

New to this subject? Start with our plain-English guide: Digital health →

Study details

• Effect size88.08%; 2.48%
• Evidence strengthLow

Read the original

• DOI: 10.1186/s12911-021-01456-3
• PubMed: 33750385
• Original source →

Free copy unknown: no free-copy check has an answer for this record yet. That does not mean it is paywalled.

Related studies in Digital health tools

• Layer-Wise Relevance Propagation for Explaining Deep Neural Network Decisions in MRI-Based Alzheimer's Disease Classification Frontiers in aging neuroscience · 2019-07-31 · usefulness score 41/100
• Classification of Alzheimer's Disease Based on Eight-Layer Convolutional Neural Network with Leaky Rectified Linear Unit and Max Pooling Journal of medical systems · 2018-03-26 · usefulness score 41/100
• Identification of Mild Cognitive Impairment From Speech in Swedish Using Deep Sequential Neural Networks Frontiers in neurology · 2018-11-15 · usefulness score 42/100
• Correlating natural language processing and automated speech analysis with clinician assessment to quantify speech-language changes in mild cognitive impairment and Alzheimer's dementia Alzheimer's research & therapy · 2021-06-04 · usefulness score 51/100
• Prediction of Alzheimer's disease progression within 6 years using speech: A novel approach leveraging language models Alzheimer's & dementia : the journal of the Alzheimer's Association · 2024-06-25 · usefulness score 57/100
• Predicting dementia from spontaneous speech using large language models PLOS digital health · 2022-12-22 · usefulness score 46/100

Also tagged “convolutional”

205 studies in the library carry the topic tag “convolutional”; these are this study's neighbours in a fixed, alphabetical-by-hash tour of them, not the closest in content. All 205 tagged studies.

• Multimodal Retinal Imaging Classification for Parkinson's Disease Using a Convolutional Neural Network Translational vision science & technology · 2024-08-01 · usefulness score 43/100 · Brain scans and imaging
• Classification and Visualization of Alzheimer’s Disease using Volumetric Convolutional Neural Network and Transfer Learning Scientific Reports · 2019-12-03 · usefulness score 45/100 · Brain scans and imaging
• A Machine Learning-Based Framework for differential screening of Neurodegenerative Dementias using routine clinical data Research Square · 2026-10-07 · usefulness score 16/100 · Getting a diagnosis

Similar studies

• AD-BERT: Using pre-trained language model to predict the progression from mild cognitive impairment to Alzheimer's disease 2023 · Digital health tools
• Alzheimer's Disease Detection via Large Pre-Trained Speech and Language Models: Taxonomy, Progress, and Open Challenges 2026 · Digital health tools
• Predicting Cognitive Function Using Transformer-Derived Speech Representations and Longitudinal Coherence Features 2026 · Cognitive tests and assessment
• Small-Sized Reasoning Language Models for Linguistic Screening of Alzheimer’s Disease 2026 · Digital health tools
• nterpretable Alzheimer’s Disease Detection from CHAT Transcripts Using Criterion-Guided LLM Prompting and In-Context Learning 2026 · Digital health tools
• PaliGemX: An Interpretable Multimodal Transformer for Early Alzheimer’s Detection 2026 · Digital health tools

More on Digital health tools

• A review that covers this ground Deep learning-based speech analysis for Alzheimer's disease detection: a literature review Alzheimer's research & therapy · 2022-12-14 · usefulness score 43/100
• Newest in this subject Speaker-Wise Dementia Screening fromNaturalistic Speech with Lexical Entropy, Semantic Drift, and Multimodal Fusion Research Square · 2026-10-21 · Preprint — not yet peer-reviewed · usefulness score 16/100
• Newest in this subject Integrating Multisource Target Evidence with Machine-Learning Potency Prediction for Alzheimer’s Disease Drug Discovery bioRxiv · 2026-10-20 · Preprint — not yet peer-reviewed · usefulness score 16/100

See all studies in Digital health tools →