Detection of dementia on voice recordings using deep learning: a Framingham Heart Study

Topic: Risk, prevention and care in population studies

Background Identification of reliable, affordable, and easy-to-use strategies for detection of dementia is sorely needed. Digital technologies, such as individual voice recordings, offer an attractive modality to assess cognition but methods that could automatically analyze such data are not readily available. Methods and findings We used 1264 voice recordings of neuropsychological examinations administered to participants from the Framingham Heart Study (FHS), a community-based longitudinal observational study. The recordings were 73 min in duration, on average, and contained at least two speakers (participant and examiner). Of the total voice recordings, 483 were of participants with normal cognition (NC), 451 recordings were of participants with mild cognitive impairment (MCI), and 330 were of participants with dementia (DE). We developed two deep learning models (a two-level long short-term memory (LSTM) network and a convolutional neural network (CNN)), which used the audio recordings to classify if the recording included a participant with only NC or only DE and to differentiate between recordings corresponding to those that had DE from those who did not have DE (i.e., NDE (NC+MCI)). Based on 5-fold cross-validation, the LSTM model achieved a mean (±std) area under the receiver operating characteristic curve (AUC) of 0.740 ± 0.017, mean balanced accuracy of 0.647 ± 0.027, and mean weighted F1 score of 0.596 ± 0.047 in classifying cases with DE from those with NC. The CNN model achieved a mean AUC of 0.805 ± 0.027, mean balanced accuracy of 0.743 ± 0.015, and mean weighted F1 score of 0.742 ± 0.033 in classifying cases with DE from those with NC. For the task related to the classification of participants with DE from NDE, the LSTM model achieved a mean AUC of 0.734 ± 0.014, mean balanced accuracy of 0.675 ± 0.013, and mean weighted F1 score of 0.671 ± 0.015. The CNN model achieved a mean AUC of 0.746 ± 0.021, mean balanced accuracy of 0.652 ± 0.020, and mean weighted F1 score of 0.635 ± 0.031 in classifying cases with DE from those who were NDE. Conclusion This proof-of-concept study demonstrates that automated deep learning-driven processing of audio recordings of neuropsychological testing performed on individuals recruited within a community cohort setting can facilitate dementia screening.

@article{xue2021-34465384,
  author  = {Xue, C. and Karjadi, C. and Paschalidis, I. C. and Au, R. and Kolachalama, V. B.},
  title   = {{Detection of dementia on voice recordings using deep learning: a Framingham Heart Study}},
  journal = {Alzheimer's research \& therapy},
  year    = {2021},
  doi     = {10.1186/s13195-021-00888-3},
  pmid    = {34465384},
  url     = {https://doi.org/10.1186/s13195-021-00888-3}
}

Key findings

• Based on 5-fold cross-validation, the LSTM model achieved a mean (±std) area under the receiver operating characteristic curve (AUC) of 0.740 ± 0.017, mean balanced accuracy of 0.647 ± 0.027, and mean weighted F1 score of 0.596 ± 0.047 in classifying cases with DE from those with NC.
• The CNN model achieved a mean AUC of 0.805 ± 0.027, mean balanced accuracy of 0.743 ± 0.015, and mean weighted F1 score of 0.742 ± 0.033 in classifying cases with DE from those with NC.
• For the task related to the classification of participants with DE from NDE, the LSTM model achieved a mean AUC of 0.734 ± 0.014, mean balanced accuracy of 0.675 ± 0.013, and mean weighted F1 score of 0.671 ± 0.015.
• The CNN model achieved a mean AUC of 0.746 ± 0.021, mean balanced accuracy of 0.652 ± 0.020, and mean weighted F1 score of 0.635 ± 0.031 in classifying cases with DE from those who were NDE.
• Conclusion This proof-of-concept study demonstrates that automated deep learning-driven processing of audio recordings of neuropsychological testing performed on individuals recruited within a community cohort setting can facilitate dementia screening.

Why it matters

In 1,264 audio recordings of memory-and-thinking test sessions from the Framingham Heart Study, two deep learning models were trained to tell recordings of people with dementia apart from those without it. The best model separated dementia from normal cognition with an AUC of 0.805 (balanced accuracy 0.743), but accuracy fell when people with mild cognitive impairment were counted in the non-dementia group (best AUC 0.746). The authors call it a proof-of-concept for automated screening from recordings averaging 73 minutes; it is not a validated diagnostic test and reports no treatment or prevention.

New to this subject? Start with our plain-English guide: Digital health →

Study details

• Study designlongitudinal study
• Evidence strengthModerate

Read the original

• DOI: 10.1186/s13195-021-00888-3
• PubMed: 34465384
• Original source →
• Free full text: Alzheimer s Research & Therapy →

Full text: free to read — OpenAlex lists this copy as free to read: the link above (checked 6 Feb 1020).

Studies this one cites

Referenced by this study and also in the alzhub library, so you can read them here.

• Incidence of Dementia over Three Decades in the Framingham Heart Study New England Journal of Medicine · 2016 · usefulness score 68/100 · How common it is
• Blood pressure from mid- to late life and risk of incident dementia Neurology · 2017-11-08 · usefulness score 67/100 · Risk factors
• Clinical diagnosis of Alzheimer's disease: report of the NINCDS-ADRDA Work Group under the auspices of Department of Health and Human Services Task Force on Alzheimer's Disease Neurology · 1984-07-01 · usefulness score 58/100 · Getting a diagnosis
• Neuropsychological Criteria for Mild Cognitive Impairment and Dementia Risk in the Framingham Heart Study Journal of the International Neuropsychological Society : JINS · 2016-03-31 · usefulness score 53/100 · Mild cognitive impairment
• Linguistic markers predict onset of Alzheimer's disease EClinicalMedicine · 2020-10-22 · usefulness score 52/100 · Long-term cohort studies
• Digital technologies as biomarkers, clinical outcomes assessment, and recruitment tools in Alzheimer's disease clinical trials Alzheimer's & dementia (New York, N. Y.) · 2018-05-24 · usefulness score 51/100 · Digital health tools
• How technology is reshaping cognitive assessment: Lessons from the Framingham Heart Study Neuropsychology · 2017-11-01 · usefulness score 39/100 · Digital health tools

Later studies that cite this one

8 studies in the alzhub library list this one in their references.

• Machine Learning for Dementia Prediction: A Systematic Review and Future Research Directions Journal of medical systems · 2023-02-01 · usefulness score 62/100 · Digital health tools
• Prediction of Alzheimer's disease progression within 6 years using speech: A novel approach leveraging language models Alzheimer's & dementia : the journal of the Alzheimer's Association · 2024-06-25 · usefulness score 57/100 · Digital health tools
• Automated detection of mild cognitive impairment and dementia from voice recordings: A natural language processing approach Alzheimer's & dementia : the journal of the Alzheimer's Association · 2022-07-07 · usefulness score 55/100 · Digital health tools
• An AI‐first framework for multimodal data in Alzheimer's disease and related dementias Alzheimer s & Dementia · 2025-09-01 · usefulness score 51/100 · Digital health tools
• Artificial intelligence and open science in discovery of disease-modifying medicines for Alzheimer's disease Cell reports. Medicine · 2024-02-01 · usefulness score 50/100 · Digital health tools
• Speech-based digital cognitive assessments for detection of mild cognitive impairment: Validation against paper-based neurocognitive assessment scores Journal of Alzheimer s Disease · 2025-05-25 · usefulness score 48/100 · Getting a diagnosis
• Obfuscation via pitch‐shifting for balancing privacy and diagnostic utility in voice‐based cognitive assessment Alzheimer s & Dementia · 2025-03-01 · usefulness score 48/100 · Getting a diagnosis
• Emerging Trends in Early Dementia Diagnosis: An Analysis on Advanced Machine Learning Approaches ACM Computing Surveys · 2025-08-28 · usefulness score 32/100 · Digital health tools

Related studies in Digital health tools

• Age and Graphomotor Decision Making Assessed with the Digital Clock Drawing Test: The Framingham Heart Study Journal of Alzheimer's disease : JAD · 2017-01-01 · usefulness score 43/100
• Detection and Classification of Alzheimer’s Disease Using Deep and Machine Learning Tomography · 2025-12-26 · usefulness score 36/100
• Voiceprints of cognitive impairment: analyzing digital voice for early detection of Alzheimer’s and related dementias npj Dementia · 2025-11-03 · usefulness score 55/100
• Deep learning-based speech analysis for Alzheimer's disease detection: a literature review Alzheimer's research & therapy · 2022-12-14 · usefulness score 43/100
• Multimodal deep learning models for early detection of Alzheimer's disease stage Scientific reports · 2021-02-05 · usefulness score 43/100
• Face Mesh for Dementia Detection: Evaluating Data Augmentation in Deep Learning and Traditional Machine Learning IEEE Access · 2025-01-01 · usefulness score 34/100

Also tagged “prevalence”

1,224 studies in the library carry the topic tag “prevalence”; these are this study's neighbours in a fixed, alphabetical-by-hash tour of them, not the closest in content. All 1,224 tagged studies.

• Ethics of Sensor-Based Surveillance of People with Dementia in Clinical Practice Sensors · 2025-04-02 · usefulness score 42/100 · Clinical safety
• A National Survey Exploring Dementia Diagnostics and Care Provisions in Primary Care in England medRxiv (preprint) · 2026-09-12 · usefulness score 16/100 · Biomarkers
• Outcomes for Dual-Eligible Beneficiaries With Dementia in Special Needs Plans and Other Medicare Advantage Plans JAMA Network Open · 2025-02-21 · usefulness score 53/100 · Health systems and care delivery

Similar studies

• Machine learning diagnosis of cognitive impairment and dementia in harmonized older adult cohorts 2025 · Digital health tools
• Alzheimer Detection Using Deep Learning Methods 2025 · Brain scans and imaging
• Alzheimers and Brain Tumor Detection Using Deep Learning 2025 · Brain scans and imaging
• Multi-model deep learning for dementia detection: addressing data and model limitations 2025 · Getting a diagnosis
• Enhancing dementia risk prediction with heart rate and machine learning in the Canadian Longitudinal Study on Aging 2025 · Digital health tools
• Evaluation of dementia in the cardiovascular health cognition study 2003 · Disparities

More on Digital health tools

• A review that covers this ground Deep Learning in Alzheimer's Disease: Diagnostic Classification and Prognostic Prediction Using Neuroimaging Data Frontiers in aging neuroscience · 2019-08-20 · usefulness score 72/100
• Newest in this subject Speaker-Wise Dementia Screening fromNaturalistic Speech with Lexical Entropy, Semantic Drift, and Multimodal Fusion Research Square · 2026-10-21 · Preprint — not yet peer-reviewed · usefulness score 16/100
• Newest in this subject Integrating Multisource Target Evidence with Machine-Learning Potency Prediction for Alzheimer’s Disease Drug Discovery bioRxiv · 2026-10-20 · Preprint — not yet peer-reviewed · usefulness score 16/100

See all studies in Digital health tools →