Home › Research › Digital health tools
Correlating natural language processing and automated speech analysis with clinician assessment to quantify speech-language changes in mild cognitive impairment and Alzheimer's dementia
Yeung A, Iaboni A, Rochon E, Lavoie M, Santiago C, Yancheva M, Novikova J, Xu M et al.. Alzheimer's research & therapy, 2021-06-04
Topic: Risk, prevention and care in population studies
Background Language impairment is an important marker of neurodegenerative disorders. Despite this, there is no universal system of terminology used to describe these impairments and large inter-rater variability can exist between clinicians assessing language. The use of natural language processing (NLP) and automated speech analysis (ASA) is emerging as a novel and potentially more objective method to assess language in individuals with mild cognitive impairment (MCI) and Alzheimer's dementia (AD). No studies have analyzed how variables extracted through NLP and ASA might also be correlated to language impairments identified by a clinician. Methods Audio recordings (n=30) from participants with AD, MCI, and controls were rated by clinicians for word-finding difficulty, incoherence, perseveration, and errors in speech. Speech recordings were also transcribed, and linguistic and acoustic variables were extracted through NLP and ASA. Correlations between clinician-rated speech characteristics and the variables were compared using Spearman's correlation. Exploratory factor analysis was applied to find common factors between variables for each speech characteristic. Results Clinician agreement was high in three of the four speech characteristics: word-finding difficulty (ICC = 0.92, p Conclusions Variables extracted through automated acoustic and linguistic analysis of MCI and AD speech were significantly correlated with clinician ratings of speech and language characteristics. Our results suggest that correlating NLP and ASA with clinician observations is an objective and novel approach to measuring speech and language changes in neurodegenerative disorders.
Key findings
- Results Clinician agreement was high in three of the four speech characteristics: word-finding difficulty (ICC = 0.92, p<0.001), incoherence (ICC = 0.91, p<0.001), and perseveration (ICC = 0.88, p<0.001).
- Our results suggest that correlating NLP and ASA with clinician observations is an objective and novel approach to measuring speech and language changes in neurodegenerative disorders.
Why it matters
Variables from automated speech and language analysis were significantly correlated with clinician ratings of speech and language in people with mild cognitive impairment and Alzheimer's dementia. This is about objective measurement of language changes, not about treating, curing, or preventing the disease.
New to this subject? Start with our plain-English guide: Digital health →
Study details
- Participants30
- Evidence strengthLow
How this study scored 51/100
Published in a peer-reviewed journal. Scored zero for study design, effect reported and randomised / blinded because the record does not state it. We have no data on senior author's track record, so that was left out: it earned 49 of the 96 points we could judge it on, which is 51 out of 100.
| Component | Points | Why |
|---|---|---|
| Journal tier | 7 / 10 | Tier 3 — peer-reviewed journal (Alzheimer's research & therapy) |
| Journal's citation record | 1 / 4 | Alzheimer s Research & Therapy: journal h-index 146 (OpenAlex, 2026-10-29) |
| Study design | 0 / 12 | No study design recorded — scores zero, not guessed (not stated in the record — scored 0) |
| Strength of evidence | 1 / 5 | Rated “low” in our summary |
| Peer-review validation | 10 / 10 | Peer-reviewed. Published as an article in a named peer-reviewed journal. |
| How recent | 3 / 5 | 2021 — within 6 year(s) of the newest study here |
| Sample size | 1 / 4 | 30 participants |
| Effect reported | 0 / 3 | No effect size reported — scores zero (not stated in the record — scored 0) |
| Studied in people | 3 / 3 | Studied in people (participant count stated) |
| Randomised / blinded | 0 / 2 | No randomisation or blinding stated — scores zero (not stated in the record — scored 0) |
| Times cited | 4 / 7 | Cited 88 times (OpenAlex, 2026-10-10) |
| Citations per year | 3 / 5 | 12.6 citations a year over 7 year(s) since 2021 |
| Impact for its field and year | 4 / 6 | Cited 8.9× the average for papers of its field and year (OpenAlex FWCI) |
| Cited by studies here | 1 / 6 | Cited by 2 other studies in this library |
| Still being cited | 3 / 4 | Cited 51 times in 2024–2026 (2026 still in progress; OpenAlex) |
| Senior author's track record | — / 4 | OpenAlex's profile for last-listed author Fariya Mostafa holds only 6 work(s), too few to judge a track record — left out of this score |
| Institution's track record | 2 / 3 | University of Toronto: institution h-index 1,923 (OpenAlex) |
| Multi-centre collaboration | 2 / 3 | OpenAlex lists 4 institution(s) in 1 country |
| Indexed in PubMed | 2 / 2 | Indexed in PubMed (PMID 34088354) |
| Free to read | 2 / 2 | Free to read |
| Points | 49 / 96 | Scored on 19 of 20 parts. We have no data for the rest, so they are left out: 49 of the 96 points we could judge it on is 51 out of 100. |
| Total | 51 / 100 | — |
What each of these 20 components measures, and what this score is not →
Read the original
- DOI: 10.1186/s13195-021-00848-x
- PubMed: 34088354
- Original source →
- Free full text: Alzheimer s Research & Therapy →
Full text: free to read — OpenAlex lists this copy as free to read: the link above (checked 6 Feb 1020).
Cite this study APA, Vancouver, BibTeX · download .bib or .ris
Built from the record alzhub holds for this paper, not looked up. Volume, issue and page numbers are not held in the record. The record's author list ends in “et al.”, so not every author is named. Check the citation against the original before you use it.
APA 7th
Yeung, A., Iaboni, A., Rochon, E., Lavoie, M., Santiago, C., Yancheva, M., et al. (2021). Correlating natural language processing and automated speech analysis with clinician assessment to quantify speech-language changes in mild cognitive impairment and Alzheimer's dementia. Alzheimer's research & therapy. https://doi.org/10.1186/s13195-021-00848-x
Vancouver
Yeung A, Iaboni A, Rochon E, Lavoie M, Santiago C, Yancheva M, et al. Correlating natural language processing and automated speech analysis with clinician assessment to quantify speech-language changes in mild cognitive impairment and Alzheimer's dementia. Alzheimer's research & therapy. 2021. doi:10.1186/s13195-021-00848-x. PMID: 34088354.
BibTeX
@article{yeung2021-34088354,
author = {Yeung, A. and Iaboni, A. and Rochon, E. and Lavoie, M. and Santiago, C. and Yancheva, M. and Novikova, J. and Xu, M. and others},
title = {{Correlating natural language processing and automated speech analysis with clinician assessment to quantify speech-language changes in mild cognitive impairment and Alzheimer's dementia}},
journal = {Alzheimer's research \& therapy},
year = {2021},
doi = {10.1186/s13195-021-00848-x},
pmid = {34088354},
url = {https://doi.org/10.1186/s13195-021-00848-x}
}
Studies this one cites
Referenced by this study and also in the alzhub library, so you can read them here.
Later studies that cite this one
2 studies in the alzhub library list this one in their references.