Home › Research › Risk factors
Evaluating Sociodemographic Bias in an Artificial Intelligence Algorithm to Detect Cognitive Impairment in Electronic Health Records
Colin Magdamo, Yignan He, John R. Dickson, Tanish Tyagi, Michael Brandon Westover, Shibani S. Mukerji, Christine Seel Ritchie, Bradley T. Hyman et al.. Alzheimer s & Dementia, 2024-12-01
Topic: Risk, prevention and care in population studies
Abstract Background Underdiagnosis of Alzheimer’s disease and related dementias (ADRD) leads to lost opportunities for timely intervention, increased healthcare costs, and underestimation of the true burden of disease. To address this problem, we developed an AI algorithm, Decipher‐AI (DEtection of Cognitive Impairment PHenotypes in EHR), to screen primary care patients for undiagnosed cognitive impairment (CI). We evaluated performance across sociodemographic groups using 3 years of EHR data before the first diagnosis or most recent visit. Method Decipher‐AI employs a two‐level hierarchical model, consisting of a large language model (LLM) to generate latent representations from unstructured clinical text and a patient‐level model that combines these representations with structured EHR data to predict the probability of CI (AUC: 0.98, 95% CI: 0.94, 1.00). Decipher‐AI was evaluated on a test set comprising 22,000 Mass General Brigham primary care patients aged 65 years or older. The selection process involved stratified random sampling across distinct racial/ethnic strata to ensure representation of diverse sociodemographic factors (Table 1). Cognitive status labels were determined based on the presence of dementia‐related diagnosis codes, while sociodemographic status was measured using the Area Deprivation Index (ADI). Result The AUC on the validation dataset was 0.80 [95% CI: 0.79, 0.81]; sensitivity was 0.75 [0.74, 0.77] and specificity 0.68 [0.67, 0.69], at the threshold of maximum accuracy. There were no significant differences in AUCs across sex or race/ethnic subgroups, but AUC was lower for patients under 75 years. (Figure 1A). Notably, Decipher‐AI exhibited diminished performance in patients from more disadvantaged neighborhoods (Figure 1B). Further analysis revealed that these disparities were associated with fewer encounters, outpatient visits, and notes containing cognition‐related keywords in patients from disadvantaged neighborhoods (Table 2). Conclusion Our study highlights the potential of Decipher‐AI for screening undiagnosed cognitive impairment in primary care. Nevertheless, disparities in algorithm performance underscore the importance of addressing sociodemographic inequities in EHR data. In the future, systematic tailored screening in primary care via standardized questionnaires coupled with AI‐assisted chart reviews, has the potential to address the underdiagnosis of ADRD in an equitable manner.
Key findings
- Result The AUC on the validation dataset was 0.80 [95% CI: 0.79, 0.81]; sensitivity was 0.75 [0.74, 0.77] and specificity 0.68 [0.67, 0.69], at the threshold of maximum accuracy.
- There were no significant differences in AUCs across sex or race/ethnic subgroups, but AUC was lower for patients under 75 years.
- Further analysis revealed that these disparities were associated with fewer encounters, outpatient visits, and notes containing cognition‐related keywords in patients from disadvantaged neighborhoods (Table 2).
Why it matters
An AI algorithm called Decipher-AI was developed to screen primary care patients for undiagnosed cognitive impairment, and it showed no significant differences in performance across sex or race/ethnic subgroups, though performance was lower for patients under 75 and disparities were linked to fewer encounters and cognition-related notes in patients from disadvantaged neighborhoods. This is about finding undiagnosed cognitive impairment with a computer tool, not about treating, curing, or preventing Alzheimer's disease.
New to this subject? Start with our plain-English guide: Risk factors, ranked →
Study details
- Effect sizeAUC: 0.98
- Evidence strengthLow
How this study scored 38/100
Published in a peer-reviewed journal. Scored zero for study design, sample size, studied in people and randomised / blinded because the record does not state it.
| Component | Points | Why |
|---|---|---|
| Journal tier | 7 / 10 | Tier 3 — peer-reviewed journal (Alzheimer s & Dementia) |
| Journal's citation record | 2 / 4 | Alzheimer s & Dementia: journal h-index 228 (OpenAlex, 2026-10-29) |
| Study design | 0 / 12 | No study design recorded — scores zero, not guessed (not stated in the record — scored 0) |
| Strength of evidence | 1 / 5 | Rated “low” in our summary |
| Peer-review validation | 10 / 10 | Peer-reviewed. Published as an article in a named peer-reviewed journal. |
| How recent | 4 / 5 | 2024 — within 3 year(s) of the newest study here |
| Sample size | 0 / 4 | Participant count not stated — scores zero (not stated in the record — scored 0) |
| Effect reported | 3 / 3 | Effect reported with a statistic (interval, p-value, ratio or similar) |
| Studied in people | 0 / 3 | Not stated whether this was studied in people — scores zero (not stated in the record — scored 0) |
| Randomised / blinded | 0 / 2 | No randomisation or blinding stated — scores zero (not stated in the record — scored 0) |
| Times cited | 1 / 7 | Cited 1 times (OpenAlex, 2026-10-10) |
| Citations per year | 0 / 5 | 0.2 citations a year over 4 year(s) since 2024 |
| Impact for its field and year | 1 / 6 | Cited 0.2× the average for papers of its field and year (OpenAlex FWCI) |
| Cited by studies here | 0 / 6 | No other study in this library cites it |
| Still being cited | 0 / 4 | Cited 1 times in 2024–2026 (2026 still in progress; OpenAlex) |
| Senior author's track record | 2 / 4 | Last-listed author Sudeshna Das: h-index 40 (OpenAlex, whole career) |
| Institution's track record | 3 / 3 | Harvard University: institution h-index 3,235 (OpenAlex) |
| Multi-centre collaboration | 1 / 3 | OpenAlex lists 2 institution(s) in 1 country |
| Indexed in PubMed | 1 / 2 | Not in PubMed, but has a DOI |
| Free to read | 2 / 2 | Free to read |
| Total | 38 / 100 | — |
What each of these 20 components measures, and what this score is not →
Read the original
Full text: free to read — OpenAlex lists this copy as free to read: the link above (checked 6 Feb 1020).
Cite this study APA, Vancouver, BibTeX · download .bib or .ris
Built from the record alzhub holds for this paper, not looked up. Volume, issue and page numbers are not held in the record. The record's author list ends in “et al.”, so not every author is named. Check the citation against the original before you use it.
APA 7th
Magdamo, C., He, Y., Dickson, J. R., Tyagi, T., Westover, M. B., Mukerji, S. S., et al. (2024). Evaluating Sociodemographic Bias in an Artificial Intelligence Algorithm to Detect Cognitive Impairment in Electronic Health Records. Alzheimer s & Dementia. https://doi.org/10.1002/alz.093404
Vancouver
Magdamo C, He Y, Dickson JR, Tyagi T, Westover MB, Mukerji SS, et al. Evaluating Sociodemographic Bias in an Artificial Intelligence Algorithm to Detect Cognitive Impairment in Electronic Health Records. Alzheimer s & Dementia. 2024. doi:10.1002/alz.093404.
BibTeX
@article{magdamo2024-b8b66aef,
author = {Magdamo, Colin and He, Yignan and Dickson, John R. and Tyagi, Tanish and Westover, Michael Brandon and Mukerji, Shibani S. and Ritchie, Christine Seel and Hyman, Bradley T. and others},
title = {{Evaluating Sociodemographic Bias in an Artificial Intelligence Algorithm to Detect Cognitive Impairment in Electronic Health Records}},
journal = {Alzheimer s \& Dementia},
year = {2024},
doi = {10.1002/alz.093404},
url = {https://doi.org/10.1002/alz.093404}
}
Related studies in Risk factors
Also tagged “assessment”
840 studies in the library carry the topic tag “assessment”; these are this study's neighbours in a fixed, alphabetical-by-hash tour of them, not the closest in content. All 840 tagged studies.