Unsupervised machine learning for the discovery of latent disease clusters and patient subgroups using electronic health records

Topic: Risk, prevention and care in population studies

Machine learning has become ubiquitous and a key technology on mining electronic health records (EHRs) for facilitating clinical research and practice. Unsupervised machine learning, as opposed to supervised learning, has shown promise in identifying novel patterns and relations from EHRs without using human created labels. In this paper, we investigate the application of unsupervised machine learning models in discovering latent disease clusters and patient subgroups based on EHRs. We utilized Latent Dirichlet Allocation (LDA), a generative probabilistic model, and proposed a novel model named Poisson Dirichlet Model (PDM), which extends the LDA approach using a Poisson distribution to model patients' disease diagnoses and to alleviate age and sex factors by considering both observed and expected observations. In the empirical experiments, we evaluated LDA and PDM on three patient cohorts, namely Osteoporosis, Delirium/Dementia, and Chronic Obstructive Pulmonary Disease (COPD)/Bronchiectasis Cohorts, with their EHR data retrieved from the Rochester Epidemiology Project (REP) medical records linkage system, for the discovery of latent disease clusters and patient subgroups. We compared the effectiveness of LDA and PDM in identifying disease clusters through the visualization of disease representations. We tested the performance of LDA and PDM in differentiating patient subgroups through survival analysis, as well as statistical analysis of demographics and Elixhauser Comorbidity Index (ECI) scores in those subgroups. The experimental results show that the proposed PDM could effectively identify distinguished disease clusters based on the latent patterns hidden in the EHR data by alleviating the impact of age and sex, and that LDA could stratify patients into differentiable subgroups with larger p-values than PDM. However, those subgroups identified by LDA are highly associated with patients' age and sex. The subgroups discovered by PDM might imply the underlying patterns of diseases of greater interest in epidemiology research due to the alleviation of age and sex. Both unsupervised machine learning approaches could be leveraged to discover patient subgroups using EHRs but with different foci.

@article{wang2020-31891765,
  author  = {Wang, Y. and Zhao, Y. and Therneau, T. M. and Atkinson, E. J. and Tafti, A. P. and Zhang, N. and Amin, S. and Limper, A. H. and others},
  title   = {{Unsupervised machine learning for the discovery of latent disease clusters and patient subgroups using electronic health records}},
  journal = {Journal of biomedical informatics},
  year    = {2020},
  doi     = {10.1016/j.jbi.2019.103364},
  pmid    = {31891765},
  url     = {https://doi.org/10.1016/j.jbi.2019.103364}
}

Key findings

• The experimental results show that the proposed PDM could effectively identify distinguished disease clusters based on the latent patterns hidden in the EHR data by alleviating the impact of age and sex, and that LDA could stratify patients into differentiable subgroups with larger p-values than PDM.
• However, those subgroups identified by LDA are highly associated with patients' age and sex.

Why it matters

Unsupervised machine learning models identified latent disease clusters and patient subgroups from electronic health records, including a Delirium/Dementia cohort. This is a method for discovering patterns in health records and does not treat, cure, or prevent Alzheimer's disease.

New to this subject? Start with our plain-English guide: Digital health →

Study details

• Evidence strengthLow

Read the original

• DOI: 10.1016/j.jbi.2019.103364
• PubMed: 31891765
• Original source →
• Free full text: Journal of Biomedical Informatics →

Full text: free to read — OpenAlex lists this copy as free to read: the link above (checked 6 Feb 1020).

Studies this one cites

Referenced by this study and also in the alzhub library, so you can read them here.

• The relationships between age, sex, and the incidence of dementia and Alzheimer disease: a meta-analysis Archives of general psychiatry · 1998-09-01 · usefulness score 66/100 · Getting a diagnosis

Related studies in Digital health tools

• Data-driven discovery of probable Alzheimer's disease and related dementia subphenotypes using electronic health records Learning health systems · 2020-09-10 · usefulness score 58/100
• Deep representation learning of electronic health records to unlock patient stratification at scale NPJ digital medicine · 2020-07-17 · usefulness score 57/100
• Cognitive performance classification of older patients using machine learning and electronic medical records Scientific Reports · 2025-02-24 · usefulness score 49/100
• Multiple morbidity across the lifespan in people with Down syndrome or intellectual disabilities: a population-based cohort study using electronic health records The Lancet. Public health · 2023-04-26 · usefulness score 55/100
• Detection of probable dementia cases in undiagnosed patients using structured and unstructured electronic health records BMC medical informatics and decision making · 2019-07-09 · usefulness score 53/100
• Antidepressant Use at the Threshold: using electronic health records to characterise people prescribed antidepressants around the time of dementia diagnosis medRxiv · 2025-12-17 · usefulness score 16/100

Also in Digital health tools

178 studies in the library are filed under Digital health tools with no topic tag they share with another; these are this study's neighbours in a fixed tour of them. The Digital health tools hub.

• Profiles of language impairment in primary progressive aphasia Archives of neurology · 1993-02-01 · usefulness score 41/100 · Digital health tools
• Diagnosis, Treatment and Future Perspectives in Artificial Intelligence and Neuronal Diseases International Journal of Emerging Multidiciplinaries Biomedical and Clinical Research · 2025-02-19 · usefulness score 28/100 · Digital health tools
• Artificial intelligence in atrial fibrillation – Timely diagnosis, risk assessment and personalized management Indian Pacing and Electrophysiology Journal · 2026-01-27 · usefulness score 38/100 · Digital health tools

Similar studies

• Unsupervised Machine Learning to Identify High Likelihood of Dementia in Population-Based Surveys: Development and Validation Study 2018 · Digital health tools
• Machine learning reveals distinct neuroanatomical signatures of cardiovascular and metabolic diseases in cognitively unimpaired individuals 2025 · Digital health tools
• Machine-learning techniques for building a diagnostic model for very mild dementia 2010 · Digital health tools
• Using machine learning and electronic health record (EHR) data for the early prediction of Alzheimer's Disease and Related Dementias 2025 · How common it is
• Using machine learning for detection of Parkinson’s disease and mild cognitive impairment 2025 · Mild cognitive impairment
• Potential value and impact of data mining and machine learning in clinical diagnostics 2021 · Digital health tools

More on Digital health tools

• A review that covers this ground The validity of dementia diagnoses in routinely collected electronic health records in the United Kingdom: A systematic review Pharmacoepidemiology and drug safety · 2019-01-22 · usefulness score 62/100
• Newest in this subject Speaker-Wise Dementia Screening fromNaturalistic Speech with Lexical Entropy, Semantic Drift, and Multimodal Fusion Research Square · 2026-10-21 · Preprint — not yet peer-reviewed · usefulness score 16/100
• Newest in this subject Integrating Multisource Target Evidence with Machine-Learning Potency Prediction for Alzheimer’s Disease Drug Discovery bioRxiv · 2026-10-20 · Preprint — not yet peer-reviewed · usefulness score 16/100

See all studies in Digital health tools →