Machine Learning to Develop and Internally Validate a Predictive Model for Post-operative Delirium in a Prospective, Observational Clinical Cohort Study of Older Surgical Patients

Topic: Risk, prevention and care in population studies

Background Our objective was to assess the performance of machine learning methods to predict post-operative delirium using a prospective clinical cohort. Methods We analyzed data from an observational cohort study of 560 older adults (≥ 70 years) without dementia undergoing major elective non-cardiac surgery. Post-operative delirium was determined by the Confusion Assessment Method supplemented by a medical chart review (N = 134, 24%). Five machine learning algorithms and a standard stepwise logistic regression model were developed in a training sample (80% of participants) and evaluated in the remaining hold-out testing sample. We evaluated three overlapping feature sets, restricted to variables that are readily available or minimally burdensome to collect in clinical settings, including interview and medical record data. A large feature set included 71 potential predictors. A smaller set of 18 features was selected by an expert panel using a consensus process, and this smaller feature set was considered with and without a measure of pre-operative mental status. Results The area under the receiver operating characteristic curve (AUC) was higher in the large feature set conditions (range of AUC, 0.62-0.71 across algorithms) versus the selected feature set conditions (AUC range, 0.53-0.57). The restricted feature set with mental status had intermediate AUC values (range, 0.53-0.68). In the full feature set condition, algorithms such as gradient boosting, cross-validated logistic regression, and neural network (AUC = 0.71, 95% CI 0.58-0.83) were comparable with a model developed using traditional stepwise logistic regression (AUC = 0.69, 95% CI 0.57-0.82). Calibration for all models and feature sets was poor. Conclusions We developed machine learning prediction models for post-operative delirium that performed better than chance and are comparable with traditional stepwise logistic regression. Delirium proved to be a phenotype that was difficult to predict with appreciable accuracy.

Key findings

• Results The area under the receiver operating characteristic curve (AUC) was higher in the large feature set conditions (range of AUC, 0.62-0.71 across algorithms) versus the selected feature set conditions (AUC range, 0.53-0.57).

Why it matters

In 560 older surgical patients without dementia, machine learning models predicted post-operative delirium with area-under-the-curve values of 0.62 to 0.71 using a large feature set. This concerns delirium after surgery, not treating, curing or preventing Alzheimer's disease.

New to this subject? Start with our plain-English guide: Digital health →

Study details

• Study designcohort study
• Participants134
• Effect sizeAUC = 0.71; AUC = 0.69
• Evidence strengthModerate

Read the original

• DOI: 10.1007/s11606-020-06238-7
• PubMed: 33078300
• Original source →
• Free full text: Journal of General Internal Medicine →

Full text: free to read — OpenAlex lists this copy as free to read: the link above (checked 6 Feb 1020).

@article{racine2021-33078300,
  author  = {Racine, A. M. and Tommet, D. and D'Aquila, M. L. and Fong, T. G. and Gou, Y. and Tabloski, P. A. and Metzger, E. D. and Hshieh, T. T. and others},
  title   = {{Machine Learning to Develop and Internally Validate a Predictive Model for Post-operative Delirium in a Prospective, Observational Clinical Cohort Study of Older Surgical Patients}},
  journal = {Journal of general internal medicine},
  year    = {2021},
  doi     = {10.1007/s11606-020-06238-7},
  pmid    = {33078300},
  url     = {https://doi.org/10.1007/s11606-020-06238-7}
}

Studies this one cites

Referenced by this study and also in the alzhub library, so you can read them here.

• Association of Delirium With Cognitive Decline in Late Life: A Neuropathologic Study of 3 Population-Based Cohort Studies JAMA psychiatry · 2017-03-01 · usefulness score 70/100 · Caregiving
• Effect of Delirium and Other Major Complications on Outcomes After Elective Surgery in Older Adults JAMA surgery · 2015-12-01 · usefulness score 68/100 · How common it is
• A predictive model for delirium in hospitalized elderly medical patients based on admission characteristics Annals of internal medicine · 1993-09-01 · usefulness score 65/100 · How common it is
• Delirium is a strong risk factor for dementia in the oldest-old: a population-based cohort study Brain : a journal of neurology · 2012-08-09 · usefulness score 64/100 · How common it is
• Novel risk markers and long-term outcomes of delirium: the successful aging after elective surgery (SAGES) study design and methods Journal of the American Medical Directors Association · 2012-09-19 · usefulness score 60/100 · Prevention and risk
• The interface between delirium and dementia in elderly adults The Lancet. Neurology · 2015-06-29 · usefulness score 56/100 · Prevention and risk
• A chart-based method for identification of delirium: validation compared with interviewer ratings using the confusion assessment method Journal of the American Geriatrics Society · 2005-02-01 · usefulness score 49/100 · Getting a diagnosis

Later studies that cite this one

3 studies in the alzhub library list this one in their references.

• Prediction and risk evaluation of delirium after surgery in older patients: development and internal validation of an algorithm from the prospective BioCog cohort study British Journal of Anaesthesia · 2026-03-17 · usefulness score 62/100 · Clinical safety
• AI-delirium guard: Predictive modeling of postoperative delirium in elderly surgical patients PLoS ONE · 2025-06-05 · usefulness score 50/100 · Clinical safety
• Development and Validation of a Routine Electronic Health Record-Based Delirium Prediction Model for Surgical Patients Without Dementia: Retrospective Case-Control Study JMIR Perioperative Medicine · 2025-01-09 · usefulness score 40/100 · Clinical safety

Related studies in Digital health tools

• Comprehensive Machine Learning-Based Prediction Model for Delirium Risk in Older Patients with Dementia: Risk Factors Identification Clinical Interventions in Aging · 2025-05-01 · usefulness score 43/100
• Prediction of Postoperative Delirium in Geriatric Hip Fracture Patients: A Clinical Prediction Model Using Machine Learning Algorithms Geriatric orthopaedic surgery & rehabilitation · 2021-12-13 · usefulness score 52/100
• Developing a machine learning model for detecting depression, anxiety, and apathy in older adults with mild cognitive impairment using speech and facial expressions: A cross-sectional observational study International journal of nursing studies · 2023-07-07 · usefulness score 52/100
• Machine Learning for Predicting Postoperative Functional Disability and Mortality Among Older Patients With Cancer: Retrospective Cohort Study JMIR Aging · 2025-05-14 · usefulness score 53/100
• Poorer outcomes and greater healthcare costs for hospitalised older people with dementia and delirium: a retrospective cohort study International journal of geriatric psychiatry · 2016-04-25 · usefulness score 57/100
• Predictive factors for dementia among older adults in South Korea: an interpretable machine learning analysis Alzheimer s Research & Therapy · 2025-11-13 · usefulness score 46/100

Similar studies

• Enhancing Delirium Prediction and Prevention in Elderly Patients Through Machine Learning-Based Analysis 2025 · Clinical safety
• Machine Learning Algorithm Using Electronic Chart-Derived Data to Predict Delirium After Elderly Hip Fracture Surgeries: A Retrospective Case-Control Study 2021 · Digital health tools
• Delirium after surgery: a retrospective study of predictors, complications, and screening patterns in the national surgical quality improvement program 2025 · Clinical safety
• Development and Validation of a Deep Learning Algorithm for Mortality Prediction in Selecting Patients With Dementia for Earlier Palliative Care Interventions 2019 · Clinical safety
• A comparison of machine learning methods for survival analysis of high-dimensional clinical data for dementia prediction 2020 · Digital health tools
• Postoperative delirium in elderly patients is associated with subsequent cognitive impairment 2017 · Clinical safety

More on Digital health tools

• A review that covers this ground Machine learning for modeling the progression of Alzheimer disease dementia using clinical data: a systematic literature review JAMIA open · 2021-07-01 · usefulness score 44/100
• Newest in this subject Speaker-Wise Dementia Screening fromNaturalistic Speech with Lexical Entropy, Semantic Drift, and Multimodal Fusion Research Square · 2026-10-21 · Preprint — not yet peer-reviewed · usefulness score 16/100
• Newest in this subject Integrating Multisource Target Evidence with Machine-Learning Potency Prediction for Alzheimer’s Disease Drug Discovery bioRxiv · 2026-10-20 · Preprint — not yet peer-reviewed · usefulness score 16/100

See all studies in Digital health tools →