Home › Research › Digital health tools
Combining billing codes, clinical notes, and medications from electronic health records provides superior phenotyping performance
Wei WQ, Teixeira PL, Mo H, Cronin RM, Warner JL, Denny JC.. Journal of the American Medical Informatics Association : JAMIA, 2015-09-02
Topic: Health services, long-term care and technology
Objective To evaluate the phenotyping performance of three major electronic health record (EHR) components: International Classification of Disease (ICD) diagnosis codes, primary notes, and specific medications. Materials and methods We conducted the evaluation using de-identified Vanderbilt EHR data. We preselected ten diseases: atrial fibrillation, Alzheimer's disease, breast cancer, gout, human immunodeficiency virus infection, multiple sclerosis, Parkinson's disease, rheumatoid arthritis, and types 1 and 2 diabetes mellitus. For each disease, patients were classified into seven categories based on the presence of evidence in diagnosis codes, primary notes, and specific medications. Twenty-five patients per disease category (a total number of 175 patients for each disease, 1750 patients for all ten diseases) were randomly selected for manual chart review. Review results were used to estimate the positive predictive value (PPV), sensitivity, andF-score for each EHR component alone and in combination. Results The PPVs of single components were inconsistent and inadequate for accurately phenotyping (0.06-0.71). Using two or more ICD codes improved the average PPV to 0.84. We observed a more stable and higher accuracy when using at least two components (mean ± standard deviation: 0.91 ± 0.08). Primary notes offered the best sensitivity (0.77). The sensitivity of ICD codes was 0.67. Again, two or more components provided a reasonably high and stable sensitivity (0.59 ± 0.16). Overall, the best performance (Fscore: 0.70 ± 0.12) was achieved by using two or more components. Although the overall performance of using ICD codes (0.67 ± 0.14) was only slightly lower than using two or more components, its PPV (0.71 ± 0.13) is substantially worse (0.91 ± 0.08). Conclusion Multiple EHR components provide a more consistent and higher performance than a single one for the selected phenotypes. We suggest considering multiple EHR components for future phenotyping design in order to obtain an ideal result.
Key findings
- Review results were used to estimate the positive predictive value (PPV), sensitivity, andF-score for each EHR component alone and in combination.
- Results The PPVs of single components were inconsistent and inadequate for accurately phenotyping (0.06-0.71).
- We observed a more stable and higher accuracy when using at least two components (mean ± standard deviation: 0.91 ± 0.08).
- Overall, the best performance (Fscore: 0.70 ± 0.12) was achieved by using two or more components.
- Although the overall performance of using ICD codes (0.67 ± 0.14) was only slightly lower than using two or more components, its PPV (0.71 ± 0.13) is substantially worse (0.91 ± 0.08).
New to this subject? Start with our plain-English guide: Digital health →
Study details
- Participants175
- Evidence strengthLow
How this study scored 45/100
Published in a peer-reviewed journal. Scored zero for study design, effect reported and randomised / blinded because the record does not state it. We have no data on journal's citation record, times cited, citations per year, impact for its field and year, cited by studies here, still being cited, senior author's track record, institution's track record and multi-centre collaboration, so those were left out: it earned 26 of the 58 points we could judge it on, which is 45 out of 100.
| Component | Points | Why |
|---|---|---|
| Journal tier | 7 / 10 | Tier 3 — peer-reviewed journal (Journal of the American Medical Informatics Association : JAMIA) |
| Journal's citation record | — / 4 | No journal record from OpenAlex — left out of this score |
| Study design | 0 / 12 | No study design recorded — scores zero, not guessed (not stated in the record — scored 0) |
| Strength of evidence | 1 / 5 | Rated “low” in our summary |
| Peer-review validation | 10 / 10 | Peer-reviewed. Published as an article in a named peer-reviewed journal. |
| How recent | 1 / 5 | 2016 — within 20 year(s) of the newest study here |
| Sample size | 2 / 4 | 175 participants |
| Effect reported | 0 / 3 | No effect size reported — scores zero (not stated in the record — scored 0) |
| Studied in people | 3 / 3 | Studied in people (participant count stated) |
| Randomised / blinded | 0 / 2 | No randomisation or blinding stated — scores zero (not stated in the record — scored 0) |
| Times cited | — / 7 | No citation count (not found in OpenAlex) — left out of this score |
| Citations per year | — / 5 | No citation count or no year — left out of this score |
| Impact for its field and year | — / 6 | No field-weighted citation impact (OpenAlex has none yet) — left out of this score |
| Cited by studies here | — / 6 | Not yet checked against this library's citation graph — left out of this score |
| Still being cited | — / 4 | No yearly citation counts — left out of this score |
| Senior author's track record | — / 4 | Senior author's record not available — left out of this score |
| Institution's track record | — / 3 | No institution listed in OpenAlex — left out of this score |
| Multi-centre collaboration | — / 3 | Institutions not listed — left out of this score |
| Indexed in PubMed | 2 / 2 | Indexed in PubMed (PMID 26338219) |
| Free to read | 0 / 2 | Behind a paywall — scores zero |
| Points | 26 / 58 | Scored on 11 of 20 parts. We have no data for the rest, so they are left out: 26 of the 58 points we could judge it on is 45 out of 100. |
| Total | 45 / 100 | — |
What each of these 20 components measures, and what this score is not →
Read the original
Free copy unknown: no free-copy check has an answer for this record yet. That does not mean it is paywalled.
Cite this study APA, Vancouver, BibTeX · download .bib or .ris
Built from the record alzhub holds for this paper, not looked up. Volume, issue and page numbers are not held in the record. Check the citation against the original before you use it.
APA 7th
Wei, W. Q., Teixeira, P. L., Mo, H., Cronin, R. M., Warner, J. L., & Denny, J. C. (2016). Combining billing codes, clinical notes, and medications from electronic health records provides superior phenotyping performance. Journal of the American Medical Informatics Association : JAMIA. https://doi.org/10.1093/jamia/ocv130
Vancouver
Wei WQ, Teixeira PL, Mo H, Cronin RM, Warner JL, Denny JC. Combining billing codes, clinical notes, and medications from electronic health records provides superior phenotyping performance. Journal of the American Medical Informatics Association : JAMIA. 2016. doi:10.1093/jamia/ocv130. PMID: 26338219.
BibTeX
@article{wei2016-26338219,
author = {Wei, W. Q. and Teixeira, P. L. and Mo, H. and Cronin, R. M. and Warner, J. L. and Denny, J. C.},
title = {{Combining billing codes, clinical notes, and medications from electronic health records provides superior phenotyping performance}},
journal = {Journal of the American Medical Informatics Association : JAMIA},
year = {2016},
doi = {10.1093/jamia/ocv130},
pmid = {26338219},
url = {https://doi.org/10.1093/jamia/ocv130}
}