Home › Research › Risk factors
Evaluating biomarkers and prediction models: predictive utility analysis with E2P Simulator
Karvelis P, Felsky D, Abi-Dargham A, Horga G, Andreea A.. PsyArXiv, 2026-09-23
Topic: Brain imaging, connectivity and assessment
Precision psychiatry aims to identify biomarkers and develop prediction models to improve diagnosis, prognosis, and treatment selection. Despite extensive research, translating findings into clinically useful tools remains challenging. Here we argue that one important contributor to this translational gap is a pervasive misalignment between routine statistical analytic practices and the requirements for real-world decision making. To help address this, we introduce predictive utility analysis: a statistical workflow for estimating the practical significance of biomarkers and prediction models by accounting for outcome base rates and measurement reliability. Starting from effect size or model performance estimates, the workflow simulates expected real-world performance in a target population and summarizes it across complementary metrics including positive and negative predictive value, Precision-Recall AUC, and Net Benefit. Analogous to how power analysis is used for determining the conditions needed to achieve statistical significance, predictive utility analysis can help determine the conditions needed to achieve practical significance. We implement and illustrate this workflow using E2P Simulator, a recently developed open-source web tool that makes predictive utility analysis accessible through interactive simulations and publication-ready visualizations. We demonstrate its application using specific examples in three key areas: diagnostic (depression, Alzheimer’s disease), treatment response (antidepressants), and risk (suicide attempts and psychosis onset) prediction, highlighting conditions under which we may expect clinically meaningful advances. While we focus on translational challenges in psychiatry, the workflow and tools presented here address general statistical challenges and are broadly applicable across biomedical and behavioral sciences.
Key findings
- Starting from effect size or model performance estimates, the workflow simulates expected real-world performance in a target population and summarizes it across complementary metrics including positive and negative predictive value, Precision-Recall AUC, and Net Benefit.
- Analogous to how power analysis is used for determining the conditions needed to achieve statistical significance, predictive utility analysis can help determine the conditions needed to achieve practical significance.
- We implement and illustrate this workflow using E2P Simulator, a recently developed open-source web tool that makes predictive utility analysis accessible through interactive simulations and publication-ready visualizations.
- We demonstrate its application using specific examples in three key areas: diagnostic (depression, Alzheimer’s disease), treatment response (antidepressants), and risk (suicide attempts and psychosis onset) prediction, highlighting conditions under which we may expect clinically meaningful advances.
- While we focus on translational challenges in psychiatry, the workflow and tools presented here address general statistical challenges and are broadly applicable across biomedical and behavioral sciences.
New to this subject? Start with our plain-English guide: Risk factors, ranked →
Study details
- Evidence strengthLow
How this study scored 16/100
Preprint — not yet peer-reviewed. It scored 21 on the same rubric, but non-peer-reviewed sources are capped at 16, so it shows as 16. Scored zero for study design, sample size, effect reported, studied in people and randomised / blinded because the record does not state it. We have no data on times cited, citations per year, impact for its field and year, cited by studies here, still being cited, senior author's track record, institution's track record and multi-centre collaboration, so those were left out: it earned 13 of the 62 points we could judge it on, which is 21 out of 100.
| Component | Points | Why |
|---|---|---|
| Journal tier | 2 / 10 | Tier 4 — not a peer-reviewed journal (PsyArXiv) |
| Journal's citation record | 0 / 4 | Not a journal article, so the journal's record does not count |
| Study design | 0 / 12 | No study design recorded — scores zero, not guessed (not stated in the record — scored 0) |
| Strength of evidence | 1 / 5 | Rated “low” in our summary |
| Peer-review validation | 4 / 10 | Preprint — not yet peer-reviewed. Hosted on a preprint server. |
| How recent | 5 / 5 | 2026 — within 1 year(s) of the newest study here |
| Sample size | 0 / 4 | Participant count not stated — scores zero (not stated in the record — scored 0) |
| Effect reported | 0 / 3 | No effect size reported — scores zero (not stated in the record — scored 0) |
| Studied in people | 0 / 3 | Not stated whether this was studied in people — scores zero (not stated in the record — scored 0) |
| Randomised / blinded | 0 / 2 | No randomisation or blinding stated — scores zero (not stated in the record — scored 0) |
| Times cited | — / 7 | No citation count (not found in OpenAlex) — left out of this score |
| Citations per year | — / 5 | No citation count or no year — left out of this score |
| Impact for its field and year | — / 6 | No field-weighted citation impact (OpenAlex has none yet) — left out of this score |
| Cited by studies here | — / 6 | Not yet checked against this library's citation graph — left out of this score |
| Still being cited | — / 4 | No yearly citation counts — left out of this score |
| Senior author's track record | — / 4 | Senior author's record not available — left out of this score |
| Institution's track record | — / 3 | No institution listed in OpenAlex — left out of this score |
| Multi-centre collaboration | — / 3 | Institutions not listed — left out of this score |
| Indexed in PubMed | 1 / 2 | Not in PubMed, but has a DOI |
| Free to read | 0 / 2 | Behind a paywall — scores zero |
| Points | 13 / 62 | Scored on 12 of 20 parts. We have no data for the rest, so they are left out: 13 of the 62 points we could judge it on is 21 out of 100. |
| Total | 21 / 100 | Capped to 16 because this is not a peer-reviewed article. |
What each of these 20 components measures, and what this score is not →
Read the original
Free copy unknown: no free-copy check has an answer for this record yet. That does not mean it is paywalled.
Cite this study APA, Vancouver, BibTeX · download .bib or .ris
Built from the record alzhub holds for this paper, not looked up. Volume, issue and page numbers are not held in the record. Check the citation against the original before you use it.
APA 7th
Karvelis, P., Felsky, D., Abi-Dargham, A., Horga, G., & Andreea, A. (2026). Evaluating biomarkers and prediction models: predictive utility analysis with E2P Simulator. PsyArXiv. https://doi.org/10.31234/osf.io/atn5m_v3
Vancouver
Karvelis P, Felsky D, Abi-Dargham A, Horga G, Andreea A. Evaluating biomarkers and prediction models: predictive utility analysis with E2P Simulator. PsyArXiv. 2026. doi:10.31234/osf.io/atn5m_v3.
BibTeX
@article{karvelis2026-d7c8a83d,
author = {Karvelis, P. and Felsky, D. and Abi-Dargham, A. and Horga, G. and Andreea, A.},
title = {{Evaluating biomarkers and prediction models: predictive utility analysis with E2P Simulator}},
journal = {PsyArXiv},
year = {2026},
doi = {10.31234/osf.io/atn5m_v3},
url = {https://doi.org/10.31234/osf.io/atn5m_v3}
}