Home › Research › Brain scans and imaging
Standardized evaluation of algorithms for computer-aided diagnosis of dementia based on structural MRI: the CADDementia challenge
Bron EE, Smits M, van der Flier WM, Vrenken H, Barkhof F, Scheltens P, Papma JM, Steketee RM et al.. NeuroImage, 2015-01-31
Topic: Risk, prevention and care in population studies
Algorithms for computer-aided diagnosis of dementia based on structural MRI have demonstrated high performance in the literature, but are difficult to compare as different data sets and methodology were used for evaluation. In addition, it is unclear how the algorithms would perform on previously unseen data, and thus, how they would perform in clinical practice when there is no real opportunity to adapt the algorithm to the data at hand. To address these comparability, generalizability and clinical applicability issues, we organized a grand challenge that aimed to objectively compare algorithms based on a clinically representative multi-center data set. Using clinical practice as the starting point, the goal was to reproduce the clinical diagnosis. Therefore, we evaluated algorithms for multi-class classification of three diagnostic groups: patients with probable Alzheimer's disease, patients with mild cognitive impairment and healthy controls. The diagnosis based on clinical criteria was used as reference standard, as it was the best available reference despite its known limitations. For evaluation, a previously unseen test set was used consisting of 354 T1-weighted MRI scans with the diagnoses blinded. Fifteen research teams participated with a total of 29 algorithms. The algorithms were trained on a small training set (n=30) and optionally on data from other sources (e.g., the Alzheimer's Disease Neuroimaging Initiative, the Australian Imaging Biomarkers and Lifestyle flagship study of aging). The best performing algorithm yielded an accuracy of 63.0% and an area under the receiver-operating-characteristic curve (AUC) of 78.8%. In general, the best performances were achieved using feature extraction based on voxel-based morphometry or a combination of features that included volume, cortical thickness, shape and intensity. The challenge is open for new submissions via the web-based framework: http://caddementia.grand-challenge.org.
Cite this study APA, Vancouver, BibTeX · download .bib or .ris
APA 7th
Bron, E. E., Smits, M., van der Flier, W. M., Vrenken, H., Barkhof, F., Scheltens, P., et al. (2015). Standardized evaluation of algorithms for computer-aided diagnosis of dementia based on structural MRI: the CADDementia challenge. NeuroImage. https://doi.org/10.1016/j.neuroimage.2015.01.048
Vancouver
Bron EE, Smits M, van der Flier WM, Vrenken H, Barkhof F, Scheltens P, et al. Standardized evaluation of algorithms for computer-aided diagnosis of dementia based on structural MRI: the CADDementia challenge. NeuroImage. 2015. doi:10.1016/j.neuroimage.2015.01.048. PMID: 25652394.
BibTeX
@article{bron2015-25652394,
author = {Bron, E. E. and Smits, M. and van der Flier, W. M. and Vrenken, H. and Barkhof, F. and Scheltens, P. and Papma, J. M. and Steketee, R. M. and others},
title = {{Standardized evaluation of algorithms for computer-aided diagnosis of dementia based on structural MRI: the CADDementia challenge}},
journal = {NeuroImage},
year = {2015},
doi = {10.1016/j.neuroimage.2015.01.048},
pmid = {25652394},
url = {https://doi.org/10.1016/j.neuroimage.2015.01.048}
}
Built from the record alzhub holds for this paper, not looked up. Volume, issue and page numbers are not held in the record. The record's author list ends in “et al.”, so not every author is named. Check the citation against the original before you use it.
Key findings
- Algorithms for computer-aided diagnosis of dementia based on structural MRI have demonstrated high performance in the literature, but are difficult to compare as different data sets and methodology were used for evaluation.
New to this subject? Start with our plain-English guide: Brain scans & tests →
Study details
- Participants30
- Effect size63.0%; 78.8%
- Evidence strengthLow
How this study scored 48/100
Published in a peer-reviewed journal. Scored zero for study design and randomised / blinded because the record does not state it. We have no data on journal's citation record, times cited, citations per year, impact for its field and year, cited by studies here, still being cited, senior author's track record and institution's track record, so those were left out: it earned 29 of the 61 points we could judge it on, which is 48 out of 100.
| Component | Points | Why |
|---|---|---|
| Journal tier | 7 / 10 | Tier 3 — peer-reviewed journal (NeuroImage) |
| Journal's citation record | — / 4 | No journal record from OpenAlex — left out of this score |
| Study design | 0 / 12 | No study design recorded — scores zero, not guessed (not stated in the record — scored 0) |
| Strength of evidence | 1 / 5 | Rated “low” in our summary |
| Peer-review validation | 10 / 10 | Peer-reviewed. Published as an article in a named peer-reviewed journal. |
| How recent | 1 / 5 | 2015 — within 20 year(s) of the newest study here |
| Sample size | 1 / 4 | 30 participants |
| Effect reported | 2 / 3 | Effect reported as a number, without an interval or test |
| Studied in people | 3 / 3 | Studied in people (participant count stated) |
| Randomised / blinded | 0 / 2 | No randomisation or blinding stated — scores zero (not stated in the record — scored 0) |
| Times cited | — / 7 | No citation count (not found in OpenAlex) — left out of this score |
| Citations per year | — / 5 | No citation count or no year — left out of this score |
| Impact for its field and year | — / 6 | No field-weighted citation impact (OpenAlex has none yet) — left out of this score |
| Cited by studies here | — / 6 | Not yet checked against this library's citation graph — left out of this score |
| Still being cited | — / 4 | No yearly citation counts — left out of this score |
| Senior author's track record | — / 4 | Senior author's record not available — left out of this score |
| Institution's track record | — / 3 | No institution listed in OpenAlex — left out of this score |
| Multi-centre collaboration | 2 / 3 | PubMed indexes it as a multicentre study |
| Indexed in PubMed | 2 / 2 | Indexed in PubMed (PMID 25652394) |
| Free to read | 0 / 2 | Behind a paywall — scores zero |
| Points | 29 / 61 | Scored on 12 of 20 parts. We have no data for the rest, so they are left out: 29 of the 61 points we could judge it on is 48 out of 100. |
| Total | 48 / 100 | — |
What each of these 20 components measures, and what this score is not →
Read the original
Free copy unknown: no free-copy check has an answer for this record yet. That does not mean it is paywalled.
Related studies in Brain scans and imaging
Also tagged “structural-mri”
3 studies in the library carry the topic tag “structural-mri”; these are this study's neighbours in a fixed, alphabetical-by-hash tour of them, not the closest in content. All 3 tagged studies.