Home › Research › How common it is
Bias in meta-analysis detected by a simple, graphical test
Matthias Egger, George Davey Smith, Martin Schneider, C. Minder. BMJ, 1997-09-13
Topic: Risk, prevention and care in population studies
Abstract Objective: Funnel plots (plots of effect estimates against sample size) may be useful to detect bias in meta-analyses that were later contradicted by large trials. We examined whether a simple test of asymmetry of funnel plots predicts discordance of results when meta-analyses are compared to large trials, and we assessed the prevalence of bias in published meta-analyses. Design: Medline search to identify pairs consisting of a meta-analysis and a single large trial (concordance of results was assumed if effects were in the same direction and the meta-analytic estimate was within 30% of the trial); analysis of funnel plots from 37 meta-analyses identified from a hand search of four leading general medicine journals 1993-6 and 38 meta-analyses from the second 1996 issue of the Cochrane Database of Systematic Reviews . Main outcome measure: Degree of funnel plot asymmetry as measured by the intercept from regression of standard normal deviates against precision. Results: In the eight pairs of meta-analysis and large trial that were identified (five from cardiovascular medicine, one from diabetic medicine, one from geriatric medicine, one from perinatal medicine) there were four concordant and four discordant pairs. In all cases discordance was due to meta-analyses showing larger effects. Funnel plot asymmetry was present in three out of four discordant pairs but in none of concordant pairs. In 14 (38%) journal meta-analyses and 5 (13%) Cochrane reviews, funnel plot asymmetry indicated that there was bias. Conclusions: A simple analysis of funnel plots provides a useful test for the likely presence of bias in meta-analyses, but as the capacity to detect bias will be limited when meta-analyses are based on a limited number of small trials the results from such analyses should be treated with considerable caution. Key messages Systematic reviews of randomised trials are the best strategy for appraising evidence; however, the findings of some meta-analyses were later contradicted by large trials Funnel plots, plots of the trials' effect estimates against sample size, are skewed and asymmetrical in the presence of publication bias and other biases Funnel plot asymmetry, measured by regression analysis, predicts discordance of results when meta-analyses are compared with single large trials Funnel plot asymmetry was found in 38% of meta-analyses published in leading general medicine journals and in 13% of reviews from the Cochrane Database of Systematic Reviews Critical examination of systematic reviews for publication and related biases should be considered a routine procedure
Why it matters
Tests whether asymmetry in funnel plots, graphs of study results against sample size, can detect bias in meta-analyses. Asymmetry indicated bias in 38% of journal meta-analyses and 13% of Cochrane reviews examined. It is a statistics method, not a dementia study.
New to this subject? Start with our plain-English guide: Alzheimer's in numbers →
How this study scored 57/100
Published in a leading specialty journal. Scored zero for study design, sample size, effect reported, studied in people and randomised / blinded because the record does not state it. We have no data on strength of evidence and senior author's track record, so those were left out: it earned 52 of the 91 points we could judge it on, which is 57 out of 100.
| Component | Points | Why |
|---|---|---|
| Journal tier | 9 / 10 | Tier 2 — leading specialty journal (BMJ) |
| Journal's citation record | 4 / 4 | BMJ: journal h-index 608 (OpenAlex, 2026-10-29) |
| Study design | 0 / 12 | No study design recorded — scores zero, not guessed (not stated in the record — scored 0) |
| Strength of evidence | — / 5 | No evidence rating recorded — left out of this score |
| Peer-review validation | 10 / 10 | Peer-reviewed. Published as an article in a named peer-reviewed journal. |
| How recent | 1 / 5 | 1997 — older than 20 years, still counted |
| Sample size | 0 / 4 | Participant count not stated — scores zero (not stated in the record — scored 0) |
| Effect reported | 0 / 3 | No effect size reported — scores zero (not stated in the record — scored 0) |
| Studied in people | 0 / 3 | Not stated whether this was studied in people — scores zero (not stated in the record — scored 0) |
| Randomised / blinded | 0 / 2 | No randomisation or blinding stated — scores zero (not stated in the record — scored 0) |
| Times cited | 7 / 7 | Cited 56,475 times (OpenAlex, 2026-10-10) |
| Citations per year | 5 / 5 | 1,821.8 citations a year over 31 year(s) since 1997 |
| Impact for its field and year | 6 / 6 | Cited 129.8× the average for papers of its field and year (OpenAlex FWCI) |
| Cited by studies here | 0 / 6 | No other study in this library cites it |
| Still being cited | 4 / 4 | Cited 12,610 times in 2024–2026 (2026 still in progress; OpenAlex) |
| Senior author's track record | — / 4 | OpenAlex's profile for last-listed author C. Minder holds only 5 work(s), too few to judge a track record — left out of this score |
| Institution's track record | 1 / 3 | University of Bristol: institution h-index 1,260 (OpenAlex) |
| Multi-centre collaboration | 1 / 3 | OpenAlex lists 2 institution(s) in 2 countries |
| Indexed in PubMed | 2 / 2 | Indexed in PubMed (PMID 9310563) |
| Free to read | 2 / 2 | Free to read |
| Points | 52 / 91 | Scored on 18 of 20 parts. We have no data for the rest, so they are left out: 52 of the 91 points we could judge it on is 57 out of 100. |
| Total | 57 / 100 | — |
What each of these 20 components measures, and what this score is not →
Read the original
Full text: free to read — OpenAlex lists this copy as free to read: the link above (checked 6 Feb 1020).
Cite this study APA, Vancouver, BibTeX · download .bib or .ris
Built from the record alzhub holds for this paper, not looked up. Volume, issue and page numbers are not held in the record. Check the citation against the original before you use it.
APA 7th
Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997). Bias in meta-analysis detected by a simple, graphical test. BMJ. https://doi.org/10.1136/bmj.315.7109.629
Vancouver
Egger M, Smith GD, Schneider M, Minder C. Bias in meta-analysis detected by a simple, graphical test. BMJ. 1997. doi:10.1136/bmj.315.7109.629. PMID: 9310563.
BibTeX
@article{egger1997-9310563,
author = {Egger, Matthias and Smith, George Davey and Schneider, Martin and Minder, C.},
title = {{Bias in meta-analysis detected by a simple, graphical test}},
journal = {BMJ},
year = {1997},
doi = {10.1136/bmj.315.7109.629},
pmid = {9310563},
url = {https://doi.org/10.1136/bmj.315.7109.629}
}