Systematic reviews and meta-analysis Hynek Pikhart Review (narrative review) • A review is any attempt to synthesize the results and conclusions of two or more publications on a given topic. Systematic review • A systematic review is a review that aims comprehensively to identify and synthesize all the literature on a given topic (sometimes called an overview). Each specific study forms a unit of analysis and the same scientific principles and rigour apply as for any study. If a review does not state clearly whether and how all relevant studies were identified and synthesized, it is not a systematic review • A rigorous, unbiased and systematic summary of available research evidence (usually peerreviewed) on a certain topic Meta-analysis • Meta-analysis is a statistical technique for assembling the results of several studies in a review into a single numerical estimate. • A systematic review involves – a well-formulated question – Developing protocol including research question, search and inclusion/exclusion – a comprehensive and replicable data search – unbiased screening, selection and abstraction – critical appraisal of data – analysis of findings and risk of bias – valid synthesis of data – interpretation of findings and write up • A meta-analysis involves systematic analysis of the results, often with the aim to produce a single estimate of an intervention effect. A meta-analysis can only be done • when more than one study has estimated an effect • when there are no differences in the study characteristics that are likely to substantially affect outcome • when the outcome has been measured in similar ways • and when the data are available. Data sources for a systematic review I • PubMed database • Web of Science, Google, Google Scholar (books are not in Medline or PubMed but some of them will be in Google Scholar) • Cochrane library • Other medical and non-medical databases (PubMed covers medical literature while some literature relevant to social epidemiology will not be covered; other databases are needed, eg. PsychInfo, Social Sciences Citation Index) Data sources for a systematic review II • Foreign language literature • "Grey literature" (theses, internal reports, nonpeer reviewed journals, pharmaceutical industry files) • References (and references of references, etc) listed in primary sources • Other unpublished sources known to experts in the field (seek by personal communication) • Raw data from published trials (seek by personal communication) Guidance • Preparation of review and systematic review of literature • Critical evaluation of papers PRISMA stands for Preferred Reporting Items for Systematic Reviews and Meta-Analyses. It is an evidence-based minimum set of items for reporting in systematic reviews and meta-analyses. Meta-analysis • systematic analysis of the results • the aim is to produce a single estimate of an intervention effect. Copyright ©1997 BMJ Publishing Group Ltd. Egger, M. et al. BMJ 1997;315:1533-1537 Fig 1 Total mortality from trials of beta blockers in secondary prevention after myocardial infarction. The black square and horizontal line correspond to odds ratio and 95% confidence interval for each trial. The size of the black square reflects the weight of each trial. The diamond represents the combined odds ratio and 95% confidence interval, showing 22% a reduction in the odds of death (references are available from the authors) Meta-analysis • Meta-analysis is a statistical technique for assembling the results of several studies in a review into a single numerical estimate. • Rationale: – Single studies too small to give clear results – Single studies not generalizable – Increased total size of the combined analysis increases chances of detecting a moderate but clinically and/or epidemiologically important effect Potential biases in meta-analysis • Publication bias • English language bias • Database bias • Citation bias • Multiple publication bias • Bias in provision of data • Poor methodological quality of small studies The Funnel plot • A screening test for bias • Plot of the effect estimate against sample size • If skewed and asymmetric, then bias probably present • Small negative studies are often missing Of meta-analyses examined, 38% in medical journals and 19% in Cochrane Library showed evidence of bias (Egger et al BMJ 1997) Example of marked publication bias - CRP and prognosis of stable coronary artery disease Relative Risk Each dot represents one study, N=83 studies • Hemingway et al 2010 Egger, M. et al. BMJ 1998;316:61-66 Funnel plot of mortality results from trials of beta-blockers in secondary prevention after myocardial infarction. The odds ratios are plotted against study sample size • Visual assessment shows some asymmetry • It indicates that there was selective non-publication of smaller trials with less sizeable benefit. • However, in formal statistical analysis the degree of asymmetry is found to be small and non-significant (P>0.1). • Furthermore, exclusion of the smaller studies had little effect on the overall estimate. • Bias does not therefore seem to have distorted the findings from this meta- analysis. Egger, M. et al. BMJ 1998;316:61-66 Resources • Greenhalgh T. Papers that summarise other papers (systematic reviews and meta-analyses). British Medical Journal 1997; 315: 672-675 • Akobeng AK. Understanding systematic reviews and meta-analysis. Arch. Dis. Child., 2005; 90(8): 845 - 848.