Abstract
BackgroundEach systematic review or systematic review version is a static snapshot. However, the evidence shift for several reasons, with new additions or problems identified in the synthesised evidence. The aim of this thesis is to explore the ways shifts in the underlying evidence could affect the reliability of systematic reviews, and consider the implications.
Methods
The growth of evidence and practices used in systematic reviews to deal with it was assessed in a series of descriptive studies, including a longitudinal study of updating in Cochrane reviews from 2003 to 2018, and a study of Cochrane reviews designated as having enough evidence. As a precursor to developing research integrity filters, a comprehensive search strategy was developed, the prevalence of retracted publications in PubMed and PubMed Central was estimated, and their discoverability assessed. Research integrity filters for post-publication events that can compromise publications were developed and piloted in assessing 36,462 trials included in Cochrane reviews and 83,302 non-Cochrane systematic reviews. The prevalence of potentially compromised included trials affecting Cochrane reviews was estimated.
Results
The number of trials and systematic reviews continues to increase dramatically, with ongoing and published clinical trials rising more quickly than systematic reviews. The number of Cochrane reviews has plateaued, and they are increasingly out of date. The median time to update of a cohort of reviews updated in 2003 was three years, but by the end of 2018, the median time since their last update was seven years.
It is uncommon for systematic reviews to resolve questions about health care, as results often give rise to further questions about the optimal use of care, and which groups of people could benefit. Other areas of concern to clinicians and patients, however, may not attract research activity. Systematic reviews may be more likely to end because of a lack of expected research than because there is enough evidence.
It is uncommon for systematic reviews to resolve questions about health care, as results often give rise to further questions about the optimal use of care, and which groups of people could benefit. Other areas of concern to clinicians and patients, however, may not attract research activity. Systematic reviews may be more likely to end because of a lack of expected research than because there is enough evidence.
Emerging evidence of compromised trials could have a similar effect on the reliability of a systematic review’s results as new studies but monitoring of these events by systematic review groups appears inadequate. System problems and journal practice contribute: a substantial proportion (over 30%) of retracted publications in PubMed/PMC are not indexed as such, and journal compliance with recommended practice is low. Using pilot research filters, 28% of a group of 2,025 Cochrane reviews with above average numbers of included studies were found to be affected by at least one trial that had been retracted or had an erratum or expression of concern published.
Conclusions
The increase in evidence and problems identified in trials is outpacing the methods, infrastructure, and collaborations needed to enable systematic reviewers to keep up with shifting evidence. Research integrity filters were shown to improve identification of potentially problematic trials. Centralised efforts to enable timely and efficient responses to these trials should be undertaken.
| Date of Award | 3 Feb 2021 |
|---|---|
| Original language | English |
| Supervisor | Paul Glasziou (Supervisor) & Chris Del Mar (Supervisor) |