Rethinking credible evidence synthesis

Government regulators and systematic reviewers both aim to generate an accurate understanding of the effects of interventions. However, the methods they use, and the evidence they consider, are different. Although both focus on randomised clinical trials for determining safety and efficacy, the mostly academic community of systematic reviewers generally get their data from published reports of clinical trials. By contrast, regulators such as the US Food and Drug Administration (FDA) evaluate a far more diverse array of data including large, computerised datasets for each clinical trial as well as a trial’s protocol and clinical study report. This is probably the most complete, single report of a trial. (Regulators can also conduct clinical site inspections, request confidential and private details such as case report forms, and inspect manufacturing facilities.) This means that while regulators and systematic reviewers may assess the same clinical trials, the data they look at differs substantially. In our latest Cochrane systematic review of the neuraminidase inhibitor class of influenza antivirals, we used a new method based on data previously seen only by regulators—over 22 000 pages of clinical study reports and over 2700 pages of regulatory comments.1 The results changed our understanding of oseltamivir (Tamiflu) and our feelings about the viability of reliable evidence synthesis in general.