Searching for unpublished trials in Cochrane reviews may not be worth the effort

Objective: To assess the value of searching for unpublished data by exploring the extent to which Cochrane reviews include unpublished data and by evaluating the quality of unpublished trials. Study Design and Setting: We screened all 2,462 completed Cochrane reviews published since 2000 in the Cochrane Database of Systematic Reviews Issue 3, 2006. In a random sample (n = 61) of 292 reviews, including unpublished trials, we studied all 116 references. Results: Unpublished trials make up 8.8% of all included trials in our sample. Thirty-eight percent of the "unpublished" trials have in fact been published. Allocation concealment was "unclear" or not adequate in 54.3% and 61.3% reported blinding. In 47.2% reported withdrawal rates were >20%. Trials that were eventually published had larger mean population sizes (P-value, 0.02). Of the reported sponsors, 87.3% were drug companies. Methodological quality and publication bias are mentioned in half of the reviews and explored in a third. Quality ratings did not have consequences for pooling, because 82.8% was included in the forest plots. Conclusions: A minority of Cochrane reviews include "unpublished trials" and many of these are eventually published. Truly unpublished studies have poor or unclear methodological quality. Therefore, it may be better to invest in regular updating of reviews, rather than in extensive searching for unpublished data.