Indirect evidence of reporting biases was found in a survey of medical research studies

Loai N Albarqouni, José A. López-López, Julian P T Higgins

Research output: Contribution to journalArticleResearchpeer-review

2 Citations (Scopus)
82 Downloads (Pure)

Abstract

OBJECTIVES: To explore indirect evidence of reporting biases by examining the distribution of P-values reported in published medical articles and to compare P-values distributions across different contexts.

STUDY DESIGN AND SETTING: We selected a random sample (N = 1,500) of articles published in PubMed in March 2014. We extracted information on study type, design, medical discipline, and P-values for the first reported outcome and primary outcome (if specified) from each article. We plotted the P-values transformed to the z-score scale and used caliper tests to investigate threshold effects.

RESULTS: Out of the 1,500 randomly selected records, 758 (50.5%) were included. We retrieved or calculated 758 P-values for first reported outcomes and 389 for primary outcomes (specified in only 51% of included studies). The first reported and the primary outcome differed in 28% (110/389) of the included studies. The distributions of P-values for first reported outcomes and primary outcomes showed a notable discontinuity at the common thresholds of statistical significance (P-value = 0.05 and P-value = 0.01). We also found marked discontinuities in the distributions of z-scores across various medical disciplines, study designs, and types.

CONCLUSION: Reporting biases are still common in medical research. We discuss their implications, strategies to detect them, and recommended practices to avoid them.

Original languageEnglish
Pages (from-to)57-64
Number of pages8
JournalJournal of Clinical Epidemiology
Volume83
Early online date11 Jan 2017
DOIs
Publication statusPublished - 1 Mar 2017

Fingerprint

PubMed
Biomedical Research
Surveys and Questionnaires

Cite this

@article{a6234e9ef53c45df9881da09b36d4d94,
title = "Indirect evidence of reporting biases was found in a survey of medical research studies",
abstract = "OBJECTIVES: To explore indirect evidence of reporting biases by examining the distribution of P-values reported in published medical articles and to compare P-values distributions across different contexts.STUDY DESIGN AND SETTING: We selected a random sample (N = 1,500) of articles published in PubMed in March 2014. We extracted information on study type, design, medical discipline, and P-values for the first reported outcome and primary outcome (if specified) from each article. We plotted the P-values transformed to the z-score scale and used caliper tests to investigate threshold effects.RESULTS: Out of the 1,500 randomly selected records, 758 (50.5{\%}) were included. We retrieved or calculated 758 P-values for first reported outcomes and 389 for primary outcomes (specified in only 51{\%} of included studies). The first reported and the primary outcome differed in 28{\%} (110/389) of the included studies. The distributions of P-values for first reported outcomes and primary outcomes showed a notable discontinuity at the common thresholds of statistical significance (P-value = 0.05 and P-value = 0.01). We also found marked discontinuities in the distributions of z-scores across various medical disciplines, study designs, and types.CONCLUSION: Reporting biases are still common in medical research. We discuss their implications, strategies to detect them, and recommended practices to avoid them.",
author = "Albarqouni, {Loai N} and L{\'o}pez-L{\'o}pez, {Jos{\'e} A.} and Higgins, {Julian P T}",
note = "Copyright {\circledC} 2017 Elsevier Inc. All rights reserved.",
year = "2017",
month = "3",
day = "1",
doi = "10.1016/j.jclinepi.2016.11.013",
language = "English",
volume = "83",
pages = "57--64",
journal = "Journal of Chronic Diseases",
issn = "0895-4356",
publisher = "Elsevier",

}

Indirect evidence of reporting biases was found in a survey of medical research studies. / Albarqouni, Loai N; López-López, José A. ; Higgins, Julian P T.

In: Journal of Clinical Epidemiology, Vol. 83, 01.03.2017, p. 57-64.

Research output: Contribution to journalArticleResearchpeer-review

TY - JOUR

T1 - Indirect evidence of reporting biases was found in a survey of medical research studies

AU - Albarqouni, Loai N

AU - López-López, José A.

AU - Higgins, Julian P T

N1 - Copyright © 2017 Elsevier Inc. All rights reserved.

PY - 2017/3/1

Y1 - 2017/3/1

N2 - OBJECTIVES: To explore indirect evidence of reporting biases by examining the distribution of P-values reported in published medical articles and to compare P-values distributions across different contexts.STUDY DESIGN AND SETTING: We selected a random sample (N = 1,500) of articles published in PubMed in March 2014. We extracted information on study type, design, medical discipline, and P-values for the first reported outcome and primary outcome (if specified) from each article. We plotted the P-values transformed to the z-score scale and used caliper tests to investigate threshold effects.RESULTS: Out of the 1,500 randomly selected records, 758 (50.5%) were included. We retrieved or calculated 758 P-values for first reported outcomes and 389 for primary outcomes (specified in only 51% of included studies). The first reported and the primary outcome differed in 28% (110/389) of the included studies. The distributions of P-values for first reported outcomes and primary outcomes showed a notable discontinuity at the common thresholds of statistical significance (P-value = 0.05 and P-value = 0.01). We also found marked discontinuities in the distributions of z-scores across various medical disciplines, study designs, and types.CONCLUSION: Reporting biases are still common in medical research. We discuss their implications, strategies to detect them, and recommended practices to avoid them.

AB - OBJECTIVES: To explore indirect evidence of reporting biases by examining the distribution of P-values reported in published medical articles and to compare P-values distributions across different contexts.STUDY DESIGN AND SETTING: We selected a random sample (N = 1,500) of articles published in PubMed in March 2014. We extracted information on study type, design, medical discipline, and P-values for the first reported outcome and primary outcome (if specified) from each article. We plotted the P-values transformed to the z-score scale and used caliper tests to investigate threshold effects.RESULTS: Out of the 1,500 randomly selected records, 758 (50.5%) were included. We retrieved or calculated 758 P-values for first reported outcomes and 389 for primary outcomes (specified in only 51% of included studies). The first reported and the primary outcome differed in 28% (110/389) of the included studies. The distributions of P-values for first reported outcomes and primary outcomes showed a notable discontinuity at the common thresholds of statistical significance (P-value = 0.05 and P-value = 0.01). We also found marked discontinuities in the distributions of z-scores across various medical disciplines, study designs, and types.CONCLUSION: Reporting biases are still common in medical research. We discuss their implications, strategies to detect them, and recommended practices to avoid them.

UR - http://www.scopus.com/inward/record.url?scp=85011578642&partnerID=8YFLogxK

U2 - 10.1016/j.jclinepi.2016.11.013

DO - 10.1016/j.jclinepi.2016.11.013

M3 - Article

VL - 83

SP - 57

EP - 64

JO - Journal of Chronic Diseases

JF - Journal of Chronic Diseases

SN - 0895-4356

ER -