GRADE guidelines: 7. Rating the quality of evidence - Inconsistency

Gordon H. Guyatt, Andrew D. Oxman, Regina Kunz, James Woodcock, Jan Brozek, Mark Helfand, Pablo Alonso-Coello, Paul Glasziou, Roman Jaeschke, Elie A. Akl, Susan Norris, Gunn Vist, Philipp Dahm, Vijay K. Shukla, Julian Higgins, Yngve Falck-Ytter, Holger J. Schünemann

Research output: Contribution to journalArticleResearchpeer-review

771 Citations (Scopus)

Abstract

This article deals with inconsistency of relative (rather than absolute) treatment effects in binary/dichotomous outcomes. A body of evidence is not rated up in quality if studies yield consistent results, but may be rated down in quality if inconsistent. Criteria for evaluating consistency include similarity of point estimates, extent of overlap of confidence intervals, and statistical criteria including tests of heterogeneity and I 2. To explore heterogeneity, systematic review authors should generate and test a small number of a priori hypotheses related to patients, interventions, outcomes, and methodology. When inconsistency is large and unexplained, rating down quality for inconsistency is appropriate, particularly if some studies suggest substantial benefit, and others no effect or harm (rather than only large vs. small effects). Apparent subgroup effects may be spurious. Credibility is increased if subgroup effects are based on a small number of a priori hypotheses with a specified direction; subgroup comparisons come from within rather than between studies; tests of interaction generate low P-values; and have a biological rationale.

Original languageEnglish
Pages (from-to)1294-1302
Number of pages9
JournalJournal of Clinical Epidemiology
Volume64
Issue number12
DOIs
Publication statusPublished - Dec 2011

Fingerprint

Guidelines
Confidence Intervals
Therapeutics
Direction compound

Cite this

Guyatt, G. H., Oxman, A. D., Kunz, R., Woodcock, J., Brozek, J., Helfand, M., ... Schünemann, H. J. (2011). GRADE guidelines: 7. Rating the quality of evidence - Inconsistency. Journal of Clinical Epidemiology, 64(12), 1294-1302. https://doi.org/10.1016/j.jclinepi.2011.03.017
Guyatt, Gordon H. ; Oxman, Andrew D. ; Kunz, Regina ; Woodcock, James ; Brozek, Jan ; Helfand, Mark ; Alonso-Coello, Pablo ; Glasziou, Paul ; Jaeschke, Roman ; Akl, Elie A. ; Norris, Susan ; Vist, Gunn ; Dahm, Philipp ; Shukla, Vijay K. ; Higgins, Julian ; Falck-Ytter, Yngve ; Schünemann, Holger J. / GRADE guidelines: 7. Rating the quality of evidence - Inconsistency. In: Journal of Clinical Epidemiology. 2011 ; Vol. 64, No. 12. pp. 1294-1302.
@article{9377ce9bb40e4616a3211425c93084df,
title = "GRADE guidelines: 7. Rating the quality of evidence - Inconsistency",
abstract = "This article deals with inconsistency of relative (rather than absolute) treatment effects in binary/dichotomous outcomes. A body of evidence is not rated up in quality if studies yield consistent results, but may be rated down in quality if inconsistent. Criteria for evaluating consistency include similarity of point estimates, extent of overlap of confidence intervals, and statistical criteria including tests of heterogeneity and I 2. To explore heterogeneity, systematic review authors should generate and test a small number of a priori hypotheses related to patients, interventions, outcomes, and methodology. When inconsistency is large and unexplained, rating down quality for inconsistency is appropriate, particularly if some studies suggest substantial benefit, and others no effect or harm (rather than only large vs. small effects). Apparent subgroup effects may be spurious. Credibility is increased if subgroup effects are based on a small number of a priori hypotheses with a specified direction; subgroup comparisons come from within rather than between studies; tests of interaction generate low P-values; and have a biological rationale.",
author = "Guyatt, {Gordon H.} and Oxman, {Andrew D.} and Regina Kunz and James Woodcock and Jan Brozek and Mark Helfand and Pablo Alonso-Coello and Paul Glasziou and Roman Jaeschke and Akl, {Elie A.} and Susan Norris and Gunn Vist and Philipp Dahm and Shukla, {Vijay K.} and Julian Higgins and Yngve Falck-Ytter and Sch{\"u}nemann, {Holger J.}",
year = "2011",
month = "12",
doi = "10.1016/j.jclinepi.2011.03.017",
language = "English",
volume = "64",
pages = "1294--1302",
journal = "Journal of Chronic Diseases",
issn = "0895-4356",
publisher = "Elsevier",
number = "12",

}

Guyatt, GH, Oxman, AD, Kunz, R, Woodcock, J, Brozek, J, Helfand, M, Alonso-Coello, P, Glasziou, P, Jaeschke, R, Akl, EA, Norris, S, Vist, G, Dahm, P, Shukla, VK, Higgins, J, Falck-Ytter, Y & Schünemann, HJ 2011, 'GRADE guidelines: 7. Rating the quality of evidence - Inconsistency', Journal of Clinical Epidemiology, vol. 64, no. 12, pp. 1294-1302. https://doi.org/10.1016/j.jclinepi.2011.03.017

GRADE guidelines: 7. Rating the quality of evidence - Inconsistency. / Guyatt, Gordon H.; Oxman, Andrew D.; Kunz, Regina; Woodcock, James; Brozek, Jan; Helfand, Mark; Alonso-Coello, Pablo; Glasziou, Paul; Jaeschke, Roman; Akl, Elie A.; Norris, Susan; Vist, Gunn; Dahm, Philipp; Shukla, Vijay K.; Higgins, Julian; Falck-Ytter, Yngve; Schünemann, Holger J.

In: Journal of Clinical Epidemiology, Vol. 64, No. 12, 12.2011, p. 1294-1302.

Research output: Contribution to journalArticleResearchpeer-review

TY - JOUR

T1 - GRADE guidelines: 7. Rating the quality of evidence - Inconsistency

AU - Guyatt, Gordon H.

AU - Oxman, Andrew D.

AU - Kunz, Regina

AU - Woodcock, James

AU - Brozek, Jan

AU - Helfand, Mark

AU - Alonso-Coello, Pablo

AU - Glasziou, Paul

AU - Jaeschke, Roman

AU - Akl, Elie A.

AU - Norris, Susan

AU - Vist, Gunn

AU - Dahm, Philipp

AU - Shukla, Vijay K.

AU - Higgins, Julian

AU - Falck-Ytter, Yngve

AU - Schünemann, Holger J.

PY - 2011/12

Y1 - 2011/12

N2 - This article deals with inconsistency of relative (rather than absolute) treatment effects in binary/dichotomous outcomes. A body of evidence is not rated up in quality if studies yield consistent results, but may be rated down in quality if inconsistent. Criteria for evaluating consistency include similarity of point estimates, extent of overlap of confidence intervals, and statistical criteria including tests of heterogeneity and I 2. To explore heterogeneity, systematic review authors should generate and test a small number of a priori hypotheses related to patients, interventions, outcomes, and methodology. When inconsistency is large and unexplained, rating down quality for inconsistency is appropriate, particularly if some studies suggest substantial benefit, and others no effect or harm (rather than only large vs. small effects). Apparent subgroup effects may be spurious. Credibility is increased if subgroup effects are based on a small number of a priori hypotheses with a specified direction; subgroup comparisons come from within rather than between studies; tests of interaction generate low P-values; and have a biological rationale.

AB - This article deals with inconsistency of relative (rather than absolute) treatment effects in binary/dichotomous outcomes. A body of evidence is not rated up in quality if studies yield consistent results, but may be rated down in quality if inconsistent. Criteria for evaluating consistency include similarity of point estimates, extent of overlap of confidence intervals, and statistical criteria including tests of heterogeneity and I 2. To explore heterogeneity, systematic review authors should generate and test a small number of a priori hypotheses related to patients, interventions, outcomes, and methodology. When inconsistency is large and unexplained, rating down quality for inconsistency is appropriate, particularly if some studies suggest substantial benefit, and others no effect or harm (rather than only large vs. small effects). Apparent subgroup effects may be spurious. Credibility is increased if subgroup effects are based on a small number of a priori hypotheses with a specified direction; subgroup comparisons come from within rather than between studies; tests of interaction generate low P-values; and have a biological rationale.

UR - http://www.scopus.com/inward/record.url?scp=80054981259&partnerID=8YFLogxK

U2 - 10.1016/j.jclinepi.2011.03.017

DO - 10.1016/j.jclinepi.2011.03.017

M3 - Article

VL - 64

SP - 1294

EP - 1302

JO - Journal of Chronic Diseases

JF - Journal of Chronic Diseases

SN - 0895-4356

IS - 12

ER -

Guyatt GH, Oxman AD, Kunz R, Woodcock J, Brozek J, Helfand M et al. GRADE guidelines: 7. Rating the quality of evidence - Inconsistency. Journal of Clinical Epidemiology. 2011 Dec;64(12):1294-1302. https://doi.org/10.1016/j.jclinepi.2011.03.017