Instruments assessing risk of bias of randomized trials frequently included items that are not addressing risk of bias issues

Ying Wang; Maryam Ghadimi; Qi Wang; Liangying Hou; Dena Zeraatkar; Atiya Iqbal; Cameron Ho; Liang Yao; Malini Hu; Zhikang Ye; Rachel Couban; Susan Armijo-Olivo; Dirk Bassler; Matthias Briel; Lise Lotte Gluud; Paul Glasziou; Rod Jackson; Sheri A Keitz; Luz M Letelier; Philippe Ravaud; Kenneth F Schulz; Reed A C Siemieniuk; Romina Brignardello-Petersen; Gordon H Guyatt

doi:10.1016/j.jclinepi.2022.10.018

Instruments assessing risk of bias of randomized trials frequently included items that are not addressing risk of bias issues

Ying Wang, Maryam Ghadimi, Qi Wang, Liangying Hou, Dena Zeraatkar, Atiya Iqbal, Cameron Ho, Liang Yao, Malini Hu, Zhikang Ye, Rachel Couban, Susan Armijo-Olivo, Dirk Bassler, Matthias Briel, Lise Lotte Gluud, Paul Glasziou, Rod Jackson, Sheri A Keitz, Luz M Letelier, Philippe RavaudKenneth F Schulz, Reed A C Siemieniuk, Romina Brignardello-Petersen, Gordon H Guyatt

Research output: Contribution to journal › Article › Research › peer-review

6 Citations (Scopus)

Abstract

OBJECTIVES: To establish whether items included in instruments published in the last decade assessing risk of bias of randomized controlled trials (RCTs) are indeed addressing risk of bias.

STUDY DESIGN AND SETTING: We searched Medline, Embase, Web of Science, and Scopus from 2010 to October 2021 for instruments assessing risk of bias of RCTs. By extracting items and summarizing their essential content, we generated an item list. Items that two reviewers agreed clearly did not address risk of bias were excluded. We included the remaining items in a survey in which 13 experts judged the issue each item is addressing: risk of bias, applicability, random error, reporting quality, or none of the above.

RESULTS: Seventeen eligible instruments included 127 unique items. After excluding 61 items deemed as clearly not addressing risk of bias, the item classification survey included 66 items, of which the majority of respondents deemed 20 items (30.3%) as addressing risk of bias; the majority deemed 11 (16.7%) as not addressing risk of bias; and there proved substantial disagreement for 35 (53.0%) items.

CONCLUSION: Existing risk of bias instruments frequently include items that do not address risk of bias. For many items, experts disagree on whether or not they are addressing risk of bias.

Original language	English
Pages (from-to)	218-225
Number of pages	8
Journal	Journal of Clinical Epidemiology
Volume	152
Early online date	28 Oct 2022
DOIs	https://doi.org/10.1016/j.jclinepi.2022.10.018
Publication status	Published - Dec 2022

Access to Document

10.1016/j.jclinepi.2022.10.018

Cite this

Wang, Y., Ghadimi, M., Wang, Q., Hou, L., Zeraatkar, D., Iqbal, A., Ho, C., Yao, L., Hu, M., Ye, Z., Couban, R., Armijo-Olivo, S., Bassler, D., Briel, M., Gluud, L. L., Glasziou, P., Jackson, R., Keitz, S. A., Letelier, L. M., ... Guyatt, G. H. (2022). Instruments assessing risk of bias of randomized trials frequently included items that are not addressing risk of bias issues. Journal of Clinical Epidemiology, 152, 218-225. https://doi.org/10.1016/j.jclinepi.2022.10.018

@article{45a1f5b4e36141daa0f8ed32ff193fde,

title = "Instruments assessing risk of bias of randomized trials frequently included items that are not addressing risk of bias issues",

abstract = "OBJECTIVES: To establish whether items included in instruments published in the last decade assessing risk of bias of randomized controlled trials (RCTs) are indeed addressing risk of bias.STUDY DESIGN AND SETTING: We searched Medline, Embase, Web of Science, and Scopus from 2010 to October 2021 for instruments assessing risk of bias of RCTs. By extracting items and summarizing their essential content, we generated an item list. Items that two reviewers agreed clearly did not address risk of bias were excluded. We included the remaining items in a survey in which 13 experts judged the issue each item is addressing: risk of bias, applicability, random error, reporting quality, or none of the above.RESULTS: Seventeen eligible instruments included 127 unique items. After excluding 61 items deemed as clearly not addressing risk of bias, the item classification survey included 66 items, of which the majority of respondents deemed 20 items (30.3%) as addressing risk of bias; the majority deemed 11 (16.7%) as not addressing risk of bias; and there proved substantial disagreement for 35 (53.0%) items.CONCLUSION: Existing risk of bias instruments frequently include items that do not address risk of bias. For many items, experts disagree on whether or not they are addressing risk of bias.",

author = "Ying Wang and Maryam Ghadimi and Qi Wang and Liangying Hou and Dena Zeraatkar and Atiya Iqbal and Cameron Ho and Liang Yao and Malini Hu and Zhikang Ye and Rachel Couban and Susan Armijo-Olivo and Dirk Bassler and Matthias Briel and Gluud, {Lise Lotte} and Paul Glasziou and Rod Jackson and Keitz, {Sheri A} and Letelier, {Luz M} and Philippe Ravaud and Schulz, {Kenneth F} and Siemieniuk, {Reed A C} and Romina Brignardello-Petersen and Guyatt, {Gordon H}",

year = "2022",

month = dec,

doi = "10.1016/j.jclinepi.2022.10.018",

language = "English",

volume = "152",

pages = "218--225",

journal = "Journal of Clinical Epidemiology",

issn = "0895-4356",

publisher = "Elsevier",

}

Wang, Y, Ghadimi, M, Wang, Q, Hou, L, Zeraatkar, D, Iqbal, A, Ho, C, Yao, L, Hu, M, Ye, Z, Couban, R, Armijo-Olivo, S, Bassler, D, Briel, M, Gluud, LL, Glasziou, P, Jackson, R, Keitz, SA, Letelier, LM, Ravaud, P, Schulz, KF, Siemieniuk, RAC, Brignardello-Petersen, R & Guyatt, GH 2022, 'Instruments assessing risk of bias of randomized trials frequently included items that are not addressing risk of bias issues', Journal of Clinical Epidemiology, vol. 152, pp. 218-225. https://doi.org/10.1016/j.jclinepi.2022.10.018

TY - JOUR

T1 - Instruments assessing risk of bias of randomized trials frequently included items that are not addressing risk of bias issues

AU - Wang, Ying

AU - Ghadimi, Maryam

AU - Wang, Qi

AU - Hou, Liangying

AU - Zeraatkar, Dena

AU - Iqbal, Atiya

AU - Ho, Cameron

AU - Yao, Liang

AU - Hu, Malini

AU - Ye, Zhikang

AU - Couban, Rachel

AU - Armijo-Olivo, Susan

AU - Bassler, Dirk

AU - Briel, Matthias

AU - Gluud, Lise Lotte

AU - Glasziou, Paul

AU - Jackson, Rod

AU - Keitz, Sheri A

AU - Letelier, Luz M

AU - Ravaud, Philippe

AU - Schulz, Kenneth F

AU - Siemieniuk, Reed A C

AU - Brignardello-Petersen, Romina

AU - Guyatt, Gordon H

PY - 2022/12

Y1 - 2022/12

N2 - OBJECTIVES: To establish whether items included in instruments published in the last decade assessing risk of bias of randomized controlled trials (RCTs) are indeed addressing risk of bias.STUDY DESIGN AND SETTING: We searched Medline, Embase, Web of Science, and Scopus from 2010 to October 2021 for instruments assessing risk of bias of RCTs. By extracting items and summarizing their essential content, we generated an item list. Items that two reviewers agreed clearly did not address risk of bias were excluded. We included the remaining items in a survey in which 13 experts judged the issue each item is addressing: risk of bias, applicability, random error, reporting quality, or none of the above.RESULTS: Seventeen eligible instruments included 127 unique items. After excluding 61 items deemed as clearly not addressing risk of bias, the item classification survey included 66 items, of which the majority of respondents deemed 20 items (30.3%) as addressing risk of bias; the majority deemed 11 (16.7%) as not addressing risk of bias; and there proved substantial disagreement for 35 (53.0%) items.CONCLUSION: Existing risk of bias instruments frequently include items that do not address risk of bias. For many items, experts disagree on whether or not they are addressing risk of bias.

AB - OBJECTIVES: To establish whether items included in instruments published in the last decade assessing risk of bias of randomized controlled trials (RCTs) are indeed addressing risk of bias.STUDY DESIGN AND SETTING: We searched Medline, Embase, Web of Science, and Scopus from 2010 to October 2021 for instruments assessing risk of bias of RCTs. By extracting items and summarizing their essential content, we generated an item list. Items that two reviewers agreed clearly did not address risk of bias were excluded. We included the remaining items in a survey in which 13 experts judged the issue each item is addressing: risk of bias, applicability, random error, reporting quality, or none of the above.RESULTS: Seventeen eligible instruments included 127 unique items. After excluding 61 items deemed as clearly not addressing risk of bias, the item classification survey included 66 items, of which the majority of respondents deemed 20 items (30.3%) as addressing risk of bias; the majority deemed 11 (16.7%) as not addressing risk of bias; and there proved substantial disagreement for 35 (53.0%) items.CONCLUSION: Existing risk of bias instruments frequently include items that do not address risk of bias. For many items, experts disagree on whether or not they are addressing risk of bias.

U2 - 10.1016/j.jclinepi.2022.10.018

DO - 10.1016/j.jclinepi.2022.10.018

M3 - Article

C2 - 36424692

SN - 0895-4356

VL - 152

SP - 218

EP - 225

JO - Journal of Clinical Epidemiology

JF - Journal of Clinical Epidemiology

ER -

Instruments assessing risk of bias of randomized trials frequently included items that are not addressing risk of bias issues

Abstract

Access to Document

Fingerprint

Cite this