The two-week systematic review (2weekSR) method was successfully blind-replicated by another team: a case study

Catalin Tufanaru; Didi Surian; Anna Mae Scott; Paul Glasziou; Enrico Coiera

doi:10.1016/j.jclinepi.2023.10.013

The two-week systematic review (2weekSR) method was successfully blind-replicated by another team: a case study

Catalin Tufanaru, Didi Surian^*, Anna Mae Scott, Paul Glasziou, Enrico Coiera

^*Corresponding author for this work

Institute for Evidence-Based Healthcare

Research output: Contribution to journal › Article › Research › peer-review

1 Citation (Scopus)

Abstract

OBJECTIVE: To assess the replicability of a 2-week systematic review (index 2weekSR) created with the assistance of automation tools using the fidelity method.

METHODS: A PRISMA-compliant SR protocol was developed based on the published information of the index 2weekSR study. The replication team consisted of three reviewers. Two reviewers blocked off time during the replication. The total time to complete tasks and the meta-analysis results were compared with the index 2weekSR study. Review process fidelity scores were calculated for review methods and outcomes. Barriers to completing the replication were identified.

RESULTS: The review was completed over 63 person-hours (11 workdays/15 calendar days). A fidelity score (FS) of 0.95 was achieved for the Methods, with 3 (of 8) tasks only partially replicated, and an FS of 0.63 for the Outcomes, with 6 (of 7) only partially replicated and 1 task was not replicated. Non-replication was mainly caused by missing information in the index 2weekSR study that was not required in standard reporting guidelines. The replication arrived at the same conclusions as the original study.

CONCLUSIONS: A 2weekSR study was replicated by a small team of 3 reviewers supported by automation tools. Including additional information when reporting SRs should improve their replicability.

Original language	English
Pages (from-to)	1-20
Number of pages	20
Journal	Journal of Clinical Epidemiology
DOIs	https://doi.org/10.1016/j.jclinepi.2023.10.013
Publication status	Accepted/In press - 23 Oct 2023

Access to Document

10.1016/j.jclinepi.2023.10.013Licence: CC BY

Cite this

@article{3764501c7e474ce99bc0bdbd5d9104d6,

title = "The two-week systematic review (2weekSR) method was successfully blind-replicated by another team: a case study",

abstract = "OBJECTIVE: To assess the replicability of a 2-week systematic review (index 2weekSR) created with the assistance of automation tools using the fidelity method.METHODS: A PRISMA-compliant SR protocol was developed based on the published information of the index 2weekSR study. The replication team consisted of three reviewers. Two reviewers blocked off time during the replication. The total time to complete tasks and the meta-analysis results were compared with the index 2weekSR study. Review process fidelity scores were calculated for review methods and outcomes. Barriers to completing the replication were identified.RESULTS: The review was completed over 63 person-hours (11 workdays/15 calendar days). A fidelity score (FS) of 0.95 was achieved for the Methods, with 3 (of 8) tasks only partially replicated, and an FS of 0.63 for the Outcomes, with 6 (of 7) only partially replicated and 1 task was not replicated. Non-replication was mainly caused by missing information in the index 2weekSR study that was not required in standard reporting guidelines. The replication arrived at the same conclusions as the original study.CONCLUSIONS: A 2weekSR study was replicated by a small team of 3 reviewers supported by automation tools. Including additional information when reporting SRs should improve their replicability.",

author = "Catalin Tufanaru and Didi Surian and Scott, {Anna Mae} and Paul Glasziou and Enrico Coiera",

year = "2023",

month = oct,

day = "23",

doi = "10.1016/j.jclinepi.2023.10.013",

language = "English",

pages = "1--20",

journal = "Journal of Clinical Epidemiology",

issn = "0895-4356",

publisher = "Elsevier",

}

TY - JOUR

T1 - The two-week systematic review (2weekSR) method was successfully blind-replicated by another team: a case study

AU - Tufanaru, Catalin

AU - Surian, Didi

AU - Scott, Anna Mae

AU - Glasziou, Paul

AU - Coiera, Enrico

PY - 2023/10/23

Y1 - 2023/10/23

N2 - OBJECTIVE: To assess the replicability of a 2-week systematic review (index 2weekSR) created with the assistance of automation tools using the fidelity method.METHODS: A PRISMA-compliant SR protocol was developed based on the published information of the index 2weekSR study. The replication team consisted of three reviewers. Two reviewers blocked off time during the replication. The total time to complete tasks and the meta-analysis results were compared with the index 2weekSR study. Review process fidelity scores were calculated for review methods and outcomes. Barriers to completing the replication were identified.RESULTS: The review was completed over 63 person-hours (11 workdays/15 calendar days). A fidelity score (FS) of 0.95 was achieved for the Methods, with 3 (of 8) tasks only partially replicated, and an FS of 0.63 for the Outcomes, with 6 (of 7) only partially replicated and 1 task was not replicated. Non-replication was mainly caused by missing information in the index 2weekSR study that was not required in standard reporting guidelines. The replication arrived at the same conclusions as the original study.CONCLUSIONS: A 2weekSR study was replicated by a small team of 3 reviewers supported by automation tools. Including additional information when reporting SRs should improve their replicability.

AB - OBJECTIVE: To assess the replicability of a 2-week systematic review (index 2weekSR) created with the assistance of automation tools using the fidelity method.METHODS: A PRISMA-compliant SR protocol was developed based on the published information of the index 2weekSR study. The replication team consisted of three reviewers. Two reviewers blocked off time during the replication. The total time to complete tasks and the meta-analysis results were compared with the index 2weekSR study. Review process fidelity scores were calculated for review methods and outcomes. Barriers to completing the replication were identified.RESULTS: The review was completed over 63 person-hours (11 workdays/15 calendar days). A fidelity score (FS) of 0.95 was achieved for the Methods, with 3 (of 8) tasks only partially replicated, and an FS of 0.63 for the Outcomes, with 6 (of 7) only partially replicated and 1 task was not replicated. Non-replication was mainly caused by missing information in the index 2weekSR study that was not required in standard reporting guidelines. The replication arrived at the same conclusions as the original study.CONCLUSIONS: A 2weekSR study was replicated by a small team of 3 reviewers supported by automation tools. Including additional information when reporting SRs should improve their replicability.

U2 - 10.1016/j.jclinepi.2023.10.013

DO - 10.1016/j.jclinepi.2023.10.013

M3 - Article

C2 - 37879542

SN - 0895-4356

SP - 1

EP - 20

JO - Journal of Clinical Epidemiology

JF - Journal of Clinical Epidemiology

ER -

The two-week systematic review (2weekSR) method was successfully blind-replicated by another team: a case study

Abstract

Access to Document

Fingerprint

Cite this