OBJECTIVE: To assess the replicability of a 2-week systematic review (index 2weekSR) created with the assistance of automation tools using the fidelity method.
METHODS: A PRISMA-compliant SR protocol was developed based on the published information of the index 2weekSR study. The replication team consisted of three reviewers. Two reviewers blocked off time during the replication. The total time to complete tasks and the meta-analysis results were compared with the index 2weekSR study. Review process fidelity scores were calculated for review methods and outcomes. Barriers to completing the replication were identified.
RESULTS: The review was completed over 63 person-hours (11 workdays/15 calendar days). A fidelity score (FS) of 0.95 was achieved for the Methods, with 3 (of 8) tasks only partially replicated, and an FS of 0.63 for the Outcomes, with 6 (of 7) only partially replicated and 1 task was not replicated. Non-replication was mainly caused by missing information in the index 2weekSR study that was not required in standard reporting guidelines. The replication arrived at the same conclusions as the original study.
CONCLUSIONS: A 2weekSR study was replicated by a small team of 3 reviewers supported by automation tools. Including additional information when reporting SRs should improve their replicability.