Did a change in Nature journals' editorial policy for life sciences research improve reporting?

The NPQIP Collaborative group

Research output: Contribution to journalArticleResearchpeer-review

31 Downloads (Pure)

Abstract

Objective To determine whether a change in editorial policy, including the implementation of a checklist, has been associated with improved reporting of measures which might reduce the risk of bias.

Methods The study protocol has been published at doi: 10.1007/s11192-016-1964-8.

Design Observational cohort study.

Population Articles describing research in the life sciences published in Nature journals, submitted after 1 May 2013.

Intervention Mandatory completion of a checklist during manuscript revision.

Comparators (1) Articles describing research in the life sciences published in Nature journals, submitted before May 2013; and (2) similar articles in other journals matched for date and topic.

Primary outcome The primary outcome is change in the proportion of Nature articles describing in vivo research published before and after May 2013 reporting the ‘Landis 4’ items (randomisation, blinding, sample size calculation and exclusions). We included 448 Nature Publishing Group (NPG) articles (223 published before May 2013, and 225 after) identified by an individual hired by NPG for this specific task, working to a standard procedure; and an independent investigator used PubMed ‘Related Citations’ to identify 448 non-NPG articles with a similar topic and date of publication from other journals; and then redacted all articles for time-sensitive information and journal name. Redacted articles were assessed by two trained reviewers against a 74-item checklist, with discrepancies resolved by a third.

Results 394 NPG and 353 matching non-NPG articles described in vivo research. The number of NPG articles meeting all relevant Landis 4 criteria increased from 0/203 prior to May 2013 to 31/181 (16.4%) after (two-sample test for equality of proportions without continuity correction, Χ²=36.2, df=1, p=1.8×10−9). There was no change in the proportion of non-NPG articles meeting all relevant Landis 4 criteria (1/164 before, 1/189 after). There were more substantial improvements in the individual prevalences of reporting of randomisation, blinding, exclusions and sample size calculations for in vivo experiments, and less substantial improvements for in vitro experiments.

Conclusion There was an improvement in the reporting of risks of bias in in vivo research in NPG journals following a change in editorial policy, to a level that to our knowledge has not been previously observed. However, there remain opportunities for further improvement.

This is an open access article distributed in accordance with the Creative Commons Attribution 4.0 Unported (CC BY 4.0) license, which permits others to copy, redistribute, remix, transform and build upon this work for any purpose, provided the original work is properly cited, a link to the licence is given, and indication of whether changes were made. See: https://creativecommons.org/licenses/by/4.0/.
Original languageEnglish
Number of pages25
JournalBMJ Open Science
Volume3
Issue number1
DOIs
Publication statusPublished - 2019

Fingerprint

Editorial Policies
Biological Science Disciplines
Licensure
Checklist
Research
Group Processes
Random Allocation
Sample Size
Manuscripts
PubMed
Names
Observational Studies
Publications
Cohort Studies
Research Personnel

Cite this

@article{5d2c8b0c8eab46beafd19d2402e72b24,
title = "Did a change in Nature journals' editorial policy for life sciences research improve reporting?",
abstract = "Objective To determine whether a change in editorial policy, including the implementation of a checklist, has been associated with improved reporting of measures which might reduce the risk of bias.Methods The study protocol has been published at doi: 10.1007/s11192-016-1964-8.Design Observational cohort study.Population Articles describing research in the life sciences published in Nature journals, submitted after 1 May 2013.Intervention Mandatory completion of a checklist during manuscript revision.Comparators (1) Articles describing research in the life sciences published in Nature journals, submitted before May 2013; and (2) similar articles in other journals matched for date and topic.Primary outcome The primary outcome is change in the proportion of Nature articles describing in vivo research published before and after May 2013 reporting the ‘Landis 4’ items (randomisation, blinding, sample size calculation and exclusions). We included 448 Nature Publishing Group (NPG) articles (223 published before May 2013, and 225 after) identified by an individual hired by NPG for this specific task, working to a standard procedure; and an independent investigator used PubMed ‘Related Citations’ to identify 448 non-NPG articles with a similar topic and date of publication from other journals; and then redacted all articles for time-sensitive information and journal name. Redacted articles were assessed by two trained reviewers against a 74-item checklist, with discrepancies resolved by a third.Results 394 NPG and 353 matching non-NPG articles described in vivo research. The number of NPG articles meeting all relevant Landis 4 criteria increased from 0/203 prior to May 2013 to 31/181 (16.4{\%}) after (two-sample test for equality of proportions without continuity correction, Χ²=36.2, df=1, p=1.8×10−9). There was no change in the proportion of non-NPG articles meeting all relevant Landis 4 criteria (1/164 before, 1/189 after). There were more substantial improvements in the individual prevalences of reporting of randomisation, blinding, exclusions and sample size calculations for in vivo experiments, and less substantial improvements for in vitro experiments.Conclusion There was an improvement in the reporting of risks of bias in in vivo research in NPG journals following a change in editorial policy, to a level that to our knowledge has not been previously observed. However, there remain opportunities for further improvement.This is an open access article distributed in accordance with the Creative Commons Attribution 4.0 Unported (CC BY 4.0) license, which permits others to copy, redistribute, remix, transform and build upon this work for any purpose, provided the original work is properly cited, a link to the licence is given, and indication of whether changes were made. See: https://creativecommons.org/licenses/by/4.0/.",
author = "{The NPQIP Collaborative group} and Emma Eaton and Privjyot Jheeta and Alice Carter and Marco Casscella and Mclean, {Aaron Lawson} and Emily Wheater and Sarah Mccann and Kyng, {Kasper Jacobsen} and Peter-paul Zwetsloot and Alexandra Bannach-brown and De-souza, {Evandro Ara{\'u}jo} and Chris Choi and Ye Liu and Thomas Barrett and Gerlei, {Klara Zsofia} and Xenios Milidonis and Thomas Ottavi and Ana Antonic and Mahajabeen Khatib and Kimberley Wever and Crawford, {Devon C} and Marie Soukupova and Kaitlyn Hair and Roncon Paolo and Hanna Vesterinen and Andrew Ying and Sulail Rajani and Fala Cramond and Daniel-cosmin Marcu and oliveira, {Cilene Lino De} and Cadi Irvine and Malcolm Macleod and Monica Dingwall and Paula Grill and Chris Sena and Jing Liao and Malcolm Macleod and Gillian Currie and Emily Sena and Monica Dingwall and Paula Grill and Cadi Irvine and Rosie Moreland and Hugh Ash and Sowmya Swaminathan and Veronique Kiermer and David Howells and Emily Sena and Malcolm Macleod and David Howells and Emily Sena and Malcolm Macleod",
year = "2019",
doi = "10.1136/bmjos-2017-000035",
language = "English",
volume = "3",
journal = "BMJ Open Science",
issn = "2398-8703",
publisher = "BMJ Publishing Group",
number = "1",

}

Did a change in Nature journals' editorial policy for life sciences research improve reporting? / The NPQIP Collaborative group.

In: BMJ Open Science, Vol. 3, No. 1, 2019.

Research output: Contribution to journalArticleResearchpeer-review

TY - JOUR

T1 - Did a change in Nature journals' editorial policy for life sciences research improve reporting?

AU - The NPQIP Collaborative group

AU - Eaton, Emma

AU - Jheeta, Privjyot

AU - Carter, Alice

AU - Casscella, Marco

AU - Mclean, Aaron Lawson

AU - Wheater, Emily

AU - Mccann, Sarah

AU - Kyng, Kasper Jacobsen

AU - Zwetsloot, Peter-paul

AU - Bannach-brown, Alexandra

AU - De-souza, Evandro Araújo

AU - Choi, Chris

AU - Liu, Ye

AU - Barrett, Thomas

AU - Gerlei, Klara Zsofia

AU - Milidonis, Xenios

AU - Ottavi, Thomas

AU - Antonic, Ana

AU - Khatib, Mahajabeen

AU - Wever, Kimberley

AU - Crawford, Devon C

AU - Soukupova, Marie

AU - Hair, Kaitlyn

AU - Paolo, Roncon

AU - Vesterinen, Hanna

AU - Ying, Andrew

AU - Rajani, Sulail

AU - Cramond, Fala

AU - Marcu, Daniel-cosmin

AU - oliveira, Cilene Lino De

AU - Irvine, Cadi

AU - Macleod, Malcolm

AU - Dingwall, Monica

AU - Grill, Paula

AU - Sena, Chris

AU - Liao, Jing

AU - Macleod, Malcolm

AU - Currie, Gillian

AU - Sena, Emily

AU - Dingwall, Monica

AU - Grill, Paula

AU - Irvine, Cadi

AU - Moreland, Rosie

AU - Ash, Hugh

AU - Swaminathan, Sowmya

AU - Kiermer, Veronique

AU - Howells, David

AU - Sena, Emily

AU - Macleod, Malcolm

AU - Howells, David

AU - Sena, Emily

AU - Macleod, Malcolm

PY - 2019

Y1 - 2019

N2 - Objective To determine whether a change in editorial policy, including the implementation of a checklist, has been associated with improved reporting of measures which might reduce the risk of bias.Methods The study protocol has been published at doi: 10.1007/s11192-016-1964-8.Design Observational cohort study.Population Articles describing research in the life sciences published in Nature journals, submitted after 1 May 2013.Intervention Mandatory completion of a checklist during manuscript revision.Comparators (1) Articles describing research in the life sciences published in Nature journals, submitted before May 2013; and (2) similar articles in other journals matched for date and topic.Primary outcome The primary outcome is change in the proportion of Nature articles describing in vivo research published before and after May 2013 reporting the ‘Landis 4’ items (randomisation, blinding, sample size calculation and exclusions). We included 448 Nature Publishing Group (NPG) articles (223 published before May 2013, and 225 after) identified by an individual hired by NPG for this specific task, working to a standard procedure; and an independent investigator used PubMed ‘Related Citations’ to identify 448 non-NPG articles with a similar topic and date of publication from other journals; and then redacted all articles for time-sensitive information and journal name. Redacted articles were assessed by two trained reviewers against a 74-item checklist, with discrepancies resolved by a third.Results 394 NPG and 353 matching non-NPG articles described in vivo research. The number of NPG articles meeting all relevant Landis 4 criteria increased from 0/203 prior to May 2013 to 31/181 (16.4%) after (two-sample test for equality of proportions without continuity correction, Χ²=36.2, df=1, p=1.8×10−9). There was no change in the proportion of non-NPG articles meeting all relevant Landis 4 criteria (1/164 before, 1/189 after). There were more substantial improvements in the individual prevalences of reporting of randomisation, blinding, exclusions and sample size calculations for in vivo experiments, and less substantial improvements for in vitro experiments.Conclusion There was an improvement in the reporting of risks of bias in in vivo research in NPG journals following a change in editorial policy, to a level that to our knowledge has not been previously observed. However, there remain opportunities for further improvement.This is an open access article distributed in accordance with the Creative Commons Attribution 4.0 Unported (CC BY 4.0) license, which permits others to copy, redistribute, remix, transform and build upon this work for any purpose, provided the original work is properly cited, a link to the licence is given, and indication of whether changes were made. See: https://creativecommons.org/licenses/by/4.0/.

AB - Objective To determine whether a change in editorial policy, including the implementation of a checklist, has been associated with improved reporting of measures which might reduce the risk of bias.Methods The study protocol has been published at doi: 10.1007/s11192-016-1964-8.Design Observational cohort study.Population Articles describing research in the life sciences published in Nature journals, submitted after 1 May 2013.Intervention Mandatory completion of a checklist during manuscript revision.Comparators (1) Articles describing research in the life sciences published in Nature journals, submitted before May 2013; and (2) similar articles in other journals matched for date and topic.Primary outcome The primary outcome is change in the proportion of Nature articles describing in vivo research published before and after May 2013 reporting the ‘Landis 4’ items (randomisation, blinding, sample size calculation and exclusions). We included 448 Nature Publishing Group (NPG) articles (223 published before May 2013, and 225 after) identified by an individual hired by NPG for this specific task, working to a standard procedure; and an independent investigator used PubMed ‘Related Citations’ to identify 448 non-NPG articles with a similar topic and date of publication from other journals; and then redacted all articles for time-sensitive information and journal name. Redacted articles were assessed by two trained reviewers against a 74-item checklist, with discrepancies resolved by a third.Results 394 NPG and 353 matching non-NPG articles described in vivo research. The number of NPG articles meeting all relevant Landis 4 criteria increased from 0/203 prior to May 2013 to 31/181 (16.4%) after (two-sample test for equality of proportions without continuity correction, Χ²=36.2, df=1, p=1.8×10−9). There was no change in the proportion of non-NPG articles meeting all relevant Landis 4 criteria (1/164 before, 1/189 after). There were more substantial improvements in the individual prevalences of reporting of randomisation, blinding, exclusions and sample size calculations for in vivo experiments, and less substantial improvements for in vitro experiments.Conclusion There was an improvement in the reporting of risks of bias in in vivo research in NPG journals following a change in editorial policy, to a level that to our knowledge has not been previously observed. However, there remain opportunities for further improvement.This is an open access article distributed in accordance with the Creative Commons Attribution 4.0 Unported (CC BY 4.0) license, which permits others to copy, redistribute, remix, transform and build upon this work for any purpose, provided the original work is properly cited, a link to the licence is given, and indication of whether changes were made. See: https://creativecommons.org/licenses/by/4.0/.

U2 - 10.1136/bmjos-2017-000035

DO - 10.1136/bmjos-2017-000035

M3 - Article

VL - 3

JO - BMJ Open Science

JF - BMJ Open Science

SN - 2398-8703

IS - 1

ER -