Designing studies to ensure that estimates of test accuracy will travel

Les M. Irwig, Patrick M M Bossuyt, Paul P. Glasziou, Constantine Gatsonis, Jeroen G. Lijmer

Research output: Chapter in Book/Report/Conference proceedingChapterResearchpeer-review

5 Citations (Scopus)

Abstract

There may be genuine differences between test accuracies in different settings, such as primary care or hospital, in different types of hospital, or between countries. Deciding whether estimates of test accuracy are transferable to other settings depends on an understanding of the possible reasons for variability in test discrimination and calibration across settings. The transferability of measures of test performance from one setting to another depends on which indicator of test performance is to be used. Real variation in the performance of diagnostic tests (such as different test types, or a different spectrum of disease) needs to be distinguished from artifactual variation resulting from study design features. These features include the target condition and reference standard used, the population and the clinical question studied, the evaluated comparison, and the way the index test was performed, calibrated, and interpreted. In preparing studies on diagnostic accuracy, a key question is how to design studies that carry more information about the transferability of results. To ensure that estimates of diagnostic accuracy will travel, before starting to design a study the following questions must be answered: How are the target condition and reference standard defined? - Is the objective to estimate global test performance or to estimate probability of disease in individuals? - What is the population and clinical problem? - Is the test being considered as a replacement or incremental test? - To what extent do you want to study the reasons for variability of the results within your population? - To what extent do you want to study the transferability of the results to other settings? Designing studies with heterogeneous study populations allows exploration of the transferability of diagnostic performance in different settings. This will require larger studies than have generally been carried out in the past for diagnostic tests.

Original languageEnglish
Title of host publicationThe Evidence Base of Clinical Diagnosis: Theory and methods of diagnostic research: Second Edition
PublisherWiley-Blackwell
Pages96-117
Number of pages22
ISBN (Print)9781405157872
DOIs
Publication statusPublished - 2 Feb 2009
Externally publishedYes

Fingerprint

Routine Diagnostic Tests
Population
Calibration
Primary Health Care

Cite this

Irwig, L. M., Bossuyt, P. M. M., Glasziou, P. P., Gatsonis, C., & Lijmer, J. G. (2009). Designing studies to ensure that estimates of test accuracy will travel. In The Evidence Base of Clinical Diagnosis: Theory and methods of diagnostic research: Second Edition (pp. 96-117). Wiley-Blackwell. https://doi.org/10.1002/9781444300574.ch6
Irwig, Les M. ; Bossuyt, Patrick M M ; Glasziou, Paul P. ; Gatsonis, Constantine ; Lijmer, Jeroen G. / Designing studies to ensure that estimates of test accuracy will travel. The Evidence Base of Clinical Diagnosis: Theory and methods of diagnostic research: Second Edition. Wiley-Blackwell, 2009. pp. 96-117
@inbook{e9da6d76ed014c6e9e92f2a808d94811,
title = "Designing studies to ensure that estimates of test accuracy will travel",
abstract = "There may be genuine differences between test accuracies in different settings, such as primary care or hospital, in different types of hospital, or between countries. Deciding whether estimates of test accuracy are transferable to other settings depends on an understanding of the possible reasons for variability in test discrimination and calibration across settings. The transferability of measures of test performance from one setting to another depends on which indicator of test performance is to be used. Real variation in the performance of diagnostic tests (such as different test types, or a different spectrum of disease) needs to be distinguished from artifactual variation resulting from study design features. These features include the target condition and reference standard used, the population and the clinical question studied, the evaluated comparison, and the way the index test was performed, calibrated, and interpreted. In preparing studies on diagnostic accuracy, a key question is how to design studies that carry more information about the transferability of results. To ensure that estimates of diagnostic accuracy will travel, before starting to design a study the following questions must be answered: How are the target condition and reference standard defined? - Is the objective to estimate global test performance or to estimate probability of disease in individuals? - What is the population and clinical problem? - Is the test being considered as a replacement or incremental test? - To what extent do you want to study the reasons for variability of the results within your population? - To what extent do you want to study the transferability of the results to other settings? Designing studies with heterogeneous study populations allows exploration of the transferability of diagnostic performance in different settings. This will require larger studies than have generally been carried out in the past for diagnostic tests.",
author = "Irwig, {Les M.} and Bossuyt, {Patrick M M} and Glasziou, {Paul P.} and Constantine Gatsonis and Lijmer, {Jeroen G.}",
year = "2009",
month = "2",
day = "2",
doi = "10.1002/9781444300574.ch6",
language = "English",
isbn = "9781405157872",
pages = "96--117",
booktitle = "The Evidence Base of Clinical Diagnosis: Theory and methods of diagnostic research: Second Edition",
publisher = "Wiley-Blackwell",
address = "United Kingdom",

}

Irwig, LM, Bossuyt, PMM, Glasziou, PP, Gatsonis, C & Lijmer, JG 2009, Designing studies to ensure that estimates of test accuracy will travel. in The Evidence Base of Clinical Diagnosis: Theory and methods of diagnostic research: Second Edition. Wiley-Blackwell, pp. 96-117. https://doi.org/10.1002/9781444300574.ch6

Designing studies to ensure that estimates of test accuracy will travel. / Irwig, Les M.; Bossuyt, Patrick M M; Glasziou, Paul P.; Gatsonis, Constantine; Lijmer, Jeroen G.

The Evidence Base of Clinical Diagnosis: Theory and methods of diagnostic research: Second Edition. Wiley-Blackwell, 2009. p. 96-117.

Research output: Chapter in Book/Report/Conference proceedingChapterResearchpeer-review

TY - CHAP

T1 - Designing studies to ensure that estimates of test accuracy will travel

AU - Irwig, Les M.

AU - Bossuyt, Patrick M M

AU - Glasziou, Paul P.

AU - Gatsonis, Constantine

AU - Lijmer, Jeroen G.

PY - 2009/2/2

Y1 - 2009/2/2

N2 - There may be genuine differences between test accuracies in different settings, such as primary care or hospital, in different types of hospital, or between countries. Deciding whether estimates of test accuracy are transferable to other settings depends on an understanding of the possible reasons for variability in test discrimination and calibration across settings. The transferability of measures of test performance from one setting to another depends on which indicator of test performance is to be used. Real variation in the performance of diagnostic tests (such as different test types, or a different spectrum of disease) needs to be distinguished from artifactual variation resulting from study design features. These features include the target condition and reference standard used, the population and the clinical question studied, the evaluated comparison, and the way the index test was performed, calibrated, and interpreted. In preparing studies on diagnostic accuracy, a key question is how to design studies that carry more information about the transferability of results. To ensure that estimates of diagnostic accuracy will travel, before starting to design a study the following questions must be answered: How are the target condition and reference standard defined? - Is the objective to estimate global test performance or to estimate probability of disease in individuals? - What is the population and clinical problem? - Is the test being considered as a replacement or incremental test? - To what extent do you want to study the reasons for variability of the results within your population? - To what extent do you want to study the transferability of the results to other settings? Designing studies with heterogeneous study populations allows exploration of the transferability of diagnostic performance in different settings. This will require larger studies than have generally been carried out in the past for diagnostic tests.

AB - There may be genuine differences between test accuracies in different settings, such as primary care or hospital, in different types of hospital, or between countries. Deciding whether estimates of test accuracy are transferable to other settings depends on an understanding of the possible reasons for variability in test discrimination and calibration across settings. The transferability of measures of test performance from one setting to another depends on which indicator of test performance is to be used. Real variation in the performance of diagnostic tests (such as different test types, or a different spectrum of disease) needs to be distinguished from artifactual variation resulting from study design features. These features include the target condition and reference standard used, the population and the clinical question studied, the evaluated comparison, and the way the index test was performed, calibrated, and interpreted. In preparing studies on diagnostic accuracy, a key question is how to design studies that carry more information about the transferability of results. To ensure that estimates of diagnostic accuracy will travel, before starting to design a study the following questions must be answered: How are the target condition and reference standard defined? - Is the objective to estimate global test performance or to estimate probability of disease in individuals? - What is the population and clinical problem? - Is the test being considered as a replacement or incremental test? - To what extent do you want to study the reasons for variability of the results within your population? - To what extent do you want to study the transferability of the results to other settings? Designing studies with heterogeneous study populations allows exploration of the transferability of diagnostic performance in different settings. This will require larger studies than have generally been carried out in the past for diagnostic tests.

UR - http://www.scopus.com/inward/record.url?scp=84889455194&partnerID=8YFLogxK

U2 - 10.1002/9781444300574.ch6

DO - 10.1002/9781444300574.ch6

M3 - Chapter

SN - 9781405157872

SP - 96

EP - 117

BT - The Evidence Base of Clinical Diagnosis: Theory and methods of diagnostic research: Second Edition

PB - Wiley-Blackwell

ER -

Irwig LM, Bossuyt PMM, Glasziou PP, Gatsonis C, Lijmer JG. Designing studies to ensure that estimates of test accuracy will travel. In The Evidence Base of Clinical Diagnosis: Theory and methods of diagnostic research: Second Edition. Wiley-Blackwell. 2009. p. 96-117 https://doi.org/10.1002/9781444300574.ch6