Simplification of a scoring system maintained overall accuracy but decreased the proportion classified as low risk

Sharon Sanders, Dylan Flaws, Martin Than, John W. Pickering, Jenny Doust, Paul Glasziou

Research output: Contribution to journalArticleResearchpeer-review

6 Citations (Scopus)

Abstract

Objectives Scoring systems are developed to assist clinicians in making a diagnosis. However, their uptake is often limited because they are cumbersome to use, requiring information on many predictors, or complicated calculations. We examined whether, and how, simplifications affected the performance of a validated score for identifying adults with chest pain in an emergency department who have low risk of major adverse cardiac events. Study Design and Setting We simplified the Emergency Department Assessment of Chest pain Score (EDACS) by three methods: (1) giving equal weight to each predictor included in the score, (2) reducing the number of predictors, and (3) using both methods - giving equal weight to a reduced number of predictors. The diagnostic accuracy of the simplified scores was compared with the original score in the derivation (n = 1,974) and validation (n = 909) data sets. Results There was no difference in the overall accuracy of the simplified versions of the score compared with the original EDACS as measured by the area under the receiver operating characteristic curve (0.74 to 0.75 for simplified versions vs. 0.75 for the original score in the validation cohort). With score cut-offs set to maintain the sensitivity of the combination of score and tests (electrocardiogram and cardiac troponin) at a level acceptable to clinicians (99%), simplification reduced the proportion of patients classified as low risk from 50% with the original score to between 22% and 42%. Conclusion Simplification of a clinical score resulted in similar overall accuracy but reduced the proportion classified as low risk and therefore eligible for early discharge compared with the original score. Whether the trade-off is acceptable, will depend on the context in which the score is to be used. Developers of clinical scores should consider simplification as a method to increase uptake, but further studies are needed to determine the best methods of deriving and evaluating simplified scores.

Original languageEnglish
Pages (from-to)32-39
Number of pages8
JournalJournal of Clinical Epidemiology
Volume69
DOIs
Publication statusPublished - 1 Jan 2016

Fingerprint

Chest Pain
Hospital Emergency Service
Weights and Measures
Troponin
ROC Curve
Electrocardiography

Cite this

@article{73bddf9a199b45dc92173dd1798c274c,
title = "Simplification of a scoring system maintained overall accuracy but decreased the proportion classified as low risk",
abstract = "Objectives Scoring systems are developed to assist clinicians in making a diagnosis. However, their uptake is often limited because they are cumbersome to use, requiring information on many predictors, or complicated calculations. We examined whether, and how, simplifications affected the performance of a validated score for identifying adults with chest pain in an emergency department who have low risk of major adverse cardiac events. Study Design and Setting We simplified the Emergency Department Assessment of Chest pain Score (EDACS) by three methods: (1) giving equal weight to each predictor included in the score, (2) reducing the number of predictors, and (3) using both methods - giving equal weight to a reduced number of predictors. The diagnostic accuracy of the simplified scores was compared with the original score in the derivation (n = 1,974) and validation (n = 909) data sets. Results There was no difference in the overall accuracy of the simplified versions of the score compared with the original EDACS as measured by the area under the receiver operating characteristic curve (0.74 to 0.75 for simplified versions vs. 0.75 for the original score in the validation cohort). With score cut-offs set to maintain the sensitivity of the combination of score and tests (electrocardiogram and cardiac troponin) at a level acceptable to clinicians (99{\%}), simplification reduced the proportion of patients classified as low risk from 50{\%} with the original score to between 22{\%} and 42{\%}. Conclusion Simplification of a clinical score resulted in similar overall accuracy but reduced the proportion classified as low risk and therefore eligible for early discharge compared with the original score. Whether the trade-off is acceptable, will depend on the context in which the score is to be used. Developers of clinical scores should consider simplification as a method to increase uptake, but further studies are needed to determine the best methods of deriving and evaluating simplified scores.",
author = "Sharon Sanders and Dylan Flaws and Martin Than and Pickering, {John W.} and Jenny Doust and Paul Glasziou",
year = "2016",
month = "1",
day = "1",
doi = "10.1016/j.jclinepi.2015.05.006",
language = "English",
volume = "69",
pages = "32--39",
journal = "Journal of Chronic Diseases",
issn = "0895-4356",
publisher = "Elsevier",

}

Simplification of a scoring system maintained overall accuracy but decreased the proportion classified as low risk. / Sanders, Sharon; Flaws, Dylan; Than, Martin; Pickering, John W.; Doust, Jenny; Glasziou, Paul.

In: Journal of Clinical Epidemiology, Vol. 69, 01.01.2016, p. 32-39.

Research output: Contribution to journalArticleResearchpeer-review

TY - JOUR

T1 - Simplification of a scoring system maintained overall accuracy but decreased the proportion classified as low risk

AU - Sanders, Sharon

AU - Flaws, Dylan

AU - Than, Martin

AU - Pickering, John W.

AU - Doust, Jenny

AU - Glasziou, Paul

PY - 2016/1/1

Y1 - 2016/1/1

N2 - Objectives Scoring systems are developed to assist clinicians in making a diagnosis. However, their uptake is often limited because they are cumbersome to use, requiring information on many predictors, or complicated calculations. We examined whether, and how, simplifications affected the performance of a validated score for identifying adults with chest pain in an emergency department who have low risk of major adverse cardiac events. Study Design and Setting We simplified the Emergency Department Assessment of Chest pain Score (EDACS) by three methods: (1) giving equal weight to each predictor included in the score, (2) reducing the number of predictors, and (3) using both methods - giving equal weight to a reduced number of predictors. The diagnostic accuracy of the simplified scores was compared with the original score in the derivation (n = 1,974) and validation (n = 909) data sets. Results There was no difference in the overall accuracy of the simplified versions of the score compared with the original EDACS as measured by the area under the receiver operating characteristic curve (0.74 to 0.75 for simplified versions vs. 0.75 for the original score in the validation cohort). With score cut-offs set to maintain the sensitivity of the combination of score and tests (electrocardiogram and cardiac troponin) at a level acceptable to clinicians (99%), simplification reduced the proportion of patients classified as low risk from 50% with the original score to between 22% and 42%. Conclusion Simplification of a clinical score resulted in similar overall accuracy but reduced the proportion classified as low risk and therefore eligible for early discharge compared with the original score. Whether the trade-off is acceptable, will depend on the context in which the score is to be used. Developers of clinical scores should consider simplification as a method to increase uptake, but further studies are needed to determine the best methods of deriving and evaluating simplified scores.

AB - Objectives Scoring systems are developed to assist clinicians in making a diagnosis. However, their uptake is often limited because they are cumbersome to use, requiring information on many predictors, or complicated calculations. We examined whether, and how, simplifications affected the performance of a validated score for identifying adults with chest pain in an emergency department who have low risk of major adverse cardiac events. Study Design and Setting We simplified the Emergency Department Assessment of Chest pain Score (EDACS) by three methods: (1) giving equal weight to each predictor included in the score, (2) reducing the number of predictors, and (3) using both methods - giving equal weight to a reduced number of predictors. The diagnostic accuracy of the simplified scores was compared with the original score in the derivation (n = 1,974) and validation (n = 909) data sets. Results There was no difference in the overall accuracy of the simplified versions of the score compared with the original EDACS as measured by the area under the receiver operating characteristic curve (0.74 to 0.75 for simplified versions vs. 0.75 for the original score in the validation cohort). With score cut-offs set to maintain the sensitivity of the combination of score and tests (electrocardiogram and cardiac troponin) at a level acceptable to clinicians (99%), simplification reduced the proportion of patients classified as low risk from 50% with the original score to between 22% and 42%. Conclusion Simplification of a clinical score resulted in similar overall accuracy but reduced the proportion classified as low risk and therefore eligible for early discharge compared with the original score. Whether the trade-off is acceptable, will depend on the context in which the score is to be used. Developers of clinical scores should consider simplification as a method to increase uptake, but further studies are needed to determine the best methods of deriving and evaluating simplified scores.

UR - http://www.scopus.com/inward/record.url?scp=84952628350&partnerID=8YFLogxK

U2 - 10.1016/j.jclinepi.2015.05.006

DO - 10.1016/j.jclinepi.2015.05.006

M3 - Article

VL - 69

SP - 32

EP - 39

JO - Journal of Chronic Diseases

JF - Journal of Chronic Diseases

SN - 0895-4356

ER -