The practical value of the standard error of measurement in borderline pass/fail decisions

Richard Hays, Tarun Sen Gupta, Jan Vietch

Research output: Contribution to journalArticleResearchpeer-review

22 Citations (Scopus)

Abstract

Objective: The purpose of this study was to explore the value of the standard error of measurement (SEM) in making decisions about students with examination scores at or below the pass/fail borderline in a new undergraduate medical course with an integrated assessment programme. Methods: An analysis of de-identified, pooled data for borderline candidates was conducted to determine the SEM for each examination and the progress of candidates according to four score bands, from pass score ± 1 SEM, 1-2 SEM below the pass score, 2-3 SEM below the pass score and > 3 SEM below the pass score. The impact of poor performance in individual subject areas was also measured. Results: Data for 1571 candidates were included in the analysis, identifying 132 students with borderline or lower scores, 45% of which were > 1 SEM below the pass score. By the third cohort the banding of students according to the SEM became highly predictive of candidate progress either through immediate remediation and re-sit examination, or by repetition of the year or withdrawal from the course. Conclusions: The SEM is a useful tool for making confident and defensible decisions about how to manage candidates with examination scores at or below the borderline mark, as long as attention is paid to established examination design principles. The improved defensibility can be used to support a patient-safety focused decision tree or similar decision support model. © Blackwell Publishing Ltd 2008.
Original languageEnglish
JournalMedical Education
Volume42
Issue number8
DOIs
Publication statusPublished - 2008
Externally publishedYes

Fingerprint

candidacy
examination
student
withdrawal
decision making
performance
Values

Cite this

Hays, Richard ; Sen Gupta, Tarun ; Vietch, Jan. / The practical value of the standard error of measurement in borderline pass/fail decisions. In: Medical Education. 2008 ; Vol. 42, No. 8.
@article{88b3c5ebcc8a46ce82f66b97a2a5fc3f,
title = "The practical value of the standard error of measurement in borderline pass/fail decisions",
abstract = "Objective: The purpose of this study was to explore the value of the standard error of measurement (SEM) in making decisions about students with examination scores at or below the pass/fail borderline in a new undergraduate medical course with an integrated assessment programme. Methods: An analysis of de-identified, pooled data for borderline candidates was conducted to determine the SEM for each examination and the progress of candidates according to four score bands, from pass score ± 1 SEM, 1-2 SEM below the pass score, 2-3 SEM below the pass score and > 3 SEM below the pass score. The impact of poor performance in individual subject areas was also measured. Results: Data for 1571 candidates were included in the analysis, identifying 132 students with borderline or lower scores, 45{\%} of which were > 1 SEM below the pass score. By the third cohort the banding of students according to the SEM became highly predictive of candidate progress either through immediate remediation and re-sit examination, or by repetition of the year or withdrawal from the course. Conclusions: The SEM is a useful tool for making confident and defensible decisions about how to manage candidates with examination scores at or below the borderline mark, as long as attention is paid to established examination design principles. The improved defensibility can be used to support a patient-safety focused decision tree or similar decision support model. {\circledC} Blackwell Publishing Ltd 2008.",
author = "Richard Hays and {Sen Gupta}, Tarun and Jan Vietch",
year = "2008",
doi = "10.1111/j.1365-2923.2008.03103.x",
language = "English",
volume = "42",
journal = "British journal of medical education",
issn = "0308-0110",
publisher = "Wiley Blackwell (American Society Bone & Mineral Research)",
number = "8",

}

The practical value of the standard error of measurement in borderline pass/fail decisions. / Hays, Richard; Sen Gupta, Tarun; Vietch, Jan.

In: Medical Education, Vol. 42, No. 8, 2008.

Research output: Contribution to journalArticleResearchpeer-review

TY - JOUR

T1 - The practical value of the standard error of measurement in borderline pass/fail decisions

AU - Hays, Richard

AU - Sen Gupta, Tarun

AU - Vietch, Jan

PY - 2008

Y1 - 2008

N2 - Objective: The purpose of this study was to explore the value of the standard error of measurement (SEM) in making decisions about students with examination scores at or below the pass/fail borderline in a new undergraduate medical course with an integrated assessment programme. Methods: An analysis of de-identified, pooled data for borderline candidates was conducted to determine the SEM for each examination and the progress of candidates according to four score bands, from pass score ± 1 SEM, 1-2 SEM below the pass score, 2-3 SEM below the pass score and > 3 SEM below the pass score. The impact of poor performance in individual subject areas was also measured. Results: Data for 1571 candidates were included in the analysis, identifying 132 students with borderline or lower scores, 45% of which were > 1 SEM below the pass score. By the third cohort the banding of students according to the SEM became highly predictive of candidate progress either through immediate remediation and re-sit examination, or by repetition of the year or withdrawal from the course. Conclusions: The SEM is a useful tool for making confident and defensible decisions about how to manage candidates with examination scores at or below the borderline mark, as long as attention is paid to established examination design principles. The improved defensibility can be used to support a patient-safety focused decision tree or similar decision support model. © Blackwell Publishing Ltd 2008.

AB - Objective: The purpose of this study was to explore the value of the standard error of measurement (SEM) in making decisions about students with examination scores at or below the pass/fail borderline in a new undergraduate medical course with an integrated assessment programme. Methods: An analysis of de-identified, pooled data for borderline candidates was conducted to determine the SEM for each examination and the progress of candidates according to four score bands, from pass score ± 1 SEM, 1-2 SEM below the pass score, 2-3 SEM below the pass score and > 3 SEM below the pass score. The impact of poor performance in individual subject areas was also measured. Results: Data for 1571 candidates were included in the analysis, identifying 132 students with borderline or lower scores, 45% of which were > 1 SEM below the pass score. By the third cohort the banding of students according to the SEM became highly predictive of candidate progress either through immediate remediation and re-sit examination, or by repetition of the year or withdrawal from the course. Conclusions: The SEM is a useful tool for making confident and defensible decisions about how to manage candidates with examination scores at or below the borderline mark, as long as attention is paid to established examination design principles. The improved defensibility can be used to support a patient-safety focused decision tree or similar decision support model. © Blackwell Publishing Ltd 2008.

U2 - 10.1111/j.1365-2923.2008.03103.x

DO - 10.1111/j.1365-2923.2008.03103.x

M3 - Article

VL - 42

JO - British journal of medical education

JF - British journal of medical education

SN - 0308-0110

IS - 8

ER -