Structured assessment using multiple patient scenarios by videoconference in rural settings

Tim J. Wilkinson, Janie D. Smith, Stephen A. Margolis, Tarun Sen Gupta, David J. Prideaux

Research output: Contribution to journalArticleResearchpeer-review

6 Citations (Scopus)

Abstract

CONTEXT The assessment blueprint of the Australian College of Rural and Remote Medicine postgraduate curriculum highlighted a need to assess clinical reasoning. We describe the development, reliability, feasibility, validity and educational impact of an 8-station assessment tool, StAMPS (structured assessment using multiple patient scenarios), conducted by videoconference.

METHODS StAMPS asks each candidate to be examined at each of 8 stations on issues relating to patient diagnosis or management. Each candidate remains located in a rural site but is examined in turn by 8 examiners who are located at a central site. Examiners were rotated through the candidates by either walking between videoconference rooms or by connecting and disconnecting the links. Reliability was evaluated using generalisability theory. Validity and educational impact were evaluated with qualitative interviews.

RESULTS Fourteen candidates were assessed on 82 scenarios with a reliability of G = 0.76. There was a reasonable correlation with level of candidate expertise (rho = 0.57). The videoconference links were acceptable to candidates and examiners but the walking rotation system was more reliable. Qualitative comments confirmed relevance and acceptability of the assessment tool and suggest it is likely to have a desirable educational impact.

CONCLUSIONS StAMPS not only reflects the content of rural and remote practice but also reflects the process of that work in that it is delivered from a distance and assesses resourcefulness and flexibility in thinking. The reliability and feasibility of this type of assessment has implications for people running any distance-based course, but the assessment could also be used in a face-to-face setting.

Original languageEnglish
Pages (from-to)480-487
Number of pages8
JournalMedical Education
Volume42
Issue number5
DOIs
Publication statusPublished - May 2008
Externally publishedYes

Cite this

Wilkinson, Tim J. ; Smith, Janie D. ; Margolis, Stephen A. ; Sen Gupta, Tarun ; Prideaux, David J. / Structured assessment using multiple patient scenarios by videoconference in rural settings. In: Medical Education. 2008 ; Vol. 42, No. 5. pp. 480-487.
@article{c58d5f6bf82a45cc90b5e9e70312b709,
title = "Structured assessment using multiple patient scenarios by videoconference in rural settings",
abstract = "CONTEXT The assessment blueprint of the Australian College of Rural and Remote Medicine postgraduate curriculum highlighted a need to assess clinical reasoning. We describe the development, reliability, feasibility, validity and educational impact of an 8-station assessment tool, StAMPS (structured assessment using multiple patient scenarios), conducted by videoconference.METHODS StAMPS asks each candidate to be examined at each of 8 stations on issues relating to patient diagnosis or management. Each candidate remains located in a rural site but is examined in turn by 8 examiners who are located at a central site. Examiners were rotated through the candidates by either walking between videoconference rooms or by connecting and disconnecting the links. Reliability was evaluated using generalisability theory. Validity and educational impact were evaluated with qualitative interviews.RESULTS Fourteen candidates were assessed on 82 scenarios with a reliability of G = 0.76. There was a reasonable correlation with level of candidate expertise (rho = 0.57). The videoconference links were acceptable to candidates and examiners but the walking rotation system was more reliable. Qualitative comments confirmed relevance and acceptability of the assessment tool and suggest it is likely to have a desirable educational impact.CONCLUSIONS StAMPS not only reflects the content of rural and remote practice but also reflects the process of that work in that it is delivered from a distance and assesses resourcefulness and flexibility in thinking. The reliability and feasibility of this type of assessment has implications for people running any distance-based course, but the assessment could also be used in a face-to-face setting.",
author = "Wilkinson, {Tim J.} and Smith, {Janie D.} and Margolis, {Stephen A.} and {Sen Gupta}, Tarun and Prideaux, {David J.}",
year = "2008",
month = "5",
doi = "10.1111/j.1365-2923.2008.03011.x",
language = "English",
volume = "42",
pages = "480--487",
journal = "British journal of medical education",
issn = "0308-0110",
publisher = "Wiley Blackwell (American Society Bone & Mineral Research)",
number = "5",

}

Structured assessment using multiple patient scenarios by videoconference in rural settings. / Wilkinson, Tim J.; Smith, Janie D.; Margolis, Stephen A.; Sen Gupta, Tarun; Prideaux, David J.

In: Medical Education, Vol. 42, No. 5, 05.2008, p. 480-487.

Research output: Contribution to journalArticleResearchpeer-review

TY - JOUR

T1 - Structured assessment using multiple patient scenarios by videoconference in rural settings

AU - Wilkinson, Tim J.

AU - Smith, Janie D.

AU - Margolis, Stephen A.

AU - Sen Gupta, Tarun

AU - Prideaux, David J.

PY - 2008/5

Y1 - 2008/5

N2 - CONTEXT The assessment blueprint of the Australian College of Rural and Remote Medicine postgraduate curriculum highlighted a need to assess clinical reasoning. We describe the development, reliability, feasibility, validity and educational impact of an 8-station assessment tool, StAMPS (structured assessment using multiple patient scenarios), conducted by videoconference.METHODS StAMPS asks each candidate to be examined at each of 8 stations on issues relating to patient diagnosis or management. Each candidate remains located in a rural site but is examined in turn by 8 examiners who are located at a central site. Examiners were rotated through the candidates by either walking between videoconference rooms or by connecting and disconnecting the links. Reliability was evaluated using generalisability theory. Validity and educational impact were evaluated with qualitative interviews.RESULTS Fourteen candidates were assessed on 82 scenarios with a reliability of G = 0.76. There was a reasonable correlation with level of candidate expertise (rho = 0.57). The videoconference links were acceptable to candidates and examiners but the walking rotation system was more reliable. Qualitative comments confirmed relevance and acceptability of the assessment tool and suggest it is likely to have a desirable educational impact.CONCLUSIONS StAMPS not only reflects the content of rural and remote practice but also reflects the process of that work in that it is delivered from a distance and assesses resourcefulness and flexibility in thinking. The reliability and feasibility of this type of assessment has implications for people running any distance-based course, but the assessment could also be used in a face-to-face setting.

AB - CONTEXT The assessment blueprint of the Australian College of Rural and Remote Medicine postgraduate curriculum highlighted a need to assess clinical reasoning. We describe the development, reliability, feasibility, validity and educational impact of an 8-station assessment tool, StAMPS (structured assessment using multiple patient scenarios), conducted by videoconference.METHODS StAMPS asks each candidate to be examined at each of 8 stations on issues relating to patient diagnosis or management. Each candidate remains located in a rural site but is examined in turn by 8 examiners who are located at a central site. Examiners were rotated through the candidates by either walking between videoconference rooms or by connecting and disconnecting the links. Reliability was evaluated using generalisability theory. Validity and educational impact were evaluated with qualitative interviews.RESULTS Fourteen candidates were assessed on 82 scenarios with a reliability of G = 0.76. There was a reasonable correlation with level of candidate expertise (rho = 0.57). The videoconference links were acceptable to candidates and examiners but the walking rotation system was more reliable. Qualitative comments confirmed relevance and acceptability of the assessment tool and suggest it is likely to have a desirable educational impact.CONCLUSIONS StAMPS not only reflects the content of rural and remote practice but also reflects the process of that work in that it is delivered from a distance and assesses resourcefulness and flexibility in thinking. The reliability and feasibility of this type of assessment has implications for people running any distance-based course, but the assessment could also be used in a face-to-face setting.

U2 - 10.1111/j.1365-2923.2008.03011.x

DO - 10.1111/j.1365-2923.2008.03011.x

M3 - Article

VL - 42

SP - 480

EP - 487

JO - British journal of medical education

JF - British journal of medical education

SN - 0308-0110

IS - 5

ER -