Factors influencing General Practitioners in their attribution of a Global Assessment in medical training

Marisa Magiros, Graham Emblem, Marie Louise Dick, Scott Preston, Gerald Ingham, Jane W Smith, Joanna Fisher

Research output: Contribution to conferenceAbstractResearchpeer-review

Abstract

Global Assessments (GAs) are used in
Australian General Practice (GP) training during selection,
in-training reviews, and within summative assessment.
This project aimed to determine factors influencing
assessors in their assignment of a GA score.
A modified Delphi process was used with participants
recruited from GP Supervisor and Medical Educator groups
nationwide. Consensus information obtained via
questionnaires was reflected to the group for comment in
subsequent Delphi rounds.
Demographic items collected included educator role, level
of experience, and total number of doctors supervised.
Participants were asked where they had performed GAs,
and the factors they considered when making a GA.
Participants ranked these factors independently, and in
relation to training level of the doctor being observed, as
well as commenting on consensus rankings. Participants
rated their confidence in GAs as an accurate determinant
of GP competence. Participants were asked to identify
personal biases, and their approach to discrepancies in GA
scores.
Of the 28 participants engaging in four Delphi rounds,
most were female, aged over 40, and had roles as Medical
Educators. GAs were most commonly used in direct
observation of practice, formatively and summatively.
Clinical knowledge, conscious incompetence,
communication skills and help-seeking practices were
ranked highly in considering GA. There was good
agreement amongst participants regarding criteria
significance across the training continuum and the
robustness of GA. There was conflicting opinion about
what skills and factors can be learnt versus what should be
inherent characteristics of a doctor.
The factors contributing to a GA are broad and not limited
to assessment of knowledge and skills, but include the
non-clinical domains, namely communication,
professionalism and organisational skills. Trust in the
validity of GA by participants was strong, particularly when
multiple assessors are involved. Personal biases do exist,
and it is unknown at this stage whether or how they are
overcome by assessors when making final judgment.
The strength of GA appears to be drawn from the breadth
of factors considered that go beyond ‘clinical’ checklists by
allowing for overall impressions and gut feeling, providing
a ‘rounded approach’ to competency.
Original languageEnglish
Pages59
Number of pages1
Publication statusPublished - 2018
EventAssociation for Medical Education in Europe Conference (AMEE) 2018 - Congress Center Basel, Basel, Switzerland
Duration: 25 Aug 201829 Aug 2018
https://amee.org/conferences/amee-past-conferences/amee-2018

Conference

ConferenceAssociation for Medical Education in Europe Conference (AMEE) 2018
Abbreviated titleAMEE
CountrySwitzerland
CityBasel
Period25/08/1829/08/18
Internet address

Fingerprint

General Practice
General Practitioners
Consensus
Communication
Checklist
Mental Competency
Emotions
Demography
Professionalism

Cite this

Magiros, M., Emblem, G., Dick, M. L., Preston, S., Ingham, G., Smith, J. W., & Fisher, J. (2018). Factors influencing General Practitioners in their attribution of a Global Assessment in medical training. 59. Abstract from Association for Medical Education in Europe Conference (AMEE) 2018, Basel, Switzerland.
Magiros, Marisa ; Emblem, Graham ; Dick, Marie Louise ; Preston, Scott ; Ingham, Gerald ; Smith, Jane W ; Fisher, Joanna. / Factors influencing General Practitioners in their attribution of a Global Assessment in medical training. Abstract from Association for Medical Education in Europe Conference (AMEE) 2018, Basel, Switzerland.1 p.
@conference{000fafd3fd5c497d86d9145355234825,
title = "Factors influencing General Practitioners in their attribution of a Global Assessment in medical training",
abstract = "Global Assessments (GAs) are used inAustralian General Practice (GP) training during selection,in-training reviews, and within summative assessment.This project aimed to determine factors influencingassessors in their assignment of a GA score.A modified Delphi process was used with participantsrecruited from GP Supervisor and Medical Educator groupsnationwide. Consensus information obtained viaquestionnaires was reflected to the group for comment insubsequent Delphi rounds.Demographic items collected included educator role, levelof experience, and total number of doctors supervised.Participants were asked where they had performed GAs,and the factors they considered when making a GA.Participants ranked these factors independently, and inrelation to training level of the doctor being observed, aswell as commenting on consensus rankings. Participantsrated their confidence in GAs as an accurate determinantof GP competence. Participants were asked to identifypersonal biases, and their approach to discrepancies in GAscores.Of the 28 participants engaging in four Delphi rounds,most were female, aged over 40, and had roles as MedicalEducators. GAs were most commonly used in directobservation of practice, formatively and summatively.Clinical knowledge, conscious incompetence,communication skills and help-seeking practices wereranked highly in considering GA. There was goodagreement amongst participants regarding criteriasignificance across the training continuum and therobustness of GA. There was conflicting opinion aboutwhat skills and factors can be learnt versus what should beinherent characteristics of a doctor.The factors contributing to a GA are broad and not limitedto assessment of knowledge and skills, but include thenon-clinical domains, namely communication,professionalism and organisational skills. Trust in thevalidity of GA by participants was strong, particularly whenmultiple assessors are involved. Personal biases do exist,and it is unknown at this stage whether or how they areovercome by assessors when making final judgment.The strength of GA appears to be drawn from the breadthof factors considered that go beyond ‘clinical’ checklists byallowing for overall impressions and gut feeling, providinga ‘rounded approach’ to competency.",
author = "Marisa Magiros and Graham Emblem and Dick, {Marie Louise} and Scott Preston and Gerald Ingham and Smith, {Jane W} and Joanna Fisher",
year = "2018",
language = "English",
pages = "59",
note = "Association for Medical Education in Europe Conference (AMEE) 2018, AMEE ; Conference date: 25-08-2018 Through 29-08-2018",
url = "https://amee.org/conferences/amee-past-conferences/amee-2018",

}

Magiros, M, Emblem, G, Dick, ML, Preston, S, Ingham, G, Smith, JW & Fisher, J 2018, 'Factors influencing General Practitioners in their attribution of a Global Assessment in medical training' Association for Medical Education in Europe Conference (AMEE) 2018, Basel, Switzerland, 25/08/18 - 29/08/18, pp. 59.

Factors influencing General Practitioners in their attribution of a Global Assessment in medical training. / Magiros, Marisa; Emblem, Graham ; Dick, Marie Louise; Preston, Scott; Ingham, Gerald ; Smith, Jane W; Fisher, Joanna.

2018. 59 Abstract from Association for Medical Education in Europe Conference (AMEE) 2018, Basel, Switzerland.

Research output: Contribution to conferenceAbstractResearchpeer-review

TY - CONF

T1 - Factors influencing General Practitioners in their attribution of a Global Assessment in medical training

AU - Magiros, Marisa

AU - Emblem, Graham

AU - Dick, Marie Louise

AU - Preston, Scott

AU - Ingham, Gerald

AU - Smith, Jane W

AU - Fisher, Joanna

PY - 2018

Y1 - 2018

N2 - Global Assessments (GAs) are used inAustralian General Practice (GP) training during selection,in-training reviews, and within summative assessment.This project aimed to determine factors influencingassessors in their assignment of a GA score.A modified Delphi process was used with participantsrecruited from GP Supervisor and Medical Educator groupsnationwide. Consensus information obtained viaquestionnaires was reflected to the group for comment insubsequent Delphi rounds.Demographic items collected included educator role, levelof experience, and total number of doctors supervised.Participants were asked where they had performed GAs,and the factors they considered when making a GA.Participants ranked these factors independently, and inrelation to training level of the doctor being observed, aswell as commenting on consensus rankings. Participantsrated their confidence in GAs as an accurate determinantof GP competence. Participants were asked to identifypersonal biases, and their approach to discrepancies in GAscores.Of the 28 participants engaging in four Delphi rounds,most were female, aged over 40, and had roles as MedicalEducators. GAs were most commonly used in directobservation of practice, formatively and summatively.Clinical knowledge, conscious incompetence,communication skills and help-seeking practices wereranked highly in considering GA. There was goodagreement amongst participants regarding criteriasignificance across the training continuum and therobustness of GA. There was conflicting opinion aboutwhat skills and factors can be learnt versus what should beinherent characteristics of a doctor.The factors contributing to a GA are broad and not limitedto assessment of knowledge and skills, but include thenon-clinical domains, namely communication,professionalism and organisational skills. Trust in thevalidity of GA by participants was strong, particularly whenmultiple assessors are involved. Personal biases do exist,and it is unknown at this stage whether or how they areovercome by assessors when making final judgment.The strength of GA appears to be drawn from the breadthof factors considered that go beyond ‘clinical’ checklists byallowing for overall impressions and gut feeling, providinga ‘rounded approach’ to competency.

AB - Global Assessments (GAs) are used inAustralian General Practice (GP) training during selection,in-training reviews, and within summative assessment.This project aimed to determine factors influencingassessors in their assignment of a GA score.A modified Delphi process was used with participantsrecruited from GP Supervisor and Medical Educator groupsnationwide. Consensus information obtained viaquestionnaires was reflected to the group for comment insubsequent Delphi rounds.Demographic items collected included educator role, levelof experience, and total number of doctors supervised.Participants were asked where they had performed GAs,and the factors they considered when making a GA.Participants ranked these factors independently, and inrelation to training level of the doctor being observed, aswell as commenting on consensus rankings. Participantsrated their confidence in GAs as an accurate determinantof GP competence. Participants were asked to identifypersonal biases, and their approach to discrepancies in GAscores.Of the 28 participants engaging in four Delphi rounds,most were female, aged over 40, and had roles as MedicalEducators. GAs were most commonly used in directobservation of practice, formatively and summatively.Clinical knowledge, conscious incompetence,communication skills and help-seeking practices wereranked highly in considering GA. There was goodagreement amongst participants regarding criteriasignificance across the training continuum and therobustness of GA. There was conflicting opinion aboutwhat skills and factors can be learnt versus what should beinherent characteristics of a doctor.The factors contributing to a GA are broad and not limitedto assessment of knowledge and skills, but include thenon-clinical domains, namely communication,professionalism and organisational skills. Trust in thevalidity of GA by participants was strong, particularly whenmultiple assessors are involved. Personal biases do exist,and it is unknown at this stage whether or how they areovercome by assessors when making final judgment.The strength of GA appears to be drawn from the breadthof factors considered that go beyond ‘clinical’ checklists byallowing for overall impressions and gut feeling, providinga ‘rounded approach’ to competency.

UR - https://amee.org/getattachment/Conferences/AMEE-Past-Conferences/AMEE-2018/AMEE-2018-Programme.pdf

M3 - Abstract

SP - 59

ER -

Magiros M, Emblem G, Dick ML, Preston S, Ingham G, Smith JW et al. Factors influencing General Practitioners in their attribution of a Global Assessment in medical training. 2018. Abstract from Association for Medical Education in Europe Conference (AMEE) 2018, Basel, Switzerland.