Construct validity and measurement invariance of MMIs: Do they allow meaningful group comparison?

Mustafa Asil, Isobel Cameron

Research output: Contribution to conferencePaperResearchpeer-review

28 Downloads (Pure)

Abstract


Introduction
Multiple mini interviews (MMI) measure multiple non-cognitive attributes and are widely used for selection to healthcare programmes. When instruments aim to assess subjective attributes, challenges for the validity and comparability of scores emerge. Thus, it is important to provide evidence that constructs have the same meaning across groups and group differences are not contaminated by group-specific characteristics (aka measurement invariance (MI)). Otherwise, interpretation of scores for members of different groups is not meaningful.

The present study investigates whether:
a.there is evidence to support factorial validity of MMI.
b.MMI measure the same underlying construct for students with different characteristics?
c.there are differences in MMI performance according to gender, age, disability status, interview time and gender of selector.

Methods
635 candidates were interviewed on a 7-station MMI in 2019/2020. Confirmatory factor analysis (CFA) assessed the factorial structure of 28-item MMI. We examined and compared the goodness-of-fit of different models. MI analyses were performed by Multi Group Confirmatory Factor Analysis (MG-CFA). Changes in CFI ≤.01 were used to determine invariance. Mean differences between groups were tested with independent samples t-tests.

Results/Discussion
Among measurement models tested, both seven-factor first order model and higher order model provided good fit to data. Reliability estimates (Cronbach’s alpha) for all factors ranged from 0.85 to 0.90. MI analyses showed MMI measures the same construct for candidates from different groups. Women and mature candidates scored significantly higher than men and younger students. Morning MMI candidates performed better than afternoon candidates.

Conclusion
This study provides psychometric evidence supporting the factorial validity and measurement equivalence (invariance) of MMIs. It is encouraging that there was no difference between MMI performances of students with and without disabilities. Equally heartening, we found no difference on MMI scores between students marked by male and female selectors.
Original languageEnglish
Number of pages1
Publication statusPublished - May 2021
EventNES Annual Virtual Conference 2021 - Virtual
Duration: 27 May 202128 May 2021
https://www.nes.scot.nhs.uk/nes-current/nes-annual-virtual-conference-2021/

Conference

ConferenceNES Annual Virtual Conference 2021
Period27/05/2128/05/21
Internet address

Fingerprint

Dive into the research topics of 'Construct validity and measurement invariance of MMIs: Do they allow meaningful group comparison?'. Together they form a unique fingerprint.

Cite this