Psychometric viability of measures of functional performance commonly used for people with dementia: A systematic review of measurement properties

Benjamin Fox, Timothy Henwood, Justin Keogh, Christine Neville

Research output: Contribution to journalArticleResearchpeer-review

Abstract

BACKGROUND: Confidence in findings can only be drawn from measurement tools that have sound psychometric properties for the population with which they are used. Within a dementia specific population, measures of physical function have been poorly justified in exercise intervention studies, with justification of measures based on validity or reliability studies from dissimilar clinical populations, such as people with bronchitis or healthy older adults without dementia.

OBJECTIVES: To review the reliability and validity of quantitative measures of pre-identified physical function, as commonly used within exercise intervention literature for adults with dementia.

INCLUSION CRITERIA TYPES OF PARTICIPANTS: Participants were adults, aged 65 years and older, with a confirmed medical diagnosis of dementia.

TYPES OF INTERVENTION(S)/PHENOMENA OF INTEREST: n/a

TYPES OF STUDIES: Desired studies were observational and cross-sectional and that assessed measures from a pre-identified list of measures of physical function.

OUTCOMES: Studies that assessed the psychometric constructs of reliability and validity were targeted. COSMIN taxology was used to define reliability and validity. This included, but were not limited to, Intra-Class Correlations, Kappa, Cronbach's Alpha, Chi Squared, Standard Error of Measurement, Minimal Detectable Change and Limits of Agreement.

SEARCH STRATEGY: Published material was sourced from the following four databases: MEDLINE, EMBASE, CINAHL and ISI Web of Science. Grey literature was searched for using ALOIS, Google Scholar and ProQuest.

METHODOLOGICAL QUALITY: The COSMIN checklist was used to assess methodological quality of included studies. Assessment was completed by two reviewers independently.

DATA EXTRACTION: Reliability and validity data was extracted from included studies using standardized Joanna Briggs Institute data collection forms. Extraction was completed by two reviewers.

DATA SYNTHESIS: A narrative synthesis of measurement properties of the tools used to measure physical function was performed. Quantitative meta-analysis was conducted for Intra-Class Correlation Coefficients only.

RESULTS: With respect to relative reliability, studies reporting assessed measures had intraclass correlation coefficients greater than 0.71, indicating their suitability for use at a group level. However, a consistent finding among studies that included assessment of absolute reliability was that intra individual variation was too large for meaningful measurement of individuals. This was indicated by large Minimal Detectable Change (MDC) scores. Walk Speed has the smallest reported Mimimal Detectable Change score at 0.11m/s. This represented a change of 35% before statistical variation could be eliminated as the cause for this change. All measures had large MDC values. Walk Speed had the smallest MDC values at 0.11m/s, which represented a necessary change of 35%. Only a limited number of studies assessed the validity of measures. This supports the use of these measures in a very narrow selection of circumstances (see Summary of Findings).

CONCLUSIONS: In summary, measures have shown appropriate levels of relative reliability. This supports their use at the group level. However, large levels of intra-individual variation undermine their applicability at the individual level. Limited studies of validity were available to this review, which limits a conclusion on whether measures are valid for people with dementia.

Original languageEnglish
Pages (from-to)115-71
Number of pages57
JournalJBI Database of Systematic Reviews and Implementation Reports
Volume14
Issue number8
DOIs
Publication statusPublished - Aug 2016

Fingerprint

Psychometrics
Reproducibility of Results
Dementia
Population
Bronchitis
Checklist
MEDLINE
Observational Studies
Meta-Analysis
Databases

Cite this

@article{718fc1b08dcb4a13aba419ccf05e83a5,
title = "Psychometric viability of measures of functional performance commonly used for people with dementia: A systematic review of measurement properties",
abstract = "BACKGROUND: Confidence in findings can only be drawn from measurement tools that have sound psychometric properties for the population with which they are used. Within a dementia specific population, measures of physical function have been poorly justified in exercise intervention studies, with justification of measures based on validity or reliability studies from dissimilar clinical populations, such as people with bronchitis or healthy older adults without dementia.OBJECTIVES: To review the reliability and validity of quantitative measures of pre-identified physical function, as commonly used within exercise intervention literature for adults with dementia.INCLUSION CRITERIA TYPES OF PARTICIPANTS: Participants were adults, aged 65 years and older, with a confirmed medical diagnosis of dementia.TYPES OF INTERVENTION(S)/PHENOMENA OF INTEREST: n/aTYPES OF STUDIES: Desired studies were observational and cross-sectional and that assessed measures from a pre-identified list of measures of physical function.OUTCOMES: Studies that assessed the psychometric constructs of reliability and validity were targeted. COSMIN taxology was used to define reliability and validity. This included, but were not limited to, Intra-Class Correlations, Kappa, Cronbach's Alpha, Chi Squared, Standard Error of Measurement, Minimal Detectable Change and Limits of Agreement.SEARCH STRATEGY: Published material was sourced from the following four databases: MEDLINE, EMBASE, CINAHL and ISI Web of Science. Grey literature was searched for using ALOIS, Google Scholar and ProQuest.METHODOLOGICAL QUALITY: The COSMIN checklist was used to assess methodological quality of included studies. Assessment was completed by two reviewers independently.DATA EXTRACTION: Reliability and validity data was extracted from included studies using standardized Joanna Briggs Institute data collection forms. Extraction was completed by two reviewers.DATA SYNTHESIS: A narrative synthesis of measurement properties of the tools used to measure physical function was performed. Quantitative meta-analysis was conducted for Intra-Class Correlation Coefficients only.RESULTS: With respect to relative reliability, studies reporting assessed measures had intraclass correlation coefficients greater than 0.71, indicating their suitability for use at a group level. However, a consistent finding among studies that included assessment of absolute reliability was that intra individual variation was too large for meaningful measurement of individuals. This was indicated by large Minimal Detectable Change (MDC) scores. Walk Speed has the smallest reported Mimimal Detectable Change score at 0.11m/s. This represented a change of 35{\%} before statistical variation could be eliminated as the cause for this change. All measures had large MDC values. Walk Speed had the smallest MDC values at 0.11m/s, which represented a necessary change of 35{\%}. Only a limited number of studies assessed the validity of measures. This supports the use of these measures in a very narrow selection of circumstances (see Summary of Findings).CONCLUSIONS: In summary, measures have shown appropriate levels of relative reliability. This supports their use at the group level. However, large levels of intra-individual variation undermine their applicability at the individual level. Limited studies of validity were available to this review, which limits a conclusion on whether measures are valid for people with dementia.",
author = "Benjamin Fox and Timothy Henwood and Justin Keogh and Christine Neville",
year = "2016",
month = "8",
doi = "10.11124/JBISRIR-2016-003064",
language = "English",
volume = "14",
pages = "115--71",
journal = "JBI Database of Systematic Reviews and Implementation Reports",
issn = "2202-4433",
publisher = "Joanna Briggs Institute",
number = "8",

}

Psychometric viability of measures of functional performance commonly used for people with dementia: A systematic review of measurement properties. / Fox, Benjamin; Henwood, Timothy; Keogh, Justin; Neville, Christine.

In: JBI Database of Systematic Reviews and Implementation Reports, Vol. 14, No. 8, 08.2016, p. 115-71.

Research output: Contribution to journalArticleResearchpeer-review

TY - JOUR

T1 - Psychometric viability of measures of functional performance commonly used for people with dementia: A systematic review of measurement properties

AU - Fox, Benjamin

AU - Henwood, Timothy

AU - Keogh, Justin

AU - Neville, Christine

PY - 2016/8

Y1 - 2016/8

N2 - BACKGROUND: Confidence in findings can only be drawn from measurement tools that have sound psychometric properties for the population with which they are used. Within a dementia specific population, measures of physical function have been poorly justified in exercise intervention studies, with justification of measures based on validity or reliability studies from dissimilar clinical populations, such as people with bronchitis or healthy older adults without dementia.OBJECTIVES: To review the reliability and validity of quantitative measures of pre-identified physical function, as commonly used within exercise intervention literature for adults with dementia.INCLUSION CRITERIA TYPES OF PARTICIPANTS: Participants were adults, aged 65 years and older, with a confirmed medical diagnosis of dementia.TYPES OF INTERVENTION(S)/PHENOMENA OF INTEREST: n/aTYPES OF STUDIES: Desired studies were observational and cross-sectional and that assessed measures from a pre-identified list of measures of physical function.OUTCOMES: Studies that assessed the psychometric constructs of reliability and validity were targeted. COSMIN taxology was used to define reliability and validity. This included, but were not limited to, Intra-Class Correlations, Kappa, Cronbach's Alpha, Chi Squared, Standard Error of Measurement, Minimal Detectable Change and Limits of Agreement.SEARCH STRATEGY: Published material was sourced from the following four databases: MEDLINE, EMBASE, CINAHL and ISI Web of Science. Grey literature was searched for using ALOIS, Google Scholar and ProQuest.METHODOLOGICAL QUALITY: The COSMIN checklist was used to assess methodological quality of included studies. Assessment was completed by two reviewers independently.DATA EXTRACTION: Reliability and validity data was extracted from included studies using standardized Joanna Briggs Institute data collection forms. Extraction was completed by two reviewers.DATA SYNTHESIS: A narrative synthesis of measurement properties of the tools used to measure physical function was performed. Quantitative meta-analysis was conducted for Intra-Class Correlation Coefficients only.RESULTS: With respect to relative reliability, studies reporting assessed measures had intraclass correlation coefficients greater than 0.71, indicating their suitability for use at a group level. However, a consistent finding among studies that included assessment of absolute reliability was that intra individual variation was too large for meaningful measurement of individuals. This was indicated by large Minimal Detectable Change (MDC) scores. Walk Speed has the smallest reported Mimimal Detectable Change score at 0.11m/s. This represented a change of 35% before statistical variation could be eliminated as the cause for this change. All measures had large MDC values. Walk Speed had the smallest MDC values at 0.11m/s, which represented a necessary change of 35%. Only a limited number of studies assessed the validity of measures. This supports the use of these measures in a very narrow selection of circumstances (see Summary of Findings).CONCLUSIONS: In summary, measures have shown appropriate levels of relative reliability. This supports their use at the group level. However, large levels of intra-individual variation undermine their applicability at the individual level. Limited studies of validity were available to this review, which limits a conclusion on whether measures are valid for people with dementia.

AB - BACKGROUND: Confidence in findings can only be drawn from measurement tools that have sound psychometric properties for the population with which they are used. Within a dementia specific population, measures of physical function have been poorly justified in exercise intervention studies, with justification of measures based on validity or reliability studies from dissimilar clinical populations, such as people with bronchitis or healthy older adults without dementia.OBJECTIVES: To review the reliability and validity of quantitative measures of pre-identified physical function, as commonly used within exercise intervention literature for adults with dementia.INCLUSION CRITERIA TYPES OF PARTICIPANTS: Participants were adults, aged 65 years and older, with a confirmed medical diagnosis of dementia.TYPES OF INTERVENTION(S)/PHENOMENA OF INTEREST: n/aTYPES OF STUDIES: Desired studies were observational and cross-sectional and that assessed measures from a pre-identified list of measures of physical function.OUTCOMES: Studies that assessed the psychometric constructs of reliability and validity were targeted. COSMIN taxology was used to define reliability and validity. This included, but were not limited to, Intra-Class Correlations, Kappa, Cronbach's Alpha, Chi Squared, Standard Error of Measurement, Minimal Detectable Change and Limits of Agreement.SEARCH STRATEGY: Published material was sourced from the following four databases: MEDLINE, EMBASE, CINAHL and ISI Web of Science. Grey literature was searched for using ALOIS, Google Scholar and ProQuest.METHODOLOGICAL QUALITY: The COSMIN checklist was used to assess methodological quality of included studies. Assessment was completed by two reviewers independently.DATA EXTRACTION: Reliability and validity data was extracted from included studies using standardized Joanna Briggs Institute data collection forms. Extraction was completed by two reviewers.DATA SYNTHESIS: A narrative synthesis of measurement properties of the tools used to measure physical function was performed. Quantitative meta-analysis was conducted for Intra-Class Correlation Coefficients only.RESULTS: With respect to relative reliability, studies reporting assessed measures had intraclass correlation coefficients greater than 0.71, indicating their suitability for use at a group level. However, a consistent finding among studies that included assessment of absolute reliability was that intra individual variation was too large for meaningful measurement of individuals. This was indicated by large Minimal Detectable Change (MDC) scores. Walk Speed has the smallest reported Mimimal Detectable Change score at 0.11m/s. This represented a change of 35% before statistical variation could be eliminated as the cause for this change. All measures had large MDC values. Walk Speed had the smallest MDC values at 0.11m/s, which represented a necessary change of 35%. Only a limited number of studies assessed the validity of measures. This supports the use of these measures in a very narrow selection of circumstances (see Summary of Findings).CONCLUSIONS: In summary, measures have shown appropriate levels of relative reliability. This supports their use at the group level. However, large levels of intra-individual variation undermine their applicability at the individual level. Limited studies of validity were available to this review, which limits a conclusion on whether measures are valid for people with dementia.

U2 - 10.11124/JBISRIR-2016-003064

DO - 10.11124/JBISRIR-2016-003064

M3 - Article

VL - 14

SP - 115

EP - 171

JO - JBI Database of Systematic Reviews and Implementation Reports

JF - JBI Database of Systematic Reviews and Implementation Reports

SN - 2202-4433

IS - 8

ER -