Assessing the suitability of written stroke materials

An evaluation of the interrater reliability of the suitability assessment of materials (SAM) checklist

Tammy Hoffmann, Yvette Ladner

Research output: Contribution to journalArticleResearchpeer-review

12 Citations (Scopus)

Abstract

Purpose: Written materials are frequently used to provide education to stroke patients and their carers. However, poor quality materials are a barrier to effective information provision. A quick and reliable method of evaluating material quality is needed. This study evaluated the interrater reliability of the Suitability Assessment of Materials (SAM) checklist in a sample of written stroke education materials. Methods: Two independent raters evaluated the materials (n = 25) using the SAM, and ratings were analyzed to reveal total percentage agreements and weighted kappa values for individual items and overall SAM rating. Results: The majority of the individual SAM items had high interrater reliability, with 17 of the 22 items achieving substantial, almost perfect, or perfect weighted kappa value scores. The overall SAM rating achieved a weighted kappa value of 0.60, with a percentage total agreement of 96%. Conclusion: Health care professionals should evaluate the content and design characteristics of written education materials before using them with patients. A tool such as the SAM checklist can be used; however, raters should exercise caution when interpreting results from items with more subjective scoring criteria. Refinements to the scoring criteria for these items are recommended. The value of the SAM is that it can be used to identify specific elements that should be modified before education materials are provided to patients.

Original languageEnglish
Pages (from-to)417-422
Number of pages6
JournalTopics in Stroke Rehabilitation
Volume19
Issue number5
DOIs
Publication statusPublished - 1 Jan 2012

Fingerprint

Checklist
Stroke
Education
Caregivers
Exercise
Delivery of Health Care

Cite this

@article{0481e9c3f7c14756869b254d79abe477,
title = "Assessing the suitability of written stroke materials: An evaluation of the interrater reliability of the suitability assessment of materials (SAM) checklist",
abstract = "Purpose: Written materials are frequently used to provide education to stroke patients and their carers. However, poor quality materials are a barrier to effective information provision. A quick and reliable method of evaluating material quality is needed. This study evaluated the interrater reliability of the Suitability Assessment of Materials (SAM) checklist in a sample of written stroke education materials. Methods: Two independent raters evaluated the materials (n = 25) using the SAM, and ratings were analyzed to reveal total percentage agreements and weighted kappa values for individual items and overall SAM rating. Results: The majority of the individual SAM items had high interrater reliability, with 17 of the 22 items achieving substantial, almost perfect, or perfect weighted kappa value scores. The overall SAM rating achieved a weighted kappa value of 0.60, with a percentage total agreement of 96{\%}. Conclusion: Health care professionals should evaluate the content and design characteristics of written education materials before using them with patients. A tool such as the SAM checklist can be used; however, raters should exercise caution when interpreting results from items with more subjective scoring criteria. Refinements to the scoring criteria for these items are recommended. The value of the SAM is that it can be used to identify specific elements that should be modified before education materials are provided to patients.",
author = "Tammy Hoffmann and Yvette Ladner",
year = "2012",
month = "1",
day = "1",
doi = "10.1310/tsr1905-417",
language = "English",
volume = "19",
pages = "417--422",
journal = "Topics in Stroke Rehabilitation",
issn = "1074-9357",
publisher = "Thomas Land Publishers Inc.",
number = "5",

}

Assessing the suitability of written stroke materials : An evaluation of the interrater reliability of the suitability assessment of materials (SAM) checklist. / Hoffmann, Tammy; Ladner, Yvette.

In: Topics in Stroke Rehabilitation, Vol. 19, No. 5, 01.01.2012, p. 417-422.

Research output: Contribution to journalArticleResearchpeer-review

TY - JOUR

T1 - Assessing the suitability of written stroke materials

T2 - An evaluation of the interrater reliability of the suitability assessment of materials (SAM) checklist

AU - Hoffmann, Tammy

AU - Ladner, Yvette

PY - 2012/1/1

Y1 - 2012/1/1

N2 - Purpose: Written materials are frequently used to provide education to stroke patients and their carers. However, poor quality materials are a barrier to effective information provision. A quick and reliable method of evaluating material quality is needed. This study evaluated the interrater reliability of the Suitability Assessment of Materials (SAM) checklist in a sample of written stroke education materials. Methods: Two independent raters evaluated the materials (n = 25) using the SAM, and ratings were analyzed to reveal total percentage agreements and weighted kappa values for individual items and overall SAM rating. Results: The majority of the individual SAM items had high interrater reliability, with 17 of the 22 items achieving substantial, almost perfect, or perfect weighted kappa value scores. The overall SAM rating achieved a weighted kappa value of 0.60, with a percentage total agreement of 96%. Conclusion: Health care professionals should evaluate the content and design characteristics of written education materials before using them with patients. A tool such as the SAM checklist can be used; however, raters should exercise caution when interpreting results from items with more subjective scoring criteria. Refinements to the scoring criteria for these items are recommended. The value of the SAM is that it can be used to identify specific elements that should be modified before education materials are provided to patients.

AB - Purpose: Written materials are frequently used to provide education to stroke patients and their carers. However, poor quality materials are a barrier to effective information provision. A quick and reliable method of evaluating material quality is needed. This study evaluated the interrater reliability of the Suitability Assessment of Materials (SAM) checklist in a sample of written stroke education materials. Methods: Two independent raters evaluated the materials (n = 25) using the SAM, and ratings were analyzed to reveal total percentage agreements and weighted kappa values for individual items and overall SAM rating. Results: The majority of the individual SAM items had high interrater reliability, with 17 of the 22 items achieving substantial, almost perfect, or perfect weighted kappa value scores. The overall SAM rating achieved a weighted kappa value of 0.60, with a percentage total agreement of 96%. Conclusion: Health care professionals should evaluate the content and design characteristics of written education materials before using them with patients. A tool such as the SAM checklist can be used; however, raters should exercise caution when interpreting results from items with more subjective scoring criteria. Refinements to the scoring criteria for these items are recommended. The value of the SAM is that it can be used to identify specific elements that should be modified before education materials are provided to patients.

UR - http://www.scopus.com/inward/record.url?scp=84866530150&partnerID=8YFLogxK

U2 - 10.1310/tsr1905-417

DO - 10.1310/tsr1905-417

M3 - Article

VL - 19

SP - 417

EP - 422

JO - Topics in Stroke Rehabilitation

JF - Topics in Stroke Rehabilitation

SN - 1074-9357

IS - 5

ER -