Use of Online or Paper Surveys by Australian Women: Longitudinal Study of Users, Devices, and Cohort Retention

David Fitzgerald, Richard Hockey, Mark Jones, Gita Mishra, Michael Waller, Annette Dobson

Research output: Contribution to journalArticleResearchpeer-review

2 Downloads (Pure)

Abstract

BACKGROUND: There is increasing use of online surveys to improve data quality and timeliness and reduce costs. While there have been numerous cross-sectional studies comparing responses to online or paper surveys, there is little research from a longitudinal perspective. OBJECTIVE: In the context of the well-established Australian Longitudinal Study on Women's Health, we examined the patterns of responses to online or paper surveys across the first two waves of the study in which both modes were offered. We compared the following: differences between women born between 1946 and 1951 and between 1973 and 1978; types of device used for online completion; sociodemographic, behavioral, and health characteristics of women who responded online or using mailed paper surveys; and associations between mode of completion in the first survey and participation and mode of completion in the second survey. METHODS: Participants in this study, who had responded to regular mailed surveys since 1996, were offered a choice of completing surveys using paper questionnaires or Web-based electronic questionnaires starting in 2012. Two groups of women were involved: an older cohort born between 1946 and 1951 aged in their 60s and a younger cohort born between 1973 and 1978 aged in their 30s when the online surveys were first introduced. We compared women who responded online on both occasions, women who responded online at the first survey and used the paper version of the second survey, women who changed from paper to online, and those who used paper for both surveys. RESULTS: Of the 9663 women in their 60s who responded to one or both surveys, more than 50% preferred paper surveys (5290/9663, 54.74%, on the first survey and 5373/8621, 62.32%, on the second survey). If they chose the online version, most used computers. In contrast, of the 8628 women in their 30s, 56.04% (4835/8628) chose the online version at the first survey. While most favored computers to phones or tablets, many did try these alternatives on the subsequent survey. Many women who completed the survey online the first time preferred the paper version on the subsequent survey. In fact, for women in their 60s, the number who went from online to paper (1151/3851, 29.89%) exceeded the number who went from paper to online (734/5290, 13.88%). The online option was more likely to be chosen by better educated and healthier women. In both cohorts, women who completed paper surveys were more likely than online completers to become nonrespondents on the next survey. Due to the large sample size, almost all differences were statistically significant, with P<.001. CONCLUSIONS: Despite the cost-saving advantages of online compared to paper surveys, paper surveys are likely to appeal to a different population of potential respondents with different sociodemographic, behavioral, and health characteristics and greater likelihood of attrition from the study. Not offering a paper version is therefore likely to induce bias in the distribution of responses unless weighting for respondent characteristics (relative to the target population) is employed. Therefore, if mixed mode (paper or online) options are feasible, they are highly likely to produce more representative results than if only the less costly online option is offered.

Original languageEnglish
Article numbere10672
Number of pages11
JournalJournal of Medical Internet Research
Volume21
Issue number3
DOIs
Publication statusPublished - 14 Mar 2019
Externally publishedYes

Fingerprint

Longitudinal Studies
Equipment and Supplies
Surveys and Questionnaires
Women's Health
Costs and Cost Analysis

Cite this

Fitzgerald, David ; Hockey, Richard ; Jones, Mark ; Mishra, Gita ; Waller, Michael ; Dobson, Annette. / Use of Online or Paper Surveys by Australian Women : Longitudinal Study of Users, Devices, and Cohort Retention. In: Journal of Medical Internet Research. 2019 ; Vol. 21, No. 3.
@article{8b16acbbde98401fafeaaf098119808e,
title = "Use of Online or Paper Surveys by Australian Women: Longitudinal Study of Users, Devices, and Cohort Retention",
abstract = "BACKGROUND: There is increasing use of online surveys to improve data quality and timeliness and reduce costs. While there have been numerous cross-sectional studies comparing responses to online or paper surveys, there is little research from a longitudinal perspective. OBJECTIVE: In the context of the well-established Australian Longitudinal Study on Women's Health, we examined the patterns of responses to online or paper surveys across the first two waves of the study in which both modes were offered. We compared the following: differences between women born between 1946 and 1951 and between 1973 and 1978; types of device used for online completion; sociodemographic, behavioral, and health characteristics of women who responded online or using mailed paper surveys; and associations between mode of completion in the first survey and participation and mode of completion in the second survey. METHODS: Participants in this study, who had responded to regular mailed surveys since 1996, were offered a choice of completing surveys using paper questionnaires or Web-based electronic questionnaires starting in 2012. Two groups of women were involved: an older cohort born between 1946 and 1951 aged in their 60s and a younger cohort born between 1973 and 1978 aged in their 30s when the online surveys were first introduced. We compared women who responded online on both occasions, women who responded online at the first survey and used the paper version of the second survey, women who changed from paper to online, and those who used paper for both surveys. RESULTS: Of the 9663 women in their 60s who responded to one or both surveys, more than 50{\%} preferred paper surveys (5290/9663, 54.74{\%}, on the first survey and 5373/8621, 62.32{\%}, on the second survey). If they chose the online version, most used computers. In contrast, of the 8628 women in their 30s, 56.04{\%} (4835/8628) chose the online version at the first survey. While most favored computers to phones or tablets, many did try these alternatives on the subsequent survey. Many women who completed the survey online the first time preferred the paper version on the subsequent survey. In fact, for women in their 60s, the number who went from online to paper (1151/3851, 29.89{\%}) exceeded the number who went from paper to online (734/5290, 13.88{\%}). The online option was more likely to be chosen by better educated and healthier women. In both cohorts, women who completed paper surveys were more likely than online completers to become nonrespondents on the next survey. Due to the large sample size, almost all differences were statistically significant, with P<.001. CONCLUSIONS: Despite the cost-saving advantages of online compared to paper surveys, paper surveys are likely to appeal to a different population of potential respondents with different sociodemographic, behavioral, and health characteristics and greater likelihood of attrition from the study. Not offering a paper version is therefore likely to induce bias in the distribution of responses unless weighting for respondent characteristics (relative to the target population) is employed. Therefore, if mixed mode (paper or online) options are feasible, they are highly likely to produce more representative results than if only the less costly online option is offered.",
author = "David Fitzgerald and Richard Hockey and Mark Jones and Gita Mishra and Michael Waller and Annette Dobson",
note = "{\circledC}David Fitzgerald, Richard Hockey, Mark Jones, Gita Mishra, Michael Waller, Annette Dobson. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 14.03.2019.",
year = "2019",
month = "3",
day = "14",
doi = "10.2196/10672",
language = "English",
volume = "21",
journal = "Journal of Medical Internet Research",
issn = "1438-8871",
publisher = "Journal of medical Internet Research",
number = "3",

}

Use of Online or Paper Surveys by Australian Women : Longitudinal Study of Users, Devices, and Cohort Retention. / Fitzgerald, David; Hockey, Richard; Jones, Mark; Mishra, Gita; Waller, Michael; Dobson, Annette.

In: Journal of Medical Internet Research, Vol. 21, No. 3, e10672, 14.03.2019.

Research output: Contribution to journalArticleResearchpeer-review

TY - JOUR

T1 - Use of Online or Paper Surveys by Australian Women

T2 - Longitudinal Study of Users, Devices, and Cohort Retention

AU - Fitzgerald, David

AU - Hockey, Richard

AU - Jones, Mark

AU - Mishra, Gita

AU - Waller, Michael

AU - Dobson, Annette

N1 - ©David Fitzgerald, Richard Hockey, Mark Jones, Gita Mishra, Michael Waller, Annette Dobson. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 14.03.2019.

PY - 2019/3/14

Y1 - 2019/3/14

N2 - BACKGROUND: There is increasing use of online surveys to improve data quality and timeliness and reduce costs. While there have been numerous cross-sectional studies comparing responses to online or paper surveys, there is little research from a longitudinal perspective. OBJECTIVE: In the context of the well-established Australian Longitudinal Study on Women's Health, we examined the patterns of responses to online or paper surveys across the first two waves of the study in which both modes were offered. We compared the following: differences between women born between 1946 and 1951 and between 1973 and 1978; types of device used for online completion; sociodemographic, behavioral, and health characteristics of women who responded online or using mailed paper surveys; and associations between mode of completion in the first survey and participation and mode of completion in the second survey. METHODS: Participants in this study, who had responded to regular mailed surveys since 1996, were offered a choice of completing surveys using paper questionnaires or Web-based electronic questionnaires starting in 2012. Two groups of women were involved: an older cohort born between 1946 and 1951 aged in their 60s and a younger cohort born between 1973 and 1978 aged in their 30s when the online surveys were first introduced. We compared women who responded online on both occasions, women who responded online at the first survey and used the paper version of the second survey, women who changed from paper to online, and those who used paper for both surveys. RESULTS: Of the 9663 women in their 60s who responded to one or both surveys, more than 50% preferred paper surveys (5290/9663, 54.74%, on the first survey and 5373/8621, 62.32%, on the second survey). If they chose the online version, most used computers. In contrast, of the 8628 women in their 30s, 56.04% (4835/8628) chose the online version at the first survey. While most favored computers to phones or tablets, many did try these alternatives on the subsequent survey. Many women who completed the survey online the first time preferred the paper version on the subsequent survey. In fact, for women in their 60s, the number who went from online to paper (1151/3851, 29.89%) exceeded the number who went from paper to online (734/5290, 13.88%). The online option was more likely to be chosen by better educated and healthier women. In both cohorts, women who completed paper surveys were more likely than online completers to become nonrespondents on the next survey. Due to the large sample size, almost all differences were statistically significant, with P<.001. CONCLUSIONS: Despite the cost-saving advantages of online compared to paper surveys, paper surveys are likely to appeal to a different population of potential respondents with different sociodemographic, behavioral, and health characteristics and greater likelihood of attrition from the study. Not offering a paper version is therefore likely to induce bias in the distribution of responses unless weighting for respondent characteristics (relative to the target population) is employed. Therefore, if mixed mode (paper or online) options are feasible, they are highly likely to produce more representative results than if only the less costly online option is offered.

AB - BACKGROUND: There is increasing use of online surveys to improve data quality and timeliness and reduce costs. While there have been numerous cross-sectional studies comparing responses to online or paper surveys, there is little research from a longitudinal perspective. OBJECTIVE: In the context of the well-established Australian Longitudinal Study on Women's Health, we examined the patterns of responses to online or paper surveys across the first two waves of the study in which both modes were offered. We compared the following: differences between women born between 1946 and 1951 and between 1973 and 1978; types of device used for online completion; sociodemographic, behavioral, and health characteristics of women who responded online or using mailed paper surveys; and associations between mode of completion in the first survey and participation and mode of completion in the second survey. METHODS: Participants in this study, who had responded to regular mailed surveys since 1996, were offered a choice of completing surveys using paper questionnaires or Web-based electronic questionnaires starting in 2012. Two groups of women were involved: an older cohort born between 1946 and 1951 aged in their 60s and a younger cohort born between 1973 and 1978 aged in their 30s when the online surveys were first introduced. We compared women who responded online on both occasions, women who responded online at the first survey and used the paper version of the second survey, women who changed from paper to online, and those who used paper for both surveys. RESULTS: Of the 9663 women in their 60s who responded to one or both surveys, more than 50% preferred paper surveys (5290/9663, 54.74%, on the first survey and 5373/8621, 62.32%, on the second survey). If they chose the online version, most used computers. In contrast, of the 8628 women in their 30s, 56.04% (4835/8628) chose the online version at the first survey. While most favored computers to phones or tablets, many did try these alternatives on the subsequent survey. Many women who completed the survey online the first time preferred the paper version on the subsequent survey. In fact, for women in their 60s, the number who went from online to paper (1151/3851, 29.89%) exceeded the number who went from paper to online (734/5290, 13.88%). The online option was more likely to be chosen by better educated and healthier women. In both cohorts, women who completed paper surveys were more likely than online completers to become nonrespondents on the next survey. Due to the large sample size, almost all differences were statistically significant, with P<.001. CONCLUSIONS: Despite the cost-saving advantages of online compared to paper surveys, paper surveys are likely to appeal to a different population of potential respondents with different sociodemographic, behavioral, and health characteristics and greater likelihood of attrition from the study. Not offering a paper version is therefore likely to induce bias in the distribution of responses unless weighting for respondent characteristics (relative to the target population) is employed. Therefore, if mixed mode (paper or online) options are feasible, they are highly likely to produce more representative results than if only the less costly online option is offered.

UR - http://www.scopus.com/inward/record.url?scp=85062827168&partnerID=8YFLogxK

U2 - 10.2196/10672

DO - 10.2196/10672

M3 - Article

VL - 21

JO - Journal of Medical Internet Research

JF - Journal of Medical Internet Research

SN - 1438-8871

IS - 3

M1 - e10672

ER -