The Polyglot Search Translator (PST): evaluation of a tool for improving searching in systematic reviews: a randomised cross-over trial

Justin Clark, Matthew Carter, David Honeyman, Gina Cleo, Yvonne Auld, D Booth, P Condron, C Dalais, Sarah Dern, Bronwyn Linthwaite, N May, L Ramsay, K Rickett, C Rutter, Sharon L Sanders, P Sondergeld, M Wallin, Elaine M Beller

Research output: Contribution to conferencePresentationResearchpeer-review

Abstract

Background:

To improve the speed and quality of literature searching for systematic reviews (SRs) we developed an online tool, the Polyglot Search Translator (PST). This tool helps to automatically translate searches across multiple databases by modifying the database-specific syntax, removing the need to do this manually.

Objectives:

To determine whether the PST improves the speed and quality of database searching for SRs.

Methods:

From a pool of 20 randomly selected database search strategies from reviews of drug and non-drug interventions, diagnostic tests, prevalence, prognosis and health technology assessments, each of 20 participants were randomly assigned 10 searches and randomly allocated to translate the search manually or with the PST. The participants were asked to translate each search into three databases from the following: PubMed or Ovid MEDLINE, Embase.com or Ovid Embase, the Cochrane Library, CINAHL, Web of Science or Scopus. We measured:

1) the time taken to translate the search strategies using manual methods or the PST (these were compared against each other);

2) the number of search results found by the participants, compared to the number found by a reference standard developed through consensus by two Cochrane information specialists;

3) errors found in the participants' searches, identified through consensus by the two Cochrane information specialists (manual errors will be compared to PST errors).

Results:

Preliminary results show that when compared with a manual method, using the PST to translate a search saved an average of 10 minutes per database; this increased to over 100 minutes per database on the most complex search. As more data become available, a more accurate representation of the time saving is expected. The number of search results found and errors in the searches are still being analysed.

Conclusions:

Initial data show that the PST saves time when searching for studies for inclusion in a SR. Analysis has not yet begun of the quality of search translations; these data will be presented.

Patient or healthcare consumer involvement:

Although not directly involving patients or health care consumers, the PST could open pathways for less experienced individuals to be involved in a SR and speed up SR production, with the benefit to consumers of having access to more up-to-date SRs.
Original languageEnglish
Publication statusPublished - Sep 2018
EventThe 25th Cochrane Colloquium - Edinburgh, United Kingdom
Duration: 16 Sep 201818 Sep 2018
Conference number: 25th

Conference

ConferenceThe 25th Cochrane Colloquium
CountryUnited Kingdom
CityEdinburgh
Period16/09/1818/09/18
OtherThe Cochrane Colloquium is an annual event, bringing people together from around the world to discuss putting research into important global health questions and promoting evidence-informed health care. This year the theme is ‘Cochrane for all - better evidence for better health decisions’. It was a Patients Included event, co-designed, co-produced and co-presented by healthcare consumers, where everyone’s input is valued equally. It focused on key goals of Cochrane’s Strategy to 2020: producing evidence, making evidence accessible, and advocating for evidence.

Fingerprint

Cross-Over Studies
Databases
Information Services
Delivery of Health Care
Biomedical Technology Assessment
Routine Diagnostic Tests
PubMed
MEDLINE
Libraries
Patient Care
Pharmaceutical Preparations

Cite this

Clark, J., Carter, M., Honeyman, D., Cleo, G., Auld, Y., Booth, D., ... Beller, E. M. (2018). The Polyglot Search Translator (PST): evaluation of a tool for improving searching in systematic reviews: a randomised cross-over trial. The 25th Cochrane Colloquium, Edinburgh, United Kingdom.
Clark, Justin ; Carter, Matthew ; Honeyman, David ; Cleo, Gina ; Auld, Yvonne ; Booth, D ; Condron, P ; Dalais, C ; Dern, Sarah ; Linthwaite, Bronwyn ; May, N ; Ramsay, L ; Rickett, K ; Rutter, C ; Sanders, Sharon L ; Sondergeld, P ; Wallin, M ; Beller, Elaine M. / The Polyglot Search Translator (PST): evaluation of a tool for improving searching in systematic reviews: a randomised cross-over trial. The 25th Cochrane Colloquium, Edinburgh, United Kingdom.
@conference{bf2e1592439040979fa70b9294c39d7c,
title = "The Polyglot Search Translator (PST): evaluation of a tool for improving searching in systematic reviews: a randomised cross-over trial",
abstract = "Background:To improve the speed and quality of literature searching for systematic reviews (SRs) we developed an online tool, the Polyglot Search Translator (PST). This tool helps to automatically translate searches across multiple databases by modifying the database-specific syntax, removing the need to do this manually.Objectives:To determine whether the PST improves the speed and quality of database searching for SRs.Methods:From a pool of 20 randomly selected database search strategies from reviews of drug and non-drug interventions, diagnostic tests, prevalence, prognosis and health technology assessments, each of 20 participants were randomly assigned 10 searches and randomly allocated to translate the search manually or with the PST. The participants were asked to translate each search into three databases from the following: PubMed or Ovid MEDLINE, Embase.com or Ovid Embase, the Cochrane Library, CINAHL, Web of Science or Scopus. We measured:1) the time taken to translate the search strategies using manual methods or the PST (these were compared against each other);2) the number of search results found by the participants, compared to the number found by a reference standard developed through consensus by two Cochrane information specialists;3) errors found in the participants' searches, identified through consensus by the two Cochrane information specialists (manual errors will be compared to PST errors).Results:Preliminary results show that when compared with a manual method, using the PST to translate a search saved an average of 10 minutes per database; this increased to over 100 minutes per database on the most complex search. As more data become available, a more accurate representation of the time saving is expected. The number of search results found and errors in the searches are still being analysed.Conclusions:Initial data show that the PST saves time when searching for studies for inclusion in a SR. Analysis has not yet begun of the quality of search translations; these data will be presented.Patient or healthcare consumer involvement:Although not directly involving patients or health care consumers, the PST could open pathways for less experienced individuals to be involved in a SR and speed up SR production, with the benefit to consumers of having access to more up-to-date SRs.",
author = "Justin Clark and Matthew Carter and David Honeyman and Gina Cleo and Yvonne Auld and D Booth and P Condron and C Dalais and Sarah Dern and Bronwyn Linthwaite and N May and L Ramsay and K Rickett and C Rutter and Sanders, {Sharon L} and P Sondergeld and M Wallin and Beller, {Elaine M}",
year = "2018",
month = "9",
language = "English",
note = "The 25th Cochrane Colloquium ; Conference date: 16-09-2018 Through 18-09-2018",

}

Clark, J, Carter, M, Honeyman, D, Cleo, G, Auld, Y, Booth, D, Condron, P, Dalais, C, Dern, S, Linthwaite, B, May, N, Ramsay, L, Rickett, K, Rutter, C, Sanders, SL, Sondergeld, P, Wallin, M & Beller, EM 2018, 'The Polyglot Search Translator (PST): evaluation of a tool for improving searching in systematic reviews: a randomised cross-over trial' The 25th Cochrane Colloquium, Edinburgh, United Kingdom, 16/09/18 - 18/09/18, .

The Polyglot Search Translator (PST): evaluation of a tool for improving searching in systematic reviews: a randomised cross-over trial. / Clark, Justin; Carter, Matthew; Honeyman, David; Cleo, Gina; Auld, Yvonne; Booth, D; Condron, P; Dalais, C; Dern, Sarah; Linthwaite, Bronwyn; May, N; Ramsay, L; Rickett, K; Rutter, C; Sanders, Sharon L; Sondergeld, P; Wallin, M; Beller, Elaine M.

2018. The 25th Cochrane Colloquium, Edinburgh, United Kingdom.

Research output: Contribution to conferencePresentationResearchpeer-review

TY - CONF

T1 - The Polyglot Search Translator (PST): evaluation of a tool for improving searching in systematic reviews: a randomised cross-over trial

AU - Clark, Justin

AU - Carter, Matthew

AU - Honeyman, David

AU - Cleo, Gina

AU - Auld, Yvonne

AU - Booth, D

AU - Condron, P

AU - Dalais, C

AU - Dern, Sarah

AU - Linthwaite, Bronwyn

AU - May, N

AU - Ramsay, L

AU - Rickett, K

AU - Rutter, C

AU - Sanders, Sharon L

AU - Sondergeld, P

AU - Wallin, M

AU - Beller, Elaine M

PY - 2018/9

Y1 - 2018/9

N2 - Background:To improve the speed and quality of literature searching for systematic reviews (SRs) we developed an online tool, the Polyglot Search Translator (PST). This tool helps to automatically translate searches across multiple databases by modifying the database-specific syntax, removing the need to do this manually.Objectives:To determine whether the PST improves the speed and quality of database searching for SRs.Methods:From a pool of 20 randomly selected database search strategies from reviews of drug and non-drug interventions, diagnostic tests, prevalence, prognosis and health technology assessments, each of 20 participants were randomly assigned 10 searches and randomly allocated to translate the search manually or with the PST. The participants were asked to translate each search into three databases from the following: PubMed or Ovid MEDLINE, Embase.com or Ovid Embase, the Cochrane Library, CINAHL, Web of Science or Scopus. We measured:1) the time taken to translate the search strategies using manual methods or the PST (these were compared against each other);2) the number of search results found by the participants, compared to the number found by a reference standard developed through consensus by two Cochrane information specialists;3) errors found in the participants' searches, identified through consensus by the two Cochrane information specialists (manual errors will be compared to PST errors).Results:Preliminary results show that when compared with a manual method, using the PST to translate a search saved an average of 10 minutes per database; this increased to over 100 minutes per database on the most complex search. As more data become available, a more accurate representation of the time saving is expected. The number of search results found and errors in the searches are still being analysed.Conclusions:Initial data show that the PST saves time when searching for studies for inclusion in a SR. Analysis has not yet begun of the quality of search translations; these data will be presented.Patient or healthcare consumer involvement:Although not directly involving patients or health care consumers, the PST could open pathways for less experienced individuals to be involved in a SR and speed up SR production, with the benefit to consumers of having access to more up-to-date SRs.

AB - Background:To improve the speed and quality of literature searching for systematic reviews (SRs) we developed an online tool, the Polyglot Search Translator (PST). This tool helps to automatically translate searches across multiple databases by modifying the database-specific syntax, removing the need to do this manually.Objectives:To determine whether the PST improves the speed and quality of database searching for SRs.Methods:From a pool of 20 randomly selected database search strategies from reviews of drug and non-drug interventions, diagnostic tests, prevalence, prognosis and health technology assessments, each of 20 participants were randomly assigned 10 searches and randomly allocated to translate the search manually or with the PST. The participants were asked to translate each search into three databases from the following: PubMed or Ovid MEDLINE, Embase.com or Ovid Embase, the Cochrane Library, CINAHL, Web of Science or Scopus. We measured:1) the time taken to translate the search strategies using manual methods or the PST (these were compared against each other);2) the number of search results found by the participants, compared to the number found by a reference standard developed through consensus by two Cochrane information specialists;3) errors found in the participants' searches, identified through consensus by the two Cochrane information specialists (manual errors will be compared to PST errors).Results:Preliminary results show that when compared with a manual method, using the PST to translate a search saved an average of 10 minutes per database; this increased to over 100 minutes per database on the most complex search. As more data become available, a more accurate representation of the time saving is expected. The number of search results found and errors in the searches are still being analysed.Conclusions:Initial data show that the PST saves time when searching for studies for inclusion in a SR. Analysis has not yet begun of the quality of search translations; these data will be presented.Patient or healthcare consumer involvement:Although not directly involving patients or health care consumers, the PST could open pathways for less experienced individuals to be involved in a SR and speed up SR production, with the benefit to consumers of having access to more up-to-date SRs.

M3 - Presentation

ER -

Clark J, Carter M, Honeyman D, Cleo G, Auld Y, Booth D et al. The Polyglot Search Translator (PST): evaluation of a tool for improving searching in systematic reviews: a randomised cross-over trial. 2018. The 25th Cochrane Colloquium, Edinburgh, United Kingdom.