Researchers Adjunct Professor Lisa Hartling, Robin Featherstone, Megan Nuspl, Kassi Shave, Dr Donna M. Dryden and Ben Vandermeer have published a freely accessible paper ‘The contribution of databases to the results of systematic reviews: a cross-sectional study’ under a Creative Commons 4.0 license in BMC Medical Research Methodology. The research team was interested in how many databases should be utilised for a comprehensive literature review.
‘One of the hallmarks of a well-conducted SR is a thorough, objective and reproducible search of a range of sources to identify as many relevant studies as possible, to minimize bias and assist in achieving reliable estimates of effects‘
They reference current guidelines:-
‘Methodological standards for the conduct of new Cochrane interventions require searches of CENTRAL, Medline and Embase, and reviewers are encouraged to consider subject specific databases (e.g. CINAHL for nursing related topics, or PsycINFO for psychological interventions) and regional databases (e.g. LILACS)‘
and identify an important gap on the evidence behind the review process itself
‘One important gap is the modest amount of empirical evidence demonstrating the impact on results and conclusions from different approaches to searching………While methodological guidance for SRs encourages comprehensive searching, there are diminishing returns with each additional database searched ….and the impact of searching each additional database in terms of the final results and conclusions is not known‘.
The research team looked to answer this question by identifying relevant databases and examining published meta-analyses to see if changing the databases accessed would impact on the results:-
‘To determine our set of databases for investigation we sampled 50 reviews conducted by three Cochrane review groups (ARI, DPLP and Airways) and developed a preliminary list of 108 information sources……The remaining information sources were reviewed by our research librarian (RF) and a second author (DMD). We selected the ten databases as those most likely to be searched in SRs of healthcare interventions‘.
Three research areas were selected by the researchers:-
‘The researchers focused on 3 key subject areas for meta-analyses – Acute Respiratory Infections (ARI; n = 57), Infectious Diseases (ID; n = 38), and Development, Psychosocial and Learning Problems (DPLP; n = 34)‘
The researchers found that there were three combinations of databases (Medline + Embase for ARI, Medline + Psychinfo for DPLP and Medline, Embase and BIOSIS for ID) that identified most of the papers in meta-analyses and that exclusion of the other databases did not on the whole significantly change the results:-
‘For ID, yield was highest for Medline (92 %), Embase (81 %), and BIOSIS (67 %). Restricting meta-analyses to trials that appeared in Medline + BIOSIS yielded fewest changes with 1 meta-analysis changing in statistical significance. Point estimates changed in 8 of 31 meta-analyses; change less than 20 % in all cases. For DPLP, identification of relevant studies was highest for Medline (75 %) and Embase (62 %). Restricting meta-analyses to trials that appeared in Medline + PsycINFO resulted in only one change in significance. Point estimates changed for 13 of 33 meta-analyses; less than 20 % in 9 cases‘
The researchers have challenged an intuitive assertion that comprehensive literature reviews should include as many database sources as possible. Given the trade-off between resources and comprehensive searches, this is a valuable addition to the literature and it will be interesting to see how this impacts on future guidelines for meta-analyses and systematic reviews.
Index: There are indices for the TAWOP site here and here
Twitter: You can follow ‘The Amazing World of Psychiatry’ Twitter by clicking on this link.
TAWOP Channel: You can follow the TAWOP Channel on YouTube by clicking on this link.
Responses: If you have any comments, you can leave them below or alternatively e-mail justinmarley17@yahoo.co.uk.
Disclaimer: The comments made here represent the opinions of the author and do not represent the profession or any body/organisation. The comments made here are not meant as a source of medical advice and those seeking medical advice are advised to consult with their own doctor. The author is not responsible for the contents of any external sites that are linked to in this blog.
Conflicts of Interest: *For potential conflicts of interest please see the About section.
