An introduction to conducting a systematic literature review for social scientists and health researchers

An introduction to conducting a systematic
literature review for social scientists and
health researchers
Luke Van Rhoon
Health Behaviour Change Research Group,
School of Psychology, NUIG
4th Nov 2020 10:00am -12:30pm

Key NUI Galway Resources
Session Overview

Learning Outcomes
By the end of this session you should be able to:
• Describe the key features of a systematic review
• Outline the pros and cons of systematic reviews
• Develop a systematic review research question
• List the steps in carrying out a systematic review
• Choose an appropriate method to synthesise results
Quantitative Meta-analysis
Narrative Review
Qualitative Meta-synthesis
What is a systematic
review?
How to conduct a
systematic review?
How to synthesise and
report the results of a
systematic review?
Session Overview

1. A type of literature review
- Google Search
- Papers recommended by colleagues
- Papers discussed in class/seminars
- Snowballing
How do you normally identify literature in
your area?

INTERVIEW QUESTION:
What is the difference between a traditional literature review
and a systematic review?

2. Differs from a traditional literature review
Kysh, Lynn (2013): Difference between a systematic review and a literature review. figshare.
https://doi.org/10.6084/m9.figshare.766364.v1 Retrieved: 13:36, Dec 05, 2017 (GMT)

3. The importance of method
• A systematic review uses explicit and replicable methods to identify,
critically appraise, and synthesize relevant studies
• These methods are reported so that other researchers could use
these methods to identify the same studies

Can you differentiate a systematic from a non-systematic review
just from the abstract?
3. The importance of method

The current paper will review existing evidence on PA rates, correlates,
consequences, and interventions in older adults with MS, and then
highlight important areas for future research on PA and health aging with
MS.

The purpose of this systematic review and meta-analysis is to evaluate
the evidence of the modifiable, individual level psychosocial
constructs associated with PA participation in people with MS..

4. Related but different terms
Meta-Analysis: A method of synthesising the results of the
studies included in a systematic review

4. Related but different terms
Meta-Analysis: A means of synthesising the results of a
systematic review
Cochrane Review: A type of systematic review following the
guidelines of the Cochrane collaboration

Systematic Reviews: Pros and Cons
 Rigorous, unbiased and reliable
 Informative as summarise large body of research
 Aid decision making, policy making, clinical decisions

• Working together to provide the best
evidence for health care
• Archie Cochrane (1909-1988), a British
epidemiologist, who advocated the use of
RCTs to reliably inform healthcare practice
• Cochrane Handbook for Systematic
Reviews of Interventions
• Gold standard

Systematic Reviews: Pros and Cons
 Rigorous, unbiased and reliable
 Informative as summarise large body of research
 Aid decision making, policy making, clinical decisions
 Provide an answer only to one specific question
 Results may be disappointing
“More research needed in the area"

More systematic theory based research is required to understand
the precipitating and perpetuating factors associated with exercise
dependence, as well as effective treatment regimes. 22

Conducting a Systematic Review: Pros and Cons
 Highly cited
 Requirement of professional qualifications
(e.g. Stage 2 Professional Health Psychology Training)
 Required for funding applications
 Develops skills in searching for and working with existing literature
 Useful skill for your CV
 Time-consuming
 Requires a lot of reading

Time to complete and publish review: 67.3 weeks
Number of reviewers: 5
Number of studies found: 27 to 92 020
Funded reviews take longer to complete and publish
and involve more authors
Conclusions: Systematic reviews take much time and
require large amounts of human resources. Existing
computing and informatics technology should be applied to
decrease this time and resource burden.
Top Tip 1: Be Realistic

Rapid reviews
 Rapid reviews – provide timely and cost-effective evidence
 Urgent evidence needed to inform policy (e.g. Ebola crisis,
humanitarian disasters etc)
 Spectrum of methods to expedite reviews:
 Searching:
 Focused, specific topics, specific outcomes
 Limit searches to English
 Limited grey literature searching/author contact
 Screening/extraction/quality appraisal:
 Single reviewer with subset verified by second reviewer
 Synthesis:
 Narrative > meta-analysis
 Limitations acknowledged, cautious conclusions
Tricco et al 2015; Tricco et al 2017

What is a Systematic Review?
Any Questions?

How to Conduct a Systematic Review

Carrying out a Systematic Review: 8 Steps
1. Define review question
2. Develop inclusion/exclusion criteria
3. Plan search terms/strategy
4. Conduct search
5. Apply inclusion/exclusion criteria to assess relevance
6. Review included studies
Extract data and assess risk of bias/quality of studies
7. Synthesise included studies
Meta-analysis/qualitative meta-synthesis/narrative review
8. Write report and publish
How to conduct a
systematic review
systematic review

Systematic Review Protocol
The plan or set of steps to be followed in a systematic review
Includes:
 Rationale for the review
 Research question
 Methods to locate, select, and critically appraise studies
 Methods to collect data from included studies
 Methods to synthesise data from included studies

The plan or set of steps to be followed in a systematic review
Includes:
 Rationale for the review
 Research question
 Methods to locate, select, and critically appraise studies
 Methods to collect data from included studies
 Methods to synthesise data from included studies
Why register a
systematic review
protocol?

Why register a
systematic review
protocol?

1. Define the Review Question
• Conduct scoping review
• Define terms precisely
• Choose a narrow and well specified question
• Specific review criteria:
Population Intervention Comparison Outcome (PICO)
Sample, Phenomenon of Interest, Design, Evaluation, Research type
(SPIDER)

Scope
Technology-driven Type 2 Diabetes Prevention Interventions (DPIs)
Define terms
Technology-driven, Type 2 Diabetes, Behaviour Change Techniques, Digital
Features
Narrow and well specified question
To determine the effectiveness of technology-driven DPIs in producing clinically
significant weight loss and improvements in additional outcomes linked to the
onset of T2D; and
Identify the BCTs and digital features most frequently used in effective
interventions
Specific review criteria
P: Adults (≥18 years) at risk of developing T2D (define criteria)
I: Active Intervention arm
C: Usual Care, non-digital DPI (single-arm studies were included in this review)
O: Weight loss, fasting glucose, glycated haemoglbin (A1c)
36

Scope
Define
terms
Specific
question
Specific
criteria
In groups,
define your own
review question
37

Group Work Systematic Review Questions

1. Define the Review Question
Top Tip 3: Be Specific
Top Tip 4: Know the Literature

2. Develop Inclusion and Exclusion Criteria
List of criteria to set out the papers to be included and excluded

3. Plan Search Strategy
Identifying papers for a systematic review
• Electronic databases
Platforms: EBSCO, OVID (search interface and features)
Databases: CINAHL, EMBASE, Medline, PsycInfo

Name Summary Coverage Comments
CINAHL Cumulative Index to Nursing and
Allied Health Literature
1937 
Nursing, biomedicine, health sciences librarianship,
alternative/complementary medicine, consumer health and other
allied health disciplines.
Health care books, nursing dissertations, selected conference
proceedings, standards of practice, educational software,
audiovisual materials and Evidence-Based Care Sheets
English Language
EMBASE Excerpta Medica dataBASE
Biomedical and pharmacological
database
European focus
1947
Drug research, pharmacology, pharmacy, pharmacoeconomics,
pharmaceutics and toxicology, clinical and experimental human
medicine, basic biological research, health policy and management,
public, occcupational and environmental health, substance
dependence and abuse, psychiatry, forensic science and biomedical
engineering and instrumentation
Selective Coverage: nursing, dentistry, veterinary medicine,
psychology and alternative medicine.
Approx 30%
duplication with
Medline
Medline Biomedical database produced
by the US National Library of
Medicine
US Focus
1946
Biomedicine and life sciences, bioengineering, public health, clinical
care and plant and animal science.
3,000 medical, dental and nursing journals
PsycINFO Psychological database produced
by the American Psychological
Society
1597
Comprehensive coverage from
the 1880s
All areas of psychology: applied, experimental, developmental,
educational, behavioural and animal psychology and psychological
disorders. Relevant items from related disciplines (eg. sociology,
linguistics, education, law, physiology, business, psychiatry and
anthropology) and dissertations are also included
Publications from more than 50 countries and 29 languages

Identifying papers for a systematic review
• Electronic databases
Platforms: EBSCO, OVID (search interface and features)
Databases: CINAHL, EMBASE, Medline, PsycInfo
Meet with Academic Librarian for advice
• Hand search key journals
E.g. Past five years of journals in your area
• Forward and backward citation searches
• Unpublished or “Grey Literature”
E.g. conference abstracts, contact authors in the field

 More comprehensive review
 Reduces risk of publication bias
 Unpublished data difficult to source
 Located studies may be unrepresentative
 Lower quality in unpublished studies
Unpublished studies include or exclude?
File drawer problem:
Published trials showed
a 9% larger intervention
effect than grey trials
(Hopewell et al., 2007)
Support for inclusion of
grey literature in
systematic reviews:
Review Authors: 86%
Journal Editors: 69%
(from Cochrane Handbook)

3. Plan Search Terms
Top Tip 5: Learn from Existing Reviews Top Tip 6: Use Available Expertise

• Use terms that will accurately identify relevant material
• Medical Subject Heading (MeSH terms)
National Library of Medicine's controlled vocabulary thesaurus
• Combine terms; AND, NOT, OR, NEAR, ADJ
• Test out in electronic databases
Modify and refine terms

Downloaded from: https://www.linkedin.com/pulse/boolean-logic-marketers-explained-part-1-
fab-capodicasa

4. Conduct the Search
• 100s or 1000s of hits
• Bibliographic Software to manage references
• Record where references were identified

5. Apply Inclusion and Exclusion Criteria: Screening
Screen
titles
Screen
abstracts
Obtain
full
article
Screen
full
article

5. Apply Inclusion and Exclusion Criteria: Screening
• Keep good records and a paper-trail of inclusion/exclusion and reasons
Create inclusion/exclusion table/bibliographic database folders
Create inclusion/exclusion flow diagram
• Assessment of eligibility of studies should be done by at least two people
independently
Second reviewer for a percentage if all not feasible
Calculate the kappa statistic to measure agreement
0.40 - 0.59 is fair
0.60 - 0.74 is good,
> 0.75 is excellent (Orwin 1994).
Top Tip 7: Work in a Team

Top Tip 8: Be Organised
Articles Included
37
Studies Included
19

59
Would you include
these papers?

6. Review Studies - Extract Study Data
• Systematically record data from each paper
• Record key information relevant to answering your review question
• Create a table to ensure info is accurately and consistently extracted
• Extraction of data should be done by at least two people independently

66
Extract the relevant data
from this paper by Kramer
et al. (2010)

6. Review Studies – Critical Appraisal
• Quality check-list and scales exist with inclusion cut-off scores
e.g. Downs and Black checklist for non randomised studies
• Recent expert groups have discouraged the use of summary
quality scales (The Cochrane Collaboration, 2009)
Due to the difficulties in assigning weights to items
Limited validity of these measures
• Recommend not to exclude studies based on critical appraisal
but provide information for discussion of the confidence in the
results

http://www.casp-uk.net/#!casp-tools-checklists/c18f8

6. Review Studies – Critical Appraisal

How to Conduct a Systematic Review
Any Questions?

How to Synthesise and Report the
Results of
a Systematic Review

Carrying out a Systematic Review: 8 Steps
1. Define review question
2. Develop inclusion/exclusion criteria
3. Plan search terms/strategy
4. Conduct search
5. Apply inclusion/exclusion criteria to assess relevance
6. Review included studies
Extract data and assess risk of bias/quality of studies
7. Synthesise included studies
Meta-analysis/narrative review/qualitative meta-synthesis
8. Write report and publish
systematic review?

7. Synthesise Included Studies
Quantitative Meta-Analysis
• Statistical approach to integrate results across studies by obtaining
an average effect size
• “The analysis of the analyses”
• Often weighted by sample size
Effect sizes based on larger sample sizes receive
greater weight than those from smaller samples
• Also used to:
Estimate the variability between effect sizes across studies
Identify moderator variables
Limited to similar studies:
“Because of their limited number and varied
designs, populations and interventions, meta-
analytic techniques were deemed inappropriate.
Instead, randomized controlled trials are
discussed in a narrative review”.”

References
Field A. & Gillett, R. (2010). How to do a meta-analysis. Br J Math Stat Psychol, 63:
665–694.
Hedges L. & Vevea J. (1998) Fixed-and random-effects models in meta-analysis. Psychol
Methods, 3: 486–504.
Example
Mc Sharry, J., Moss‐Morris, R., & Kendrick, T. (2011). Illness perceptions and glycaemic
control in diabetes: a systematic review with meta‐analysis. Diabetic Medicine, 28(11),
1300-1310.
Cochrane logo

Post-Trial Physical Activity (Self-Report)

Post-Trial Physical Activity (Objective)

Narrative Review
•Summarise the types of studies and patterns of results giving
consideration to risk of bias
•Use headings and sub-headings as appropriate
•Try to explain differences between study findings
•Used for broad research questions when included studies differ

Narrative Review
References
J. Popay, H. Roberts, A. Sowden, M. Petticrew, L. Arai, M. Rodgers et al. (2006).
Guidance on the conduct of narrative synthesis in systematic reviews. University of
Lancaster, UK (2006).
Dixon-Woods , M., Agarwal, S., Jones, D., Young, B. & Sutton, A. (2005). Synthesising
qualitative and quantitative evidence: a review of possible methods. J Health Serv Res
Policy 2005, 10:45-53.
Example
Hynes, L., Byrne, M., Dinneen, S. F., McGuire, B. E., O'Donnell, M., & Mc Sharry, J.
(2014). Barriers and facilitators associated with attendance at hospital diabetes clinics
among young adults (15–30 years) with type 1 diabetes mellitus: a systematic review.
Pediatric diabetes.

Narrative Review

Techniques for narrative synthesis

Setting Facilitators No. of
Studies
Barriers No. of
Studies
Transition Transition programme 4 Lack of collaboration between
paediatric and adult clinics
6
Support and information during
transition
5 Unsupported transition 6
Adult diabetes clinic Continuity of staff in adult clinic 4 Challenges to relationship
development
3
Valued clinic characteristics and
resources
4 Low perceived value of attendance 2
Clear procedure for appointment
making & breaking
1 Difficulties communicating with
service
4
Conflicting schedules 3
Facilitators & Barriers

8. Write Report and Publish
• PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses)
Provides a guide the reporting key methodology information
• Summarise results and answer your review question
• Implications: Further research, clinical practice, policy
• Limitations: Of the primary studies and of the review methodology.

8. Write Report and Publish
• Identify relevant journal
Contact editor if unsure
Some journals only accept invited reviews
• Tailor write-up to specific journal and target audience
• Check word-count and maximum number of references

How to Synthesise and Report the
Results of a Systematic Review
Any Questions?

https://nuighealthpsyc
hology.wordpress.com/
2014/10/
Top Tip 1: Be Realistic
Top Tip 2: Register Your Protocol
Top Tip 8: Be Organised
Top Tip 5: Learn from Existing
Reviews
Top Tip 3: Be Specific
Top Tip 4: Know the Literature
TOP TIPS
Top Tip 6: Use Available Expertise
Top Tip 7: Work in a Team

Systematic Review Methods:
•Higgins, J. P. T. (2011). Green S. Cochrane handbook for systematic reviews of interventions version 5.1.
0. The Cochrane Collaboration, 5(0).
•Moher, D., Liberati, A., Tetzlaff, J., & Altman, D. G. (2009). Preferred reporting items for systematic
reviews and meta-analyses: the PRISMA statement. Annals of Internal Medicine, 151(4), 264-269.
•Hopewell, S., McDonald, S., Clarke, M., & Egger, M. (2007). Grey literature in meta-analyses of
randomized trials of health care interventions. Cochrane Database Syst Rev, 2(2)
•Katrak, P., Bialocerkowski, A. E., Massy-Westropp, N., Kumar, V. S., & Grimmer, K. A. (2004). A systematic
review of the content of critical appraisal tools. BMC Medical Research Methodology, 4(1), 22.
•Shamseer, L., Moher, D., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., ... & Stewart, L. A. (2015).
Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration
and explanation. BMJ,349, g7647.
•Tricco et al 2015. A scoping review of rapid review methods, BMC Medicine, 13: 224.
•Tricco et al 2017.Rapid reviews to strengthen health policy and systems: a practical guide. 2017.
Sample Reviews:
•Mc Sharry, J., Moss‐Morris, R., & Kendrick, T. (2011). Illness perceptions and glycaemic control in
diabetes: a systematic review with meta‐analysis. Diabetic Medicine, 28(11), 1300-1310.
•Hynes, L., Byrne, M., Dinneen, S. F., McGuire, B. E., O'Donnell, M., & Mc Sharry, J. (2014). Barriers and
facilitators associated with attendance at hospital diabetes clinics among young adults (15–30 years) with
type 1 diabetes mellitus: a systematic review. Pediatric Diabetes.
•McSharry, J., McGowan, L., Farmer, A. J., & French, D. P. (2016). Perceptions and experiences of taking
oral medications for the treatment of Type 2 diabetes mellitus: a systematic review and meta‐synthesis of
References

Synthesis Methods:
• Popay, H. Roberts, A. Sowden, M. Petticrew, L. Arai, M. Rodgers et al. (2006). Guidance on the conduct
of narrative synthesis in systematic reviews. University of Lancaster, UK (2006).
• Dixon-Woods , M., Agarwal, S., Jones, D., Young, B. & Sutton, A. (2005). Synthesising qualitative and
quantitative evidence: a review of possible methods. J Health Serv Res Policy 10, 45-53.
• Noblit, G.W., Hare, R.D. (1988). Meta-ethnography: Synthesizing Qualitative Studies. Newbury Park:
Sage Publications; 1988.
• Booth, A., Noyes, J., Fleming, K., Gerhardus, A., Wahlster, P., Van Der Wilt, G.J., Moztgemba, K., Refolo,
P., Sacchini, D., Tummers, M., Rehfuess, E. (2016) Guidance on choosing qualitative evidence synthesis
methods for use in health technology assessments of complex interventions [Online]. Available from:
http://www.integrate-hta.eu/downloads/
• Tong, A., Flemming, K., McInnes, E., Oliver, S., & Craig, J. (2012). Enhancing transparency in reporting
the synthesis of qualitative research: ENTREQ. BMC medical research methodology, 12(1), 181.
• Field A. & Gillett, R. (2010). How to do a meta-analysis. Br J Math Stat Psychol, 63, 665–694.
• Hedges L. & Vevea J. (1998) Fixed-and random-effects models in meta-analysis. Psychol Methods, 3,
486–504.
• http://libguides.library.nuigalway.ie/systematicreviews
• Cochrane Collaboration Website: http://www.cochrane.org/
• PROSPERO: http://www.prospero.com/
• Andy Field: A Bluffer’s Guide to Meta-analysis: http://www.statisticshell.com/docs/meta.pdf
• http://nuighealthpsychology.wordpress.com/2014/10/13/5-tips-on-conducting-a-systematic-review/
References

An introduction to conducting a systematic literature review for social scientists and health researchers

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to An introduction to conducting a systematic literature review for social scientists and health researchers

Similar to An introduction to conducting a systematic literature review for social scientists and health researchers (20)

More from rosie.dunne

More from rosie.dunne (9)

Recently uploaded

Recently uploaded (20)

An introduction to conducting a systematic literature review for social scientists and health researchers