Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

An introduction to conducting a systematic literature review for social scientists and health researchers

An introduction to conducting a systematic literature review for social scientists and health researchers presented by Luke van Rhoon Health Behaviour Change Research Group, School of Psychology, NUI Galway November 2020

Related Audiobooks

Free with a 30 day trial from Scribd

See all
  • Be the first to comment

An introduction to conducting a systematic literature review for social scientists and health researchers

  1. 1. An introduction to conducting a systematic literature review for social scientists and health researchers Luke Van Rhoon Health Behaviour Change Research Group, School of Psychology, NUIG 4th Nov 2020 10:00am -12:30pm
  2. 2. Key NUI Galway Resources Session Overview
  3. 3. Learning Outcomes By the end of this session you should be able to: • Describe the key features of a systematic review • Outline the pros and cons of systematic reviews • Develop a systematic review research question • List the steps in carrying out a systematic review • Choose an appropriate method to synthesise results Quantitative Meta-analysis Narrative Review Qualitative Meta-synthesis What is a systematic review? How to conduct a systematic review? How to synthesise and report the results of a systematic review? Session Overview
  4. 4. What is a Systematic Review?
  5. 5. 1. A type of literature review - Google Search - Papers recommended by colleagues - Papers discussed in class/seminars - Snowballing How do you normally identify literature in your area?
  6. 6. INTERVIEW QUESTION: What is the difference between a traditional literature review and a systematic review?
  7. 7. 2. Differs from a traditional literature review Kysh, Lynn (2013): Difference between a systematic review and a literature review. figshare. https://doi.org/10.6084/m9.figshare.766364.v1 Retrieved: 13:36, Dec 05, 2017 (GMT)
  8. 8. 3. The importance of method • A systematic review uses explicit and replicable methods to identify, critically appraise, and synthesize relevant studies • These methods are reported so that other researchers could use these methods to identify the same studies
  9. 9. Can you differentiate a systematic from a non-systematic review just from the abstract? 3. The importance of method
  10. 10. The current paper will review existing evidence on PA rates, correlates, consequences, and interventions in older adults with MS, and then highlight important areas for future research on PA and health aging with MS.
  11. 11. The purpose of this systematic review and meta-analysis is to evaluate the evidence of the modifiable, individual level psychosocial constructs associated with PA participation in people with MS..
  12. 12. 4. Related but different terms Meta-Analysis: A method of synthesising the results of the studies included in a systematic review
  13. 13. 4. Related but different terms Meta-Analysis: A means of synthesising the results of a systematic review Cochrane Review: A type of systematic review following the guidelines of the Cochrane collaboration
  14. 14. Systematic Reviews: Pros and Cons  Rigorous, unbiased and reliable  Informative as summarise large body of research  Aid decision making, policy making, clinical decisions
  15. 15. • Working together to provide the best evidence for health care • Archie Cochrane (1909-1988), a British epidemiologist, who advocated the use of RCTs to reliably inform healthcare practice • Cochrane Handbook for Systematic Reviews of Interventions • Gold standard
  16. 16. Systematic Reviews: Pros and Cons  Rigorous, unbiased and reliable  Informative as summarise large body of research  Aid decision making, policy making, clinical decisions  Provide an answer only to one specific question  Results may be disappointing “More research needed in the area"
  17. 17. More systematic theory based research is required to understand the precipitating and perpetuating factors associated with exercise dependence, as well as effective treatment regimes. 22
  18. 18. Conducting a Systematic Review: Pros and Cons  Highly cited  Requirement of professional qualifications (e.g. Stage 2 Professional Health Psychology Training)  Required for funding applications  Develops skills in searching for and working with existing literature  Useful skill for your CV  Time-consuming  Requires a lot of reading
  19. 19. Time to complete and publish review: 67.3 weeks Number of reviewers: 5 Number of studies found: 27 to 92 020 Funded reviews take longer to complete and publish and involve more authors Conclusions: Systematic reviews take much time and require large amounts of human resources. Existing computing and informatics technology should be applied to decrease this time and resource burden. Top Tip 1: Be Realistic
  20. 20. Rapid reviews  Rapid reviews – provide timely and cost-effective evidence  Urgent evidence needed to inform policy (e.g. Ebola crisis, humanitarian disasters etc)  Spectrum of methods to expedite reviews:  Searching:  Focused, specific topics, specific outcomes  Limit searches to English  Limited grey literature searching/author contact  Screening/extraction/quality appraisal:  Single reviewer with subset verified by second reviewer  Synthesis:  Narrative > meta-analysis  Limitations acknowledged, cautious conclusions Tricco et al 2015; Tricco et al 2017
  21. 21. What is a Systematic Review? Any Questions?
  22. 22. How to Conduct a Systematic Review
  23. 23. Carrying out a Systematic Review: 8 Steps 1. Define review question 2. Develop inclusion/exclusion criteria 3. Plan search terms/strategy 4. Conduct search 5. Apply inclusion/exclusion criteria to assess relevance 6. Review included studies Extract data and assess risk of bias/quality of studies 7. Synthesise included studies Meta-analysis/qualitative meta-synthesis/narrative review 8. Write report and publish How to conduct a systematic review How to synthesise and report the results of a systematic review
  24. 24. Systematic Review Protocol The plan or set of steps to be followed in a systematic review Includes:  Rationale for the review  Research question  Methods to locate, select, and critically appraise studies  Methods to collect data from included studies  Methods to synthesise data from included studies
  25. 25. Systematic Review Protocol The plan or set of steps to be followed in a systematic review Includes:  Rationale for the review  Research question  Methods to locate, select, and critically appraise studies  Methods to collect data from included studies  Methods to synthesise data from included studies Why register a systematic review protocol?
  26. 26. Systematic Review Protocol Why register a systematic review protocol?
  27. 27. 34 AIMS SAMPLE REVIEW
  28. 28. 1. Define the Review Question • Conduct scoping review • Define terms precisely • Choose a narrow and well specified question • Specific review criteria: Population Intervention Comparison Outcome (PICO) Sample, Phenomenon of Interest, Design, Evaluation, Research type (SPIDER)
  29. 29. Scope Technology-driven Type 2 Diabetes Prevention Interventions (DPIs) Define terms Technology-driven, Type 2 Diabetes, Behaviour Change Techniques, Digital Features Narrow and well specified question To determine the effectiveness of technology-driven DPIs in producing clinically significant weight loss and improvements in additional outcomes linked to the onset of T2D; and Identify the BCTs and digital features most frequently used in effective interventions Specific review criteria P: Adults (≥18 years) at risk of developing T2D (define criteria) I: Active Intervention arm C: Usual Care, non-digital DPI (single-arm studies were included in this review) O: Weight loss, fasting glucose, glycated haemoglbin (A1c) 36
  30. 30. Scope Define terms Specific question Specific criteria In groups, define your own review question 37
  31. 31. Group Work Systematic Review Questions
  32. 32. 1. Define the Review Question Top Tip 3: Be Specific Top Tip 4: Know the Literature
  33. 33. 2. Develop Inclusion and Exclusion Criteria List of criteria to set out the papers to be included and excluded
  34. 34. 3. Plan Search Strategy Identifying papers for a systematic review • Electronic databases Platforms: EBSCO, OVID (search interface and features) Databases: CINAHL, EMBASE, Medline, PsycInfo
  35. 35. Name Summary Coverage Comments CINAHL Cumulative Index to Nursing and Allied Health Literature 1937  Nursing, biomedicine, health sciences librarianship, alternative/complementary medicine, consumer health and other allied health disciplines. Health care books, nursing dissertations, selected conference proceedings, standards of practice, educational software, audiovisual materials and Evidence-Based Care Sheets English Language EMBASE Excerpta Medica dataBASE Biomedical and pharmacological database European focus 1947 Drug research, pharmacology, pharmacy, pharmacoeconomics, pharmaceutics and toxicology, clinical and experimental human medicine, basic biological research, health policy and management, public, occcupational and environmental health, substance dependence and abuse, psychiatry, forensic science and biomedical engineering and instrumentation Selective Coverage: nursing, dentistry, veterinary medicine, psychology and alternative medicine. Approx 30% duplication with Medline Medline Biomedical database produced by the US National Library of Medicine US Focus 1946 Biomedicine and life sciences, bioengineering, public health, clinical care and plant and animal science. 3,000 medical, dental and nursing journals PsycINFO Psychological database produced by the American Psychological Society 1597 Comprehensive coverage from the 1880s All areas of psychology: applied, experimental, developmental, educational, behavioural and animal psychology and psychological disorders. Relevant items from related disciplines (eg. sociology, linguistics, education, law, physiology, business, psychiatry and anthropology) and dissertations are also included Publications from more than 50 countries and 29 languages
  36. 36. 3. Plan Search Strategy Identifying papers for a systematic review • Electronic databases Platforms: EBSCO, OVID (search interface and features) Databases: CINAHL, EMBASE, Medline, PsycInfo Meet with Academic Librarian for advice • Hand search key journals E.g. Past five years of journals in your area • Forward and backward citation searches • Unpublished or “Grey Literature” E.g. conference abstracts, contact authors in the field
  37. 37. 3. Plan Search Strategy  More comprehensive review  Reduces risk of publication bias  Unpublished data difficult to source  Located studies may be unrepresentative  Lower quality in unpublished studies Unpublished studies include or exclude? File drawer problem: Published trials showed a 9% larger intervention effect than grey trials (Hopewell et al., 2007) Support for inclusion of grey literature in systematic reviews: Review Authors: 86% Journal Editors: 69% (from Cochrane Handbook)
  38. 38. 3. Plan Search Strategy
  39. 39. 3. Plan Search Terms Top Tip 5: Learn from Existing Reviews Top Tip 6: Use Available Expertise
  40. 40. 3. Plan Search Terms • Use terms that will accurately identify relevant material • Medical Subject Heading (MeSH terms) National Library of Medicine's controlled vocabulary thesaurus • Combine terms; AND, NOT, OR, NEAR, ADJ • Test out in electronic databases Modify and refine terms
  41. 41. 3. Plan Search Terms Downloaded from: https://www.linkedin.com/pulse/boolean-logic-marketers-explained-part-1- fab-capodicasa
  42. 42. 4. Conduct the Search • 100s or 1000s of hits • Bibliographic Software to manage references • Record where references were identified
  43. 43. 5. Apply Inclusion and Exclusion Criteria: Screening Screen titles Screen abstracts Obtain full article Screen full article
  44. 44. 5. Apply Inclusion and Exclusion Criteria: Screening • Keep good records and a paper-trail of inclusion/exclusion and reasons Create inclusion/exclusion table/bibliographic database folders Create inclusion/exclusion flow diagram • Assessment of eligibility of studies should be done by at least two people independently Second reviewer for a percentage if all not feasible Calculate the kappa statistic to measure agreement 0.40 - 0.59 is fair 0.60 - 0.74 is good, > 0.75 is excellent (Orwin 1994). Top Tip 7: Work in a Team
  45. 45. Top Tip 8: Be Organised Articles Included 37 Studies Included 19
  46. 46. 59 Would you include these papers?
  47. 47. 6. Review Studies - Extract Study Data • Systematically record data from each paper • Record key information relevant to answering your review question • Create a table to ensure info is accurately and consistently extracted • Extraction of data should be done by at least two people independently
  48. 48. 63
  49. 49. 64
  50. 50. 65
  51. 51. 66 Extract the relevant data from this paper by Kramer et al. (2010)
  52. 52. 6. Review Studies – Critical Appraisal • Quality check-list and scales exist with inclusion cut-off scores e.g. Downs and Black checklist for non randomised studies • Recent expert groups have discouraged the use of summary quality scales (The Cochrane Collaboration, 2009) Due to the difficulties in assigning weights to items Limited validity of these measures • Recommend not to exclude studies based on critical appraisal but provide information for discussion of the confidence in the results
  53. 53. 68
  54. 54. http://www.casp-uk.net/#!casp-tools-checklists/c18f8
  55. 55. 6. Review Studies – Critical Appraisal
  56. 56. 6. Review Studies – Critical Appraisal
  57. 57. How to Conduct a Systematic Review Any Questions?
  58. 58. How to Synthesise and Report the Results of a Systematic Review
  59. 59. Carrying out a Systematic Review: 8 Steps 1. Define review question 2. Develop inclusion/exclusion criteria 3. Plan search terms/strategy 4. Conduct search 5. Apply inclusion/exclusion criteria to assess relevance 6. Review included studies Extract data and assess risk of bias/quality of studies 7. Synthesise included studies Meta-analysis/narrative review/qualitative meta-synthesis 8. Write report and publish How to synthesise and report the results of a systematic review?
  60. 60. 7. Synthesise Included Studies Quantitative Meta-Analysis • Statistical approach to integrate results across studies by obtaining an average effect size • “The analysis of the analyses” • Often weighted by sample size Effect sizes based on larger sample sizes receive greater weight than those from smaller samples • Also used to: Estimate the variability between effect sizes across studies Identify moderator variables Limited to similar studies: “Because of their limited number and varied designs, populations and interventions, meta- analytic techniques were deemed inappropriate. Instead, randomized controlled trials are discussed in a narrative review”.”
  61. 61. 7. Synthesise Included Studies Quantitative Meta-Analysis References Field A. & Gillett, R. (2010). How to do a meta-analysis. Br J Math Stat Psychol, 63: 665–694. Hedges L. & Vevea J. (1998) Fixed-and random-effects models in meta-analysis. Psychol Methods, 3: 486–504. Example Mc Sharry, J., Moss‐Morris, R., & Kendrick, T. (2011). Illness perceptions and glycaemic control in diabetes: a systematic review with meta‐analysis. Diabetic Medicine, 28(11), 1300-1310. Cochrane logo
  62. 62. 79
  63. 63. Quantitative Meta-Analysis Post-Trial Physical Activity (Self-Report)
  64. 64. Post-Trial Physical Activity (Objective)
  65. 65. 7. Synthesise Included Studies Narrative Review •Summarise the types of studies and patterns of results giving consideration to risk of bias •Use headings and sub-headings as appropriate •Try to explain differences between study findings •Used for broad research questions when included studies differ
  66. 66. 7. Synthesise Included Studies Narrative Review References J. Popay, H. Roberts, A. Sowden, M. Petticrew, L. Arai, M. Rodgers et al. (2006). Guidance on the conduct of narrative synthesis in systematic reviews. University of Lancaster, UK (2006). Dixon-Woods , M., Agarwal, S., Jones, D., Young, B. & Sutton, A. (2005). Synthesising qualitative and quantitative evidence: a review of possible methods. J Health Serv Res Policy 2005, 10:45-53. Example Hynes, L., Byrne, M., Dinneen, S. F., McGuire, B. E., O'Donnell, M., & Mc Sharry, J. (2014). Barriers and facilitators associated with attendance at hospital diabetes clinics among young adults (15–30 years) with type 1 diabetes mellitus: a systematic review. Pediatric diabetes.
  67. 67. 84
  68. 68. 7. Synthesise Included Studies Narrative Review
  69. 69. Techniques for narrative synthesis
  70. 70. Setting Facilitators No. of Studies Barriers No. of Studies Transition Transition programme 4 Lack of collaboration between paediatric and adult clinics 6 Support and information during transition 5 Unsupported transition 6 Adult diabetes clinic Continuity of staff in adult clinic 4 Challenges to relationship development 3 Valued clinic characteristics and resources 4 Low perceived value of attendance 2 Clear procedure for appointment making & breaking 1 Difficulties communicating with service 4 Conflicting schedules 3 Facilitators & Barriers
  71. 71. 8. Write Report and Publish • PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses) Provides a guide the reporting key methodology information • Summarise results and answer your review question • Implications: Further research, clinical practice, policy • Limitations: Of the primary studies and of the review methodology.
  72. 72. 8. Write Report and Publish • Identify relevant journal Contact editor if unsure Some journals only accept invited reviews • Tailor write-up to specific journal and target audience • Check word-count and maximum number of references
  73. 73. How to Synthesise and Report the Results of a Systematic Review Any Questions?
  74. 74. Learning Outcomes By the end of this session you should be able to: • Describe the key features of a systematic review • Outline the pros and cons of systematic reviews • Develop a systematic review research question • List the steps in carrying out a systematic review • Choose an appropriate method to synthesise results Quantitative Meta-analysis Narrative Review Qualitative Meta-synthesis What is a systematic review? How to conduct a systematic review? How to synthesise and report the results of a systematic review? Session Overview
  75. 75. https://nuighealthpsyc hology.wordpress.com/ 2014/10/ Top Tip 1: Be Realistic Top Tip 2: Register Your Protocol Top Tip 8: Be Organised Top Tip 5: Learn from Existing Reviews Top Tip 3: Be Specific Top Tip 4: Know the Literature TOP TIPS Top Tip 6: Use Available Expertise Top Tip 7: Work in a Team
  76. 76. Systematic Review Methods: •Higgins, J. P. T. (2011). Green S. Cochrane handbook for systematic reviews of interventions version 5.1. 0. The Cochrane Collaboration, 5(0). •Moher, D., Liberati, A., Tetzlaff, J., & Altman, D. G. (2009). Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Annals of Internal Medicine, 151(4), 264-269. •Hopewell, S., McDonald, S., Clarke, M., & Egger, M. (2007). Grey literature in meta-analyses of randomized trials of health care interventions. Cochrane Database Syst Rev, 2(2) •Katrak, P., Bialocerkowski, A. E., Massy-Westropp, N., Kumar, V. S., & Grimmer, K. A. (2004). A systematic review of the content of critical appraisal tools. BMC Medical Research Methodology, 4(1), 22. •Shamseer, L., Moher, D., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., ... & Stewart, L. A. (2015). Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. BMJ,349, g7647. •Tricco et al 2015. A scoping review of rapid review methods, BMC Medicine, 13: 224. •Tricco et al 2017.Rapid reviews to strengthen health policy and systems: a practical guide. 2017. Sample Reviews: •Mc Sharry, J., Moss‐Morris, R., & Kendrick, T. (2011). Illness perceptions and glycaemic control in diabetes: a systematic review with meta‐analysis. Diabetic Medicine, 28(11), 1300-1310. •Hynes, L., Byrne, M., Dinneen, S. F., McGuire, B. E., O'Donnell, M., & Mc Sharry, J. (2014). Barriers and facilitators associated with attendance at hospital diabetes clinics among young adults (15–30 years) with type 1 diabetes mellitus: a systematic review. Pediatric Diabetes. •McSharry, J., McGowan, L., Farmer, A. J., & French, D. P. (2016). Perceptions and experiences of taking oral medications for the treatment of Type 2 diabetes mellitus: a systematic review and meta‐synthesis of References
  77. 77. Synthesis Methods: • Popay, H. Roberts, A. Sowden, M. Petticrew, L. Arai, M. Rodgers et al. (2006). Guidance on the conduct of narrative synthesis in systematic reviews. University of Lancaster, UK (2006). • Dixon-Woods , M., Agarwal, S., Jones, D., Young, B. & Sutton, A. (2005). Synthesising qualitative and quantitative evidence: a review of possible methods. J Health Serv Res Policy 10, 45-53. • Noblit, G.W., Hare, R.D. (1988). Meta-ethnography: Synthesizing Qualitative Studies. Newbury Park: Sage Publications; 1988. • Booth, A., Noyes, J., Fleming, K., Gerhardus, A., Wahlster, P., Van Der Wilt, G.J., Moztgemba, K., Refolo, P., Sacchini, D., Tummers, M., Rehfuess, E. (2016) Guidance on choosing qualitative evidence synthesis methods for use in health technology assessments of complex interventions [Online]. Available from: http://www.integrate-hta.eu/downloads/ • Tong, A., Flemming, K., McInnes, E., Oliver, S., & Craig, J. (2012). Enhancing transparency in reporting the synthesis of qualitative research: ENTREQ. BMC medical research methodology, 12(1), 181. • Field A. & Gillett, R. (2010). How to do a meta-analysis. Br J Math Stat Psychol, 63, 665–694. • Hedges L. & Vevea J. (1998) Fixed-and random-effects models in meta-analysis. Psychol Methods, 3, 486–504. • http://libguides.library.nuigalway.ie/systematicreviews • Cochrane Collaboration Website: http://www.cochrane.org/ • PROSPERO: http://www.prospero.com/ • Andy Field: A Bluffer’s Guide to Meta-analysis: http://www.statisticshell.com/docs/meta.pdf • http://nuighealthpsychology.wordpress.com/2014/10/13/5-tips-on-conducting-a-systematic-review/ References

×