Systematic reviews address predetermined research inquiries employing clear, replicable methodologies to locate, evaluate, and amalgamate findings from primary research investigations. The essential features of a systematic review include: (a) clearly defined objectives with a methodological approach that is transparent and replicable; (b) a methodical search strategy aimed at identifying all relevant studies meeting the predefined criteria; (c) an evaluation of the reliability of the results from the selected studies, such as assessing bias risk and confidence in cumulative estimates; and (d) a systematic and organized presentation and analysis of the identified studies' characteristics and outcomes.
Navigating the landscape of literature reviews can be daunting, especially when terms like 'systematic review' are used interchangeably with others that may not align with the rigorous standards expected. Whether you're embarking on your first literature search or seeking to refine your search strategies, this guide will serve as a helpful introduction to navigate the world of systematic literature searching effectively.
Systematic searches may form part of a much larger systematic review. Systematic reviews are complex endeavors, demanding careful planning and expertise, and should not be approached lightly. You cannot do one on your own without collaboration and input from multiple individuals with diverse expertise. It's a team effort that ensures the rigor, comprehensiveness, and reliability of the review.
Consider the following questions before undertaking a systematic review:
Research Question Scope: Assess the depth and scope of your research question to determine if a systematic review is warranted.
Resource Evaluation: Take into account available resources expertise, including time, personnel, and access to relevant databases and literature. You may require additional training.
Expertise and Collaboration: Assess your own expertise and consider collaborating with experts in specific areas related to your research question.
Methodologies and Tools: Familiarise yourself with the methodologies and tools required for quality assessment, data extraction, and analysis.
Clear Research Protocol: Ensure a clear and well-defined research protocol is in place before initiating the systematic review process.
Taking these factors into account will contribute to a more successful and well-executed systematic review. Speak to a member of the Academic Librarians team if you are still not sure.
Moher, D., Shamseer, L., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., Shekelle, P., Stewart, L. A. and Group, P.-P. (2015) 'Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement', Systematic Reviews, 4(1), article number 1. Available at: https://systematicreviewsjournal.biomedcentral.com/articles/10.1186/2046-4053-4-1 (Accessed: 13 March 2024).
Cochrane Collaboration (2016) What are systematic reviews? 27 January. Available at: https://www.youtube.com/watch?v=egJlW4vkb1Y (Accessed: 13 March 2024).
Research Masterminds (2021) Narrative vs systematic vs scoping review, what’s the difference? 19 December. Available at: https://www.youtube.com/watch?v=Nwp9tkhpLCo (Accessed: 13 March 2024).
Registering your review at an early stage minimises the chances of duplication and enables contact with others interested in the topic.
Registration sites include:
There are more than 30 types of systematic reviews. While this table provides a summary of some approaches, it is not exhaustive. It's crucial to recognise that there isn't always consensus on the boundaries and distinctions between the approaches described below.
If you are new to systematic reviews, the terminology may be a little confusing, leading to instances where students are asked to conduct a full systematic review when, in reality, the requirement is often more aligned with conducting a literature review systematically. The systematic review methodology is intentionally rigorous and may not be conducive to individual researchers or those working within constrained timelines.
|
Label |
Description |
Search |
Appraisal |
Synthesis |
Analysis |
|
Critical review |
Aims to demonstrate writer has extensively researched literature and critically evaluated its quality. Goes beyond mere description to include degree of analysis and conceptual innovation. Typically results in hypothesis or mode |
Seeks to identify most significant items in the field |
No formal quality assessment. Attempts to evaluate according to contribution |
Typically narrative, perhaps conceptual or chronological |
Significant component: seeks to identify conceptual contribution to embody existing or derive new theory |
|
Literature review |
Generic term: published materials that provide examination of recent or current literature. Can cover wide range of subjects at various levels of completeness and comprehensiveness. May include research findings |
May or may not include comprehensive searching |
May or may not include quality assessment |
Typically narrative |
Analysis may be chronological, conceptual, thematic, etc. |
|
Mapping review/ systematic map |
Map out and categorise existing literature from which to commission further reviews and/or primary research by identifying gaps in research literature |
Completeness of searching determined by time/scope constraints |
No formal quality assessment |
May be graphical and tabular |
Characterizes quantity and quality of literature, perhaps by study design and other key features. May identify need for primary or secondary research |
|
Meta-analysis |
Technique that statistically combines the results of quantitative studies to provide a more precise effect of the results |
Aims for exhaustive, comprehensive searching. May use funnel plot to assess completeness |
Quality assessment may determine inclusion/ exclusion and/or sensitivity analyses |
Graphical and tabular with narrative commentary |
Numerical analysis of measures of effect assuming absence of heterogeneity |
|
Mixed studies review/mixed methods review |
Refers to any combination of methods where one significant component is a literature review (usually systematic). Within a review context it refers to a combination of review approaches for example combining quantitative with qualitative research or outcome with process studies |
Requires either very sensitive search to retrieve all studies or separately conceived quantitative and qualitative strategies |
Requires either a generic appraisal instrument or separate appraisal processes with corresponding checklists |
Typically both components will be presented as narrative and in tables. May also employ graphical means of integrating quantitative and qualitative studies |
Analysis may characterise both literatures and look for correlations between characteristics or use gap analysis to identify aspects absent in one literature but missing in the other |
|
Overview |
Generic term: summary of the [medical] literature that attempts to survey the literature and describe its characteristics |
May or may not include comprehensive searching (depends whether systematic overview or not) |
May or may not include quality assessment (depends whether systematic overview or not) |
Synthesis depends on whether systematic or not. Typically narrative but may include tabular features |
Analysis may be chronological, conceptual, thematic, etc. |
|
Qualitative systematic review/qualitative evidence synthesis |
Method for integrating or comparing the findings from qualitative studies. It looks for ‘themes’ or ‘constructs’ that lie in or across individual qualitative studies |
May employ selective or purposive sampling |
Quality assessment typically used to mediate messages not for inclusion/exclusion |
Qualitative, narrative synthesis |
Thematic analysis, may include conceptual models |
|
Rapid review |
Assessment of what is already known about a policy or practice issue, by using systematic review methods to search and critically appraise existing research |
Completeness of searching determined by time constraints |
Time-limited formal quality assessment |
Typically narrative and tabular |
Quantities of literature and overall quality/direction of effect of literature |
|
Scoping review |
Preliminary assessment of potential size and scope of available research literature. Aims to identify nature and extent of research evidence (usually including ongoing research) |
Completeness of searching determined by time/scope constraints. May include research in progress |
No formal quality assessment |
Typically tabular with some narrative commentary |
Characterizes quantity and quality of literature, perhaps by study design and other key features. Attempts to specify a viable review |
|
State-of-the-art review |
Tend to address more current matters in contrast to other combined retrospective and current approaches. May offer new perspectives |
Aims for comprehensive searching of current literature |
No formal quality assessment |
Typically narrative, may have tabular accompaniment |
Current state of knowledge and priorities for future investigation and research |
|
Systematic review |
Seeks to systematically search for, appraise and synthesis research evidence, often adhering to guidelines on the conduct of a review |
Aims for exhaustive, comprehensive searching |
Quality assessment may determine inclusion/exclusion |
Typically narrative with tabular accompaniment |
What is known; recommendations for practice. What remains unknown; uncertainty around findings, recommendations for future research |
|
Systematic search and review |
Combines strengths of critical review with a comprehensive search process. Typically addresses broad questions to produce ‘best evidence synthesis’ |
Aims for exhaustive, comprehensive searching |
May or may not include quality assessment |
Minimal narrative, tabular summary of studies |
What is known; recommendations for practice. Limitations |
|
Systematized review |
Attempt to include elements of systematic review process while stopping short of systematic review. Typically conducted as postgraduate student assignment |
May or may not include comprehensive searching |
May or may not include quality assessment |
Typically narrative with tabular accompaniment |
What is known; uncertainty around findings; limitations of methodology |
|
Umbrella review |
Specifically refers to review compiling evidence from multiple reviews into one accessible and usable document. Focuses on broad condition or problem for which there are competing interventions and highlights reviews that address these interventions and their results |
Identification of component reviews, but no search for primary studies |
Quality assessment of studies within component reviews and/or of reviews themselves |
Graphical and tabular with narrative commentary |
What is known; recommendations for practice. What remains unknown; recommendations for future research |
The table is based on the following work:
Grant, M.J., and Booth, A. (2009), 'A typology of reviews: an analysis of 14 review types and associated methodologies', https://doi.org/10.1111/j.1471-1842.2009.00848.x (Accessed: 23 January 2024).
To explore various review methodologies and determine the most suitable approach for your topic, we recommend consulting the following articles:
Grant, M.J., and Booth, A. (2009), 'A typology of reviews: an analysis of 14 review types and associated methodologies', https://doi.org/10.1111/j.1471-1842.2009.00848.x (Accessed: 3 January 2024).
Sutton, A., Clowes, M., Preston, L., and Booth, A. (2019), 'Meeting the review family: exploring review types and associated information retrieval requirements', Health Information & Libraries Journal, 36(3), pp. 202-222. Available at: https://doi.org/10.1111/hir.12276 (Accessed: 22 January 2024).
Munn, Z., Stern, C., Aromataris, E., Lockwood, C., and Jordan, Z. (2018), 'What kind of systematic review should I conduct? A proposed typology and guidance for systematic reviewers in the medical and health sciences', BMC Medical Research Methodology, 18, article number 5. Available at: https://10.1186/s12874-017-0468-4 (Accessed: 22 January 2024).
Siddaway A.P., Wood, A.A., and Hedges, L.V. (2018) 'How to Do a Systematic Review: A Best Practice Guide for Conducting and Reporting Narrative Reviews, Meta-Analyses, and Meta-Syntheses', Annual Review of Psychology, 70, pp. 747-770. Available at: https://doi.org/10.1146/annurev-psych-010418-102803 (Accessed: 10 February 2024).
Pollock, A. and Berge, E. (2018) 'How to do a systematic review', International Journal of Stroke, 13(2), pp. 138-156. Available at: https://doi.org/10.1177/1747493017743796#bibr5-1747493017743796 (Accessed: 14 February 2024).
PRISMA, which stands for Preferred Reporting Items for Systematic Reviews and Meta-Analyses, is a widely used guideline for reporting systematic reviews and meta-analyses. It was developed to help authors improve the quality and transparency of their research reporting in these types of studies. The PRISMA statement consists of a 27-item checklist and a four-phase flow diagram, which provide detailed guidance on reporting various aspects of systematic reviews, including the search strategy, study selection process, data extraction, risk of bias assessment, and synthesis of results. Adhering to the PRISMA guidelines helps ensure that systematic reviews are conducted and reported in a rigorous and transparent manner, enhancing their reliability and credibility.
For additional guidance and best practice, consult the following references and resources:
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., Chou, R., Glanville, J., Grimshaw, J. M., Hrobjartsson, A., Lalu, M. M., Li, T., Loder, E. W., Mayo-Wilson, E., McDonald, S., McGuinness, L. A., Stewart, L. A., Thomas, J., Tricco, A. C., Welch, V. A., Whiting, P. and Moher, D. (2021) 'The PRISMA 2020 statement: an updated guideline for reporting systematic reviews', BMJ, 372, pp. n71. Available at https://www.bmj.com/content/372/bmj.n71.long (Accessed: 10 February 2024).
Page, M. J., Moher, D., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., Chou, R., Glanville, J., Grimshaw, J. M., Hrobjartsson, A., Lalu, M. M., Li, T., Loder, E. W., Mayo-Wilson, E., McDonald, S., McGuinness, L. A., Stewart, L. A., Thomas, J., Tricco, A. C., Welch, V. A., Whiting, P. and McKenzie, J. E. (2021) 'PRISMA 2020 explanation and elaboration: updated guidance and exemplars for reporting systematic reviews', BMJ, 372, pp. n160. Available at: https://www.bmj.com/content/372/bmj.n160.long (Accessed: 10 February 2024).
Tricco, A. C., Lillie, E., Zarin, W., O'Brien, K. K., Colquhoun, H., Levac, D., Moher, D., Peters, M. D. J., Horsley, T., Weeks, L., Hempel, S., Akl, E. A., Chang, C., McGowan, J., Stewart, L., Hartling, L., Aldcroft, A., Wilson, M. G., Garritty, C., Lewin, S., Godfrey, C. M., Macdonald, M. T., Langlois, E. V., Soares-Weiser, K., Moriarty, J., Clifford, T., Tuncalp, O. and Straus, S. E. (2018) 'PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation', Ann Intern Med, 169(7), pp. 467-473. Available at: https://doi.org/10.7326/M18-0850 (Accessed: 10 February 2024).