Skip to main content


Conducting the Review


The goal of the conduct stage of the systematic review process is to perform the required systematic review processes as defined in the review protocol. 


A common theme throughout descriptions of systematic review processes is that the individual tasks comprising each process should be conducted by several different members of the review team. Results obtained by different team members undertaking the same task can then be compared and any disagreements resolved. This has been adopted as the gold standard process for conducting systematic reviews in order to:

  1. Minimize human errors resulting from fatigue or stress such as misunderstanding or misinterpreting a primary study report, or wrongly transcribing information or data reported in a primary study.
  2. Reduce the risk of personal bias affecting review conclusions (people often tend to overlook or ignore information that does not agree with their own views on a topic).

If you are not intending to use the standard approach, your own approach needs to have been explained and justified in your review protocol. For example, it may be non-standard because:

  • The review is a Rapid Review which, in order to minimize elapsed time, is being done by a single researcher.
  • The review team has specialized tool support to provide independent checking for some tasks.
  • The review is a postgraduate study where a supervisor checks only a subset of the tasks to ensure their student performs individual tasks correctly and leaves the student to complete the tasks as a single researcher.
  • The review is being done by a single researcher using a test-retest approach to check various tasks.

Another challenge for conducting a review is that, except for extremely simple systematic reviews (e.g., only a few relevant papers and a small review team), there is a substantial management overhead required to organize a secondary study and to keep track of the status of all the process tasks. 

For example, the search process may generate large numbers of candidate primary studies including multiple citations of the same paper, and papers that report multiple studies of the same research question (see for example, papers reporting the analysis of families of experiments). The final decision whether to include or exclude each paper or individual empirical study found by the search process requires at least two members of the review team to assess each paper and for any disagreements among them to be resolved. It is important to ensure that no candidate primary studies are lost or forgotten. It is also important to ensure that different researchers are using the eligibility criteria consistently. It is useful to calculate agreement statistics during the selection process rather than wait until all the papers have been assessed as this may identify where the eligibility criteria are not being used consistently by different team members.  In addition, review reporting standards recommend specifying how many papers entered in each stage of the search and selection processes and how many were excluded in each stage. For papers that were read in full, guidelines recommend identifying the reason for any papers rejected at that stage. 

Similar issues affect data extraction and the assessment of primary study quality/risk of bias because we usually require at least two team members to extract information from each primary study, and the results need to be compared and any disagreements resolved. Although the number of agreed primary studies will be considerably fewer than the number of candidate primary studies found by the search process, we still need to ensure that all primary studies are properly processed. Again, agreement statistics are best calculated while the process is ongoing in order to check that different reviewers are using the data extraction forms consistently.

These issues mean that, unless your systematic review is small in terms both of candidate primary studies and team members, one team member must act as the team leader. Furthermore, they will need some tools to support the management process ranging from spreadsheets and citation management systems to comprehensive database systems tailored to SRs. The larger the scope of the SR the more important such tools become.

The processes included in the conduct stage of the review are:

  • Search
  • Selection
  • Data Extraction
  • Critical Appraisal Process (also termed Quality Assessment or Risk of Bias Assessment)
  • Data Analysis and Synthesis 
  • Assessing the Strength of Evidence (also referred to as  Certainty Assessment)


The outcomes of the conduct stage are the outcomes of each process including information about the validation processes used within each process and any deviations from the original review protocol.

This page was updated in July 2023.