Many evidence syntheses are conducted in response to the evidence needs of organisations looking to make evidence-informed decisions. An increasing number of CEE Systematic Reviews and Maps are commissioned by various organisations, institutes and agencies all over the world.
To get the best possible outcome from a commissioned evidence synthesis, to plan effectively the use of resources and timing of the work of the review team, make sure the review is of the best possible quality and independent of vested interest, many commissioners now work closely with CEE and make use of its guidelines and standards. Here are a few tips to help you when considering commissioning a systematic review. Don’t hesitate to contact us for more information at firstname.lastname@example.org.
CEE Evidence Syntheses are either Systematic Reviews or Maps as defined below.
What is a Systematic Review?
A Systematic Review collates, critically appraises, and synthesizes all available evidence relevant to a question. Reviewers use pre-defined methods to minimize bias and thus provide more reliable findings that could inform decision making.
What is a Systematic Map?
A Systematic Map collates, codes, and configures all available evidence relevant to a question. Reviewers use pre-defined methods to minimize bias and assess the extent of the evidence to provide a basis that could inform further research and synthesis.
Why commission a CEE Systematic Review?
A key purpose of a systematic review is to provide a summary of the best available evidence to support decision making. The methodology is designed to reduce bias in collection, synthesis and reporting of evidence. A systematic review is a transparent, replicable, upgradable compilation of existing evidence from the peer-reviewed scientific and grey literature (reports, theses…) in order to produce a synthesis of knowledge on a specific question (for broader questions see evidence mapping below).
You might commission a systematic review if you
Systematic reviews comply to a pre-defined methodology. Genuine systematic reviews will either be posted in the libraries of one of the recognized collaborations (Cochrane, Campbell or CEE) or have met the standards for transparency of conduct as set out in the guidelines of these collaborations. For CEE, please peruse our Guidelines and Instructions for Authors for more details.
The protocol of a systematic review should always be made available prior to the conduct of the review itself. It should preferably be registered/published in one of the libraries of a Collaboration (Cochrane, Campbell, CEE) and peer-reviewed before the systematic review is conducted.
Publication of a protocol prior to knowledge of the available studies reduces the impact of review authors’ biases, increases transparency of methods and processes, reduces the potential for duplication, and allows peer review of the planned methods.
Systematic Reviews are conducted on specific questions with well defined interventions and outcomes. Often a decision maker will be faced with a broader question involving many interventions and outcomes. In this case it may be appropriate to start with a systematic mapping of the evidence. CEE Systematic Maps collects, organises and describes the evidence on a broader question without attempting to synthesise. It asks how much and what type of evidence exists rather than what the evidence suggests. CEE Systematic Maps identify gaps in the evidence base as well as where a sufficient evidence exists for a systematic review and synthesis.
Many organisation will commission a systematic review using a tendering process. Commissioners may wish to consult CEE at the planning stage in order to ensure the effective use of the CEE process. We suggest that commissioners state in the ‘call for tender’ that review teams should comply with CEE guidelines and standards. We recommend that this includes the process of submission of both the protocol and full review to CEE for peer review and publication through its journal ‘Environmental Evidence‘. This provides the commissioning body with both independent assessment of standards at the start and completion of the process, and an independent open-access platform for the systematic review, demonstrating its standard and quality.
The conduct of systematic reviews is relatively new in environmental management and many potential review teams will need some training to achieve a product of the necessary standard. Our experience in training and assessing the work of review teams tells us that the quality of a review will be enhanced if there are regular exchanges between the review team and the CEE. Often this is linked to a tailored training programme to ensure that the Review Team comply with the requirements of the Collaboration.
At the CEE, past experience tells us that a good systematic review requires careful planning and execution. Its exact duration depends on the complexity of the review question, the skills of the review team and the type of data they will deal with.
The initial steps leading to the publication of the review protocol are similar to a project management exercise and usually take about a third of the length of the review, because a lot of work is required to anticipate all possible decisions and problems, make links with stakeholders, and get feedback from stakeholders and peer-reviewers. Sometimes during this stage it becomes obvious that the review question needs to be revisited as it is too broad or the workload will be too demanding within the decided timeframe.
The peer-review process of a protocol takes on average 1-2 months. It can be extended to 2 months for a full review (depending on its length, availability of peer-reviewers…). The Editor may ask for amendments prior to the peer-review if the manuscript does not fully comply with the instructions for authors.
Total costs of SRs vary greatly and it would be misleading to try and quote a figure. The range is probably 30 to 300 US dollars (20 to 200K GB pounds). By far the most substantial cost of the process is the time of skilled personnel to conduct the review. The basic CEE article processing charges, for editorial and publishing costs, are fixed and usually constitute a small percentage of the total costs.
Our top tip for saving time and money is to ask, as part of the tender submission, that a protocol be drafted.
Question setting: contact us when you are discussing about your evidence needs and deciding whether to commission a systematic review. We can provide feedback on its feasibility, likely costs and how to phrase the review question, thus saving time and preventing false expectations.
Get a good understanding of the challenges and steps of a review by attending our introductory workshops or by organising a workshop in your workplace. See Training page.
Include in the call for tender the possibility for the review team to benefit from our training programme.