What is the EMG?
The Ecological Methods Group (EMG) focuses on the science of research
synthesis with an emphasis on quantitative synthesis. The amount
of ecological information is growing at a huge rate, with around
12500 ISI indexed ecological articles produced in 2007. Ecologists
are therefore doing a good job of producing new data (leaving aside
the issue of quality). For ecological science, to progress not only
must new facts be accumulated and presented but their relation to
old ones must be explored.
Ecologists have traditionally used narrative reviews for this purpose but are increasingly turning to the use of meta-analysis and systematic review. Clearly, the methods used in such syntheses impact on the results and the uncertainty surrounding them. The methods group is concerned with the thoughtful and critical use of meta-analysis for research synthesis in ecology to improve the power and rigour of ecological meta-analysis.
|
|
Study quality |
|
|
|
High |
Low |
Synthesis |
High |
Robust and reliable evidence |
Potentially useful (identification of knowledge gaps and hypotheses) |
Low |
Can be repeated |
Misleading and erroneous evidence |
|
History
The EMG evolved from an NCEAS working group on meta-analysis in ecology augmented with other interested parties. It is modelled on the methods groups of the Cochrane and Campbell Collaborations and serves the same function for the colloboration for environmental evidence. The group is not a static edifice and it is hoped that it will evolve and grow with time, perhaps resulting in the formation of multiple groups with interests in specific methodological challenges.
Some Definitions
Meta-analysis represents a set of statistical methods for quantitative research synthesis developed in medicine and social sciences in late 1970s and introduced to ecology in early 1990s. It provides a more transparent, repeatable and robust alternative to narrative reviews and "vote-counting" approaches traditionally used for research synthesis in ecology.
Despite its great potential in addressing both basic and applied research questions, the progress of meta-analytic applications in ecology is still hindered by the:
- limited availability of meta-analytic training for ecology students
- limited palette of meta-analytic techniques and tools available in ecology compared to that available in medicine and social sciences
- lack of research to rigorously assess the relative performance of existing meta-analytical tools and to develop new ones
- lack of research to adjust these techniques to account for the structure of ecological data and the nature of ecological questions
- lack of a fully developed evidence-based framework in ecology.
Systematic review is a review of a clearly formulated question that uses systematic and explicit methods to identify, select, and critically appraise relevant research, and to collect and analyse data from the studies that are included in the review. Statistical methods (meta-analysis) may or may not be used to analyse and summarise the results of the included studies.
Vote counting procedures simply sum the number of positive, neutral and negative outcomes across multiple studies. Vote counting can be very misleading for three reasons. Firstly, subjective decisions are often made when defining positive or negative. Secondly, statistical significance is often used to decide if results should be included confounding the magnitude of effect and sample size. Thirdly, vote-counting takes no account of the differential weights given to each study. Vote-counting is therefore only utilised as a last resort in evidence-based frameworks such as in situations when standard meta-analytical methods cannot be applied. Inclusion of critical appraisal is strongly advocated as part of vote counting and methods for presenting such syntheses are currently under development (Ogilvie et al. 2008). Vote counting can also be used legitimately to simply characterise an evidence base (e.g. record the number of studies using a certain methodology or reporting particular outcomes) in order to assess the potential for more robust synthesis.
Ogilvie, D., Fayter. D., Petticrew, M., Sowden, A., Thomas, S., Whitehead, M., Worthy, G. (2008) The harvest plot: A method for synthesising evidence about the differential effects of interventions. BMC Medical Research Methodology, 8:8
