A number of methodological issues are of particular concern:
Bias
There is a risk of bias being introduced into a systematic review and/or meta-analysis if the characteristics and distribution of results (related to an impact or intervention) are associated with the statistical significance or precision of results, or the direction of the results themselves, and if the reviewer does not take account of these associations. The impact of different types of bias and methods for controlling and detecting bias are of interest to the EMG.
Effect size generation
A formal definition of meta-analysis is the combination of weighted effect sizes. Generation of effect sizes is therefore of critical importance. However, in ecology spatial and temporal autocorrelation coupled with the diversity of experimental designs present considerable complexity, meaning effect sizes can be generated in many ways. Developing robust and repeatable methods for the generation of effect sizes and exploring methodological variation are of interest to the EMG.
Combination of effect sizes
Just as there is a diverse array of effect sizes (and numerous ways to derive them) there are many different methods for combining effect sizes. The relative strengths and weaknesses of the different techniques are of interest to the EMG. The application of Bayesian hierarchical models is of particular interest to some members.
Exploring heterogeneity
Methodological and ecological reasons for heterogeneity are the primary focus of interest in ecological analyses (as opposed to pooled effects). There are many interesting issues regarding exploring heterogeneity relating to weighting, power, independence, phylogenetics and interactions with bias. These are all of interest to members of the group.
Other issues
There is a tendency to focus on statistical aspects of meta-analysis but other methodological issues can be just as critical for a robust synthesis particularly if it is needs-led. Information science and data retrieval are very important skills. Stakeholder dialogue may be critical for needs-led reviews and effective dissemination mechanisms are required if policy impacts are desired. The use of qualitative techniques to inform quantitative synthesis is also of interest.
The EMG is interested in exploring all these issues and would like to conduct methodological systematic reviews and analyses, provide advice to the Collaboration for Environmental Evidence, hold workshops and publicize its findings through appropriate means.
