Introduction to Meta-Analysis in Comparative Physiology

This webpage hosts material for our meta-analysis workshop that was run complimentary to the A21 session “Illuminating hot topics in experimental biology: A meta-analytic approach to understanding biological phenomenon” at the Society for Experimental Biology (SEB) 2022 conference in Montpellier, France. The workshop was a team effort, run by Daniel Noble, Nicholas Wu and Essie Rodgers with help from Patrice Pottier.

You can watch the full workshop recording on YouTube.

About the Workshop

The material covered here is geared towards researchers interested in conducting a meta-analysis in the field of comparative physiology. In other words, the examples have been chosen from the comparative physiology literature so that the workshop is relevant to the SEB audience. Having said that, all the principles learnt here apply to meta-analyses conducted in any research field.

Meta-analysis is a huge, complex topic. We can only hope to touch the surface of how it’s done in a single morning workshop. As such, we will not have time to cover a critical aspect of meta-analysis; the process of systematic searching and data curation.

Systematic searches are important to comprehensive, transparent and reproducible meta-analyses. The question needs to be clearly defined, and systematic searches need to be well documented, carefully refined, have detailed inclusion/exclusion criteria so that they may be repeated in the future. Foo et al. 2021 is an excellent resource for this stage of a meta-analysis. We also recommend following the PRISMA Eco-Evo protocol proposed by O’Dea et al. 2020. For data curation in preparation for a meta-analysis, we recommend following advice by Schwanz et al. 2022.

We also provide a list of useful readings and software for earlier elements of a meta-analysis (e.g., extracting data from figures literature searching, snowballing etc). Please cite these software in your meta-analysis. Many of the authors spend a huge amount of time developing and maintaining these free resources. They are as important to cite as the statistical software used to analyse the data.

Preparation for the workshop

In preparation for the workshop, we highly recommend the following background knowledge as we will use the metafor R package (Viechtbauer 2010) as our primary method for constructing meta-analytic models:

• Understanding the basis for doing a meta-analysis (see Introduction: why do meta-analysis?)
• Familiarise yourself with the metafor package and its functions (see metafor website)
• As our workshop focuses on real-world data (and the complexities that it comes with) we highly recommend working through the “Meta-Analysis in R with {metafor} tutorial from “UseR Olso”. Having said that, this is not essential as we will walk participants through the code we provide.

Recommended readings

• Ten appraisal questions for biologist conducting meta-analysis studies (Nakagawa et al. 2017)
• Meta-analytic approaches and effect sizes to account for ‘nuisance heterogeneity’ in comparative physiology (Noble et al. 2022)
• Methodological issues and advances in biological meta-analysis (Nakagawa & Santos 2012)
• Non‐independence and sensitivity analyses in ecological and evolutionary meta‐analyses (Noble et al. 2017)
• Importance of phylogeny in ecological and evolutionary meta-analyses (Cinar et al. 2021)
• Testing for publication bias in ecological and evolutionary meta-analyses (Nakagawa et al. 2021)

For a comprehensive guide on meta-analysis starting from a scoping study to writing a meta-analysis paper, see the YouTube Hard-Boiled Synthesis course by Marc Lajeunesse. It covers materials in detail beyond our workshop.

Table of Contents

• Introduction
  • Why do meta-analysis?
• Effect sizes: what to use, when to use them, and what they mean?
  • Calculating and interpreting effect sizes
  • Controlling for nuisance heterogeneity
• Meta-analytic models
  • Fixed vs random effect meta-analytic models
  • Multi-level meta-analytic models
  • Complex non-independence – shared controls and robust variance estimators
  • Phylogenetic meta-analytic models
  • Multi-level meta-regression models
• Interpreting and reporting meta-analytic model results
  • Reporting heterogeneity – a critical aspect of meta-analysis
• Publication bias
  • Visualising publicaton bias
  • Sensitivity analysis to test and correct for publicaton bias

Sponsors

We thank the Society for Experimental Biology and the Company of Biologist for sponsoring the workshop and allowing the recording to be freely available for the public.