Widely used and widely trusted, meta-analyses of randomised controlled trials (RCTs) are treated as the gold standard in evidence synthesis. But how do you evaluate a treatment or intervention when there are no RCTs to pool? This is the problem routinely faced by those generating, synthesising and communicating evidence when the published literature consists mostly of observational, single-arm studies. The solution lies in a technique that deserves to be better understood: proportional meta-analysis.
What is meta-analysis?
Over the last two decades, systematic reviews and meta-analyses have grown in prominence to become, perhaps, the most influential type of evidence used in contemporary healthcare decision-making. By statistically pooling outcomes from multiple independent studies to produce a single, precise estimate of treatment effect, the meta-analysis makes it easier to detect true effects, resolve conflicts between studies and assess study heterogeneity.
Most meta-analyses are pairwise, comparing a treatment with a suitable comparator to generate a pooled estimate of effect. Network meta-analysis extends this to simultaneous comparisons across multiple interventions using a common comparator as the anchor. Among the many different meta-analysis methodologies, however, there is one that arguably stands out: the practical and versatile “meta-analysis of proportions”.
What is a proportional meta-analysis?
The proportional meta-analysis, as it is usually known, generates a pooled effect – but not a pooled comparison – by using outcome data reported as a rate or percentage in individual studies to calculate an overall proportion. Commonly used to summarise event rates, complication rates and procedural success, proportional meta-analysis can also be used to characterise the impact of an intervention on a particular condition. It has a valuable role to play in specialties such as surgery and in the medical devices sector where, because RCTs are often impractical or unethical, decision-makers are reliant on observational studies.
Here, then, are five practical applications of proportional meta-analysis that synthesise evidence from the real world:
1. Supporting health technology assessment
Health technology assessment (HTA) bodies such as the National Institute for Health and Care Excellence (NICE) not only tolerate but expect to see non-randomised studies in settings where randomised studies are uncommon – such as when estimating the effects of medical devices. NICE’s Real-World Evidence Framework has legitimised the use of single-arm and observational studies – provided the right analytical methods have been used to minimise the risk of bias and characterise uncertainty. For HTAs, a proportional meta-analysis of these studies signals that a device-maker has engaged with the totality of the evidence base rather than conveniently cherry-picking the individual studies with the most favourable outcomes.
2. Facilitating clinical conversations
Systematic reviews that include proportional meta-analysis are quantifying real-world results at scale – reflecting routine practice better than many trials. By including broader patient populations, different care settings and varied operators they can facilitate more credible discussions with clinicians about the outcomes that matter to them – such as technical success and complication rates – anchored in published, peer-reviewed evidence that is more powerful than a scattered set of study-specific claims.
3. Populating health economic models
Health economic models need parameter estimates – such as transition probabilities, event rates, adverse event frequencies and long-term outcomes. For most medical devices these cannot be reliably drawn from a single trial. A proportional meta-analysis provides a pooled central estimate and a principled uncertainty range for each parameter. This is a far more robust approach than picking a rate from a single “representative” study that is really just a fragile number that happens to suit the value story. It enables the model’s foundations to be built with defensible, auditable, real-world data.
4. Enhanced post-market surveillance
Compliance with the EU Medical Device Regulation requires manufacturers to undertake post-market surveillance to confirm the continued safety and performance of a device throughout its lifecycle – a task that should be done by synthesising the accumulating real-world evidence, but often is not. A proportional meta-analysis turns this from a narrative compliance exercise into a quantitative one. A pooled estimate of complication rates, technical success and durability across the published literature gives regulators a rigorously derived headline figure. It can also surface rare safety signals that would be invisible at single-study level because they are too infrequent to reach statistical significance. Used in this way, proportional meta-analysis can turn a compliance burden into a strategic source of real-world intelligence on device performance.
5. Identifying evidence gaps and shaping future research
A proportional meta-analysis does not simply conjure up a pooled estimate. It requires reviewers to build a comprehensive inventory – a structured map of the populations, settings, follow-up durations and outcomes that constitute the entire evidence base. The empty spaces in that map – eg, the under-represented subgroups, absent comparators and inconsistent outcome definitions – can be commercially consequential. There may be patient subgroups with no published evidence, important care settings that have never been studied or an absence of head-to-head data against the competitor you most want to displace. A gap analysis can therefore show you where the next investment in evidence generation, such as a registry, will have the biggest impact. Exploited in this way, a proportional meta-analysis is not just a rear-view mirror that synthesises what has passed – it is also a forward-looking tool for shaping your future evidence generation strategy.
A valuable investment
A proportional meta-analysis is at its most valuable not when used as a one-off deliverable for a single submission, but as a living, breathing, multipurpose asset that strengthens HTA submissions, engages clinicians, powers economic models, sharpens post-market surveillance and shapes future evidence generation. That all adds up to a considerable return on investment in a single review.
