The conversation with Pucksoppet made me think of the situation in diabetes, which may be moving the whole business of clinical trials forward. Hence this overlong ramble about that.
Brief story, back in 2007 or so a bloke called Nissen did a meta-analysis on a diabetes drug, rosiglitazone, and found an increase in mortality. Everybody jumped on that in a shock horror way, including Goldacre, as it happens, even though none of these trials were powered for outcomes, and a lot of them were only six weeks long, and IIRC none were longer than a couple of years. Garbage in, garbage out. Nobody much believes this any more, even though rosiglitazone is dead, as is its stable mate, pioglitazone, pretty much, even though pioglitazone improved (secondary) CV outcomes in a randomised cardiovascular outcomes trial (CVOT), and probably would have hit its primary outcome if that had been specified better (see her
Anyway, that led to people thinking, hang on, we really don’t do anything to guarantee the cardiovascular (CV) safety of these drugs before they hit the market. So, the FDA came up with a cunning plan
. Very briefly:
* All Phase II and Phase III trialsa
must have design that makes them suitable for inclusion in a meta-analysis for effects on hard CV outcomes. This is always a composite outcome, usually “three-point major adverse cardiovascular outcomes (3P-MACE)”, typically death from a CV cause OR myocardial infarction [heart attack] OR stroke. Sometimes 4P-MACE also includes hospitalisation for unstable angina or heart failure or something, if something else is of special interest.
*If the 2-sided upper 95% confidence interval (CI) for MACE against placebo is below a 1.3, a CVOT is generally not required, i.e. the drug is considered safe (enough). I don't think this has happened, because there aren't enough bodies in the meta-analyses to get that sort of precision. Even if it did, the company would need to do a CVOT, as all the competition has, as per below.
*If the upper 95%CI is between 1.3 and 1.8, a CVOT is needed to show that an upper 95%CI of less than 1.3. This trial can happen while the drug is on the market.
*If your meta-analysis had the upper 95%CI more than 1.8, then tough tit, you need to do more trialling to get it below 1.8 before you can submit your application to the FDA, or no go.
This has had major implications for trials and data:
a) There have been a lot of new drugs coming out in diabetes recently. In practice, they always hit the 1.3-1.8 range, so all need a postmarketing CVOT. This means a shitload of CVOTs, and subsequent meta-analysing (Christmas every day for NEJM, Lancet and the leading diabetes journals). All of the many trials here are powered for outcomes, at least for non-inferiority (see below).
b) Remember, all
the Phase II trials go into that FDA-mandated meta-analysis. The FDA routinely puts the application documents online (see this
for one of them, that shows all the trials). Nowhere to hide there.
c) The trials are usually designed as safety studies. The statistical plan usually demonstrates non-inferiority vs. placebo first, followed by a hierarchy of effect on, say, primary outcome (usually 3P-MACE), then MI, then CHD death, then all-cause death…whatever…and as soon as one of these is not significant, analysis for superiority stops. Any nominally significant effect on any other endpoint further down is only viewed as exploratory (New Engl J Med doesn’t even allow p values to be displayed for these).
d) If superiority for a given outcome or component of an outcome is demonstrated, the drug will get an indication for that.
e) This has led to muttering from senior triallists that it’s a something of a lottery – what ends up as significant, and only these end up in the label as a therapeutic indication, depends on the order in which the endpoints are analysed.
This has certainly increased the transparency of CV (and other) safety evaluation of these drugs by provision of loads of data, though clearly there is enough commercial interest in the diabetes field to justify CVOTs. Doesn't address other things, like how representative these trials are to Mrs Scrote in Barnsley, who has 17 different comorbid ailments and who wouldn't get within 7 miles of inclusion in a trial, but there you go, that's what observational studies are for, or "real world data" as we all call it now.
Quick, basic and undoubtedly incomplete overview of trial phases for anybody new to this, reasonably tyoical of diabetes field, at least…
– first administration to humans, typically a dozen or two healthy volunteers, wide range of doses given, measure things like pharmacokinetics (PK; how long it stays in circulation), maybe first idea of pharmacodynamics (what it’s supposed to do) and safety/tolerability (look for unexpected side effects, don’t usually find any much). Modelling of PK is useful to set dosing for Phase II...
– first demonstration of efficacy, and initial tolerability/safety. Several. of these, usually. Size varies, maybe fifty to a couple of hundred patients with the disease in question, randomised for something like 6 weeks to 6 months. Some dose ranging [use trial(s) to select one or two doses for the next step], good initial idea of tolerability and safety.
– the biggie, that leads to a label, usually only one. Hundreds of patients (maybe thousands), longer duration (up to a few years, often until a pre-specified number of endpoints are measured, enough to drive the statistics). Formal statistical demonstration of efficacy, much better idea of tolerability and safety. Can address a surrogate endpoint (e.g. blood pressure), or hard clinical outcomes (e.g. MACE). Hard outcomes are adjudicated, i.e. a committee blinded to treatment reviews records and decides if Patient 003456y/2b really did have a heart attack or not - reduces bias and increases accuracy.
– a trial conducted while the drug is being marketed. Often observational, may be randomised.
PS: Pucksoppet - I'm still going to get seven shades of agreement out of you - outside, now
Just tell 'em I'm broke and don't come round here no more.