Short abstract

Regulatory problem

In February 1990 an article raised concerns that the recently licensed fluoxetine might trigger suicide acts in depressed patients.³ A series of meta-analyses of published and unpublished antidepressant trials subsequently failed to show benefit in terms of suicidal acts with active treatment compared with placebo.^4-9 In fact, each analysis showed a small excess risk with active treatment for all classes of antidepressants, although the increases are compatible with chance and the original authors concluded there were no differences. For much of the 1990s campaigners were saying trials with placebo controls in depression were unethical, and these analyses were attempts to justify placebo controlled trials.

I recently participated in a cumulative meta-analysis of published trials that found an excess of suicide attempts in patients taking selective serotonin reuptake inhibitors (SSRIs) compared with those taking placebo.¹⁰ The numbers in the individual trials are small, so that although from 1988 onwards the point estimate indicates roughly a doubling of the risks of suicidal acts with SSRIs, the effect has only recently been consistently significant. Nevertheless, the trend should have been seen by both companies and regulators as something that required investigation before it became significant.

In October 1990, a medical officer within the division of neuropharmacological drug products of the US Food and Drug Administration informed SmithKline Beecham that his division did not see the relation between fluoxetine and suicide as: “a real issue, but rather as a public relations problem.”¹¹ If the FDA's view reflected that expressed in this communication, this position was adopted without holding a scientific advisory meeting. When the FDA held an advisory meeting on the issue of fluoxetine and suicide in September 1991, evidence on two other SSRIs, sertraline and paroxetine, already with FDA for close to two years, was not presented at the meeting. The combined raw data from trials of adults taking these drugs has never been shown to an FDA advisory panel.

Trials in children conducted from the mid-1990s indicated a risk ratio for suicidal acts (no suicides occurred) with antidepressants compared with placebo of 2.19 (95% confidence interval 1.50 to 3.19; P = 0.00005).¹² These results have recently formed the basis of warnings about the use of SSRIs in children. Clinical trials in adults submitted for regulatory approval of all new antidepressants show a similar risk ratio for suicidal acts compared with placebo of 2.17 (1.39 to 3.39; P = 0.0004) and for suicides of 4.61 (1.13 to 18.74; P = 0.0187).¹³ However, until May 2006 no warnings were issued for adults.

Manipulation of data

Although data submitted to the FDA show an excess of suicides with every antidepressant licensed since 1987 compared with placebo, this simple but crucial finding continues to be obscured. When presenting data on fluoxetine, sertraline, and paroxetine to both regulators and journals, the manufacturers included a series of suicidal acts that happened in the run-in phase before patients were randomised, presenting these as a post-randomisation placebo group. Figure 1 shows this disposition of the data schematically.

Fig 1

Time of occurrence and reporting of suicidal acts in adult trials of paroxetine, fluoxetine, and sertraline

FDA reviewers noted this recoding at the time.^14,15 Pfizer and GlaxoSmithKline, faced with the claim made here about the way in which data had been presented to regulators,¹⁶ have not denied what happened, although both companies argue that other factors such as duration of exposure to treatment need to be taken into account to get a complete picture.^17,18 Pfizer makes it clear that: “Pfizer's 1990 report to FDA plainly shows... 3 placebo attempts as having occurred during single blind placebo phases.” They add: “FDA has neither criticized these data or the report as inappropriate, nor required additional analyses.”¹⁹ The FDA in this case noted the recoding of suicides and suicide attempts but did nothing to give the problem publicity that might have led to guidelines being issued to avoid its recurrence in the future.

Crucially until GlaxoSmithKline's recent letter, the publicly available figures for suicides among patients on placebo in trials of paroxetine contained three suicides, all of which occurred after the active treatment phase of trials had finished.²⁰ One of these occurred 33 days after the end of active treatment, another in a patient started on fluoxetine, and a third in a patient on whom there were no clinical details.²¹

Previous meta-analyses have contained a mixture of controlled and uncontrolled data, and none have controlled for trial when pooling the data. In the case of sertraline and fluoxetine, I have obtained access to data that permits an analysis restricted to the double blind phase of placebo controlled trials and an analysis by trial from 1994. After I excluded patients who committed suicidal acts during the run-in period or after the end of treatment, 11 acts occurred among 2126 patients randomised to sertraline and two among 1196 patients receiving placebo. The analysis by trials gives a Mantel-Haenszel pooled risk ratio of suicides and suicidal acts with sertraline of 2.50 (95% confidence interval 0.72 to 8.67). When placebo controlled data on fluoxetine presented in the original new drug application and in a 1991 company analysis³ are analysed by trial, there are 1398 acts among patients randomised to fluoxetine and 645 among those randomised to placebo, giving a risk ratio of infinity. Combining the two drugs gives a risk ratio of 3.78 (1.13 to 12.67).

Interpretation of evidence

The potential availability of figures like this to the FDA and Medicines and Healthcare Products Regulatory Agency (MHRA) from the early 1990s suggests that regulatory approaches to data on safety and efficacy are asymmetric. For efficacy purposes, trials are seen as “assay systems,” and any positive results outweigh what may be a majority of negative results. In the case of sertraline, although only one of the initial five trials and five of the first 16 trials had clearly positive results, the FDA and MHRA opted to be guided by indications of possible efficacy that came from a small subset of trials.

As of 2004, a willingness to be guided by indicative data would have provided the regulators with a basis for attaching warnings about suicide acts to the general use of SSRIs since trials in children had established a causal link between antidepressants and suicide acts. However, instead the regulators continued to insist on an all but unreachable threshold. Regulators and companies have stated repeatedly that because the confidence interval for individual drugs and for a pooled analysis of the SSRI trials referenced above¹³ overlaps 1.0, there is no credible evidence of a suicide risk, even though the confidence intervals from trials in children for individual drugs include 1.0.

Figure 2 shows the distribution of risk for adult suicides in trials of SSRIs versus placebo with run-in cases removed. The figure shows an increased risk of suicides from active treatment, although the individual trials were not powered to settle the question. The best estimate for the likely risk of suicide on SSRIs over placebo is 2.6, and although minimal or no risk is compatible with the scientific data, the data are also consistent with a 10-fold increase in risk, and 2.6 is the point on the confidence interval with the greatest probability. Statements such as these would mandate further trials powered to settle the issue.²² Regulators use of statistical concepts of significance in lieu of further trials may stem from the need to make decisions and a hope that a simple rule will remove the risk from making decisions.

Fig 2

Distribution of significance of risk ratios for adult suicides (SSRI v placebo). The line parallel to the x axis represents the 95% confidence interval. The vertical line indicates the point where the confidence interval passes through 1.0. Redrawn from ...

Risk periods

Another way the problem has been obscured relates to risk periods. In the case of sertraline, for instance, FDA officials agreed with Pfizer that the company should do survival analyses and that it should control for exposure to the drug in a manner that assumed there was a constant hazard from treatment.²³ This assumption, which averages out periods of greatest risk, is problematic when the clinical evidence points strongly to a clear risk period at the start of treatment rather than a constant hazard.

One analysis combining both the recoding approaches and survival analysis with constant hazard showed a fivefold greater risk of suicidal acts in the placebo group than the paroxetine group.²⁴ However, the raw data point to a four times greater risk of suicidal acts in the paroxetine group.^13,14 This analysis and several others included events occurring in the run-in period,^4-7 even though the true disposition of the data (fig 1 left) was in the public domain for some years before.^13,14

Confounding effect?

In contrast, a recent FDA analysis of the data for suicides from adult placebo controlled trials of antidepressants indicates a doubling of relative risk of suicide with antidepressants compared with placebo: 1.98 (95% confidence interval 1.81 to 2.18).²⁵ But the presentation of these data has two puzzling aspects. Firstly, the FDA claims that by controlling for age, sex, and both inpatient versus outpatient and US versus non-US trial settings it can eliminate the risk of suicide. This assertion is dubious because controlling for age and sex in randomised data ought to be unnecessary. Imbalances in these variables should be contained in the confidence interval that lies clearly in the region of the adverse effect. Had there been substantial confounding, the more general validity of these trials would have been questioned from the start by both companies and regulators. The other two variables are not confounders but effect modifiers that identify high-risk subgroups, if there is an accepted causal effect to begin with. If there is such a causal effect, and there are grounds to think that the risk:benefit ratio between inpatient and outpatient treatment might differ, we should be told about this.

The second puzzle is that FDA officials responsible for this analysis, which was published as a brief abstract, refuse to hand over the data for confirmatory analyses, and no paper has since been submitted outlining the methods or arguments. This is inconsistent with recent regulatory approaches to data on suicidal acts in children. Under scrutiny from Congress, the FDA claims it needs to reanalyse data that it has had for over a decade.

Conclusions

Two factors may account for the above approach of the FDA and MHRA in applying statistical methods to clinical trial data. Firstly, regulators had a comparative lack of statistical expertise when this controversy began. Secondly, they are more accustomed to dealing with drugs individually rather than as a group. Remedying the statistical issues would not lengthen drug approval processes, and a more appropriate presentation of the data might lead to better evidence based inputs to public health decisions. Greater data transparency and statistical sophistication might not curb the enthusiasm with which both clinicians and patients take up new drugs, but it might lead to earlier research to discriminate between those who do well on new drugs and those who do not.

Such research, for instance, indicates that SSRIs can be effective for obsessive compulsive disorder in children. Some SSRIs are licensed for this purpose in the UK and US, but the British regulator has opted to manage the risk of suicide acts by contraindicating antidepressants for children rather than warning about hazards. A comparable proscription of SSRIs for adults would clearly be inappropriate, but failing either to warn or to demand suitably powered studies of the risks of treatment is also inappropriate. The regulators, however, seem stuck in a world where balancing evidence of potential benefit against actual risk causes real problems. The SSRI and rofecoxib disasters have harmed public confidence in drugs. We urgently need to learn how to regulate both the risks and benefits of new treatments more effectively. In making the data on paroxetine available, GlaxoSmithKline may have helped considerably.

Notes

References