Barriers to access to clinical trial data: Obstruction of a RIAT reanalysis of the treatment for adolescents with depression study

1.Introduction

Public access to data is a key tenet of transparency in randomised clinical trials [1,2]. In order to establish confidence in an investigator’s conclusions, trial data should be open to scrutiny by the scientific community. Unfortunately, this is rarely the case, leading to biased reporting in journal articles [3,4]. The larger research community should have access to raw datasets, particularly with respect to harms, in efforts to increase confidence in the scientific method.

The restoring invisible and abandoned trials (RIAT) initiative was established by Dr Peter Doshi and others to address misreporting of randomised clinical trials (https://restoringtrials.org/). Our team is currently sponsored and guided by the RIAT Center in re-analysing the original treatment for adolescents with depression study (TADS) dataset, to clarify how safe and effective fluoxetine and cognitive behavioural therapy (CBT) are in treating adolescent depression [5].

In preparation for our RIAT reanalysis, we obtained data archive data use certification from the US National Institute of Mental Health (NIMH) in 2018. We currently have access to effectiveness and harms summary data via the US National Institutes of Health (NIH) National Database for Autism Research (NDAR). However, that dataset does not contain patient level information, which severely compromises the analysis of adverse effects which have never been adequately reported (see below). We therefore sought individual patient adverse events data for use in our reanalysis of TADS, and we wish to update the research community on significant roadblocks we have experienced in obtaining access to these data.

TADS was a multi-center parallel four-arm randomized controlled trial conducted in the early 2000s by the Department of Psychiatry and the Duke Clinical Research Institute at Duke University Medical Center, and sponsored by the NIMH [6]. Adolescents with depression (n = 439) were randomized to four treatment groups: fluoxetine alone (n = 109), CBT alone (n = 111), fluoxetine with CBT (n = 107) or pill placebo (n = 112) during TADS’ acute 12-week phase. The original investigators reported combination treatment with fluoxetine and CBT ‘offered the most favourable tradeoff between benefit and risk for adolescents with major depressive disorder’ [6, p. 807]. Such reporting has been widely criticised, encompassing issues with study design [7], statistical reporting and interpretation [7–9], discrepancies between article abstracts and their content [10], and misreporting of harms [11–13]. In fact, a recent meta-analysis criticised 19 international clinical practice guidelines for their reliance on TADS findings without taking into account the failure of TADS authors adequately to report adverse events [14].

Despite taking place over 20 years ago, TADS continues to be a highly influential RCT. Key TADS papers such as [6] have been cited many hundreds of times (at least 1351 citations, which are generally uncritical). Consequently, TADS’ findings have influenced clinical treatment guidelines in many countries, establishing the combination of fluoxetine and CBT as gold standard treatment for adolescent depression [15–17]. Furthermore, there is widespread acceptance that fluoxetine is a first-line treatment for adolescent depression, even by some who have expressed reservations about the safety and efficacy of antidepressants [18].

The original TADS investigators have published over 30 papers, without any containing adequate information on SAEs that occurred during the acute or follow-up phases. The most highly cited TADS paper [6] reports only that 23 of 33 events of self-harm and two of 51 psychiatric adverse events were classified as SAEs, without stating participants’ treatment allocations or indicating whether these cases are mutually exclusive. In response to concerns about antidepressants causing paediatric suicidality, the paper states that the number of suicide attempts in TADS ‘was too small to analyse statistically, and their lethality was low to moderate’ [6, p. 818]. It also states that, although the rate of suicide attempts in depressed adolescents is unknown, the rate of ‘harm-related adverse events’ in TADS’ 12 week phase was ‘below what might be expected in an untreated sample of depressed youth’ [6, p. 817]. Our review of the TADS dataset reposited on NDAR indicates that, of the 30 SAEs recorded during the study’s acute phase, 12 were suicide attempts among participants taking SSRIs, compared with only two attempts for participants not taking SSRIs [19].

As noted, TADS data available from NDAR do not include the data collection forms/case record forms (CRFs) completed by the Independent Evaluator at each clinic assessment visit, nor any adverse event narratives or other individual patient level data, which have been demonstrated to be essential for rigorous reanalysis [20]. In fact, the NDAR’s TADS dataset contains no information about SAEs beyond MedDRA terms and severity ratings. Our previous experience with reanalysing antidepressant trials [20,21] demonstrated there is much useful information contained in SAE narratives that is not captured in the various TADS spreadsheets. Specifically, the profile of adverse events changed significantly when we were able to correct errors made by the original authors of Study 329’s clinical study report. As with the Study 329 reanalysis, our proposed method of examining SAE narratives from TADS was to blind our MedDRA expert and have them code any adverse events described by the narratives. In addition, qualitative analysis would be carried out to discern any patterns that did not emerge in the MedDRA process.

2.Duke University

We approached Duke University, where the coordinating center team for TADS was based, regarding our reanalysis. Appendix A documents the extensive chronology of our attempts to obtain the SAE data, with hyperlinks to each of the emails or letters (see Table 1). The TADS website (https://tads.dcri.org/ organization) lists Dr John March as the Principal Investigator (PI) for the study, and notes that study enrolment closed in July 2003. We therefore initially wrote to Dr March, requesting access to the de-identified clinical record forms.

After several emails to Dr March received no reply, the Duke Clinical Research Institute informed us that Dr March had retired and that Dr John Curry was the current PI. Dr Curry reported that Duke was unable to provide us with any clinical report forms, because they were destroyed in 2017, in accordance with Duke’s document retention policies. He provided a spreadsheet supporting the document destruction dates (Supplementary Appendix A, item 22).

Dr Curry then advised that he had personally reviewed every archived study box still available from the TADS study to determine what, if any, patient level data might be available. He informed us that ‘the only patient-level data in the boxes are the serious adverse event (SAE) forms’ (Supplementary Appendix A, item 33). We therefore requested the provision of the remaining SAE forms following de-identification, and we executed a data use agreement with Duke (Supplementary Appendix A, item 62) after negotiations to reduce what we thought to be an excessive price for the provision of de-identified forms.

Subsequent correspondence with Duke’s Data Solutions Program Manager established that there were a total of 68 ‘patient SAE folders’ (Supplementary Appendix A, item 55), rather than the 66 SAEs recorded by the NDAR database (although, without seeing the folders, we cannot be sure that some were not duplicate patients).

Nearly two months after signing off on the data use agreement, Duke, in consultation with Dr Curry and its Institutional Review Board, withdrew its agreement to release the SAE forms, stating that the narratives the forms contained were too idiographic to be sufficiently de-identified. Dr Curry wrote:

However, providing the SAE forms to us would not breach the Federal Certificate of Confidentiality [22]. Firstly, names and sensitive information would be redacted prior to provision of data. Secondly, our team has NIMH approval to access TADS data through the NDAR, with appropriate measures taken to assure participant confidentiality. NIH policy asserts that:

Additionally, Duke’s response to us is inconsistent with its provision of patient-level data to the Columbia Suicidality Classification Group (currently known as the Columbia Lighthouse Project) for ‘reclassification of all possibly suicidal events’ [23, p. 1443]. Columbia University participated in TADS as a trial site, but the reclassification of suicide events was undertaken as a separate exercise, with FDA-commissioned suicidology specialists [23] reviewing suicide-related adverse event reports. We therefore assume that the reports must have been sufficiently de-identified for use outside the TADS project, which undermines Duke’s justification for refusing our request. We contacted the Columbia Lighthouse Project staff (led by Dr Kelly Posner Gerstenhaber), but they have not responded to our repeated requests to clarify whether they received any de-identified adverse event forms (Supplementary Appendix A, items 103, 104 and 107).

We believed Duke’s response placed them in breach of our data use agreement, and we therefore sought legal support from our institution (the University of Adelaide) to challenge the decision. However, this support was declined.

We sought independent legal advice from a US law firm (Baum Hedlund Aristei & Goldman). Their lawyers advised us that Duke’s stance was invalid because:

We offered to execute an additional data use agreement with Duke to reassure confidentiality and address any perceived risks (Supplementary Appendix A, item 71), but Duke has maintained its refusal to provide any de-identified SAE narratives (Supplementary Appendix A, items 73 through 106). We passed on to our University’s legal department the offer from Baum Hedlund to pursue the data on our behalf. We have not been given permission to publish the University’s response, on the grounds that it might compromise legal professional privilege.

RIAT principal investigator, Dr Peter Doshi, also advocated for our access to SAE data from Duke (Supplementary Appendix A, items 74–102) and Lilly (see below, Supplementary Appendix B, item 13), to no avail. Based on considerable experience reanalysing trials, Dr Doshi tried to negotiate alternate avenues for facilitating reanalysis of the SAEs and suggested legal considerations around confidentiality to Duke. Duke declined permission for us to publish their responses to Dr Doshi.

More recently, Duke provided us with a brief summary of SAEs that was not contained in their NIMH repository. This summary provided some minor elaboration of terms to describe each SAE (e.g. ‘suicide attempt (drug overdose)’, as opposed to ‘suicide attempt’ in the NDAR database). However, the newly provided summary could not be linked to existing NDAR databases to contextualise our reanalysis (Supplementary Appendix A, item 135). We therefore made two further requests that were both declined by Duke: clarification of why one SAE was excluded from reporting due to an ‘IRB violation’; and data on the timing of concomitant medication administration with respect to occurrence of SAEs (Supplementary Appendix A, items 135, 137 and 139). Concomitant medications were sometimes taken by participants allocated to placebo and CBT groups, despite their adverse events still reported by TADS authors under an intention-to-treat analysis rather than actual treatment [6,23,26] (also see Supplementary Appendix A, items 131–139 for a summary of our arguments to Duke). Discrepancies in the number of SAEs recorded are critical; because numbers are small, misattribution of suicidal events to different treatment groups can mask important harms [13].

Absent the availability of data from Duke, we have pursued other sources of information about SAEs, as outlined below.

3.Lilly Australia/Eli Lilly

We contacted Lilly, who provided TADS investigators with an educational grant to supply study medication [6], requesting all data they received on adverse events that occurred during TADS. We assert that Lilly should have access to TADS SAE data, because:

Lilly did not provide us with any data, and refused to have any of their correspondence published (see Supplementary Appendix C).

4.US Food and Drug Administration (FDA)

SAE data from TADS were also submitted to the FDA, as discussed extensively during the joint meeting of the Center for Drug Evaluation and Research and FDA Pediatric Advisory Committee in September 2004 (see [29] for full transcript), which led to the implementation of a black box warning for SSRIs. At that meeting, Dr John March (TADS’ PI) presented preliminary TADS findings, and Drs Tarek Hammad and Andrew Mosholder (Center for Drug Evaluation and Research medical reviewers) presented critical analyses of SSRI safety data. Because these medical reviewers subsequently published a reanalysis of suicide data collected during TADS [30], we contacted them to ascertain whether they also received de-identified SAE data from TADS (see Supplementary Appendix C). Dr Hammad confirmed that the meta-analysis required ‘detailed de-identified patient level data’ and it:

The 2004 Advisory Committee meeting transcript confirms that TADS SAE data were indeed included in the adjudication [29, pp. 35–37, 44, 153, 155].

Since May 2019, we have made two Freedom of Information Act applications to the FDA for patient-level narratives of SAEs (see Supplementary Appendix C). Firstly, we requested all data pertaining to TADS; we were not granted expedited processing of the applications, on the grounds that our request did not meet a ‘compelling need’ for urgent processing (Supplementary Appendix C, item 1). We subsequently amended our request to TADS SAE data only, in an attempt to reduce the request’s complexity and hasten processing time (Supplementary Appendix C, item 16). However, the FOIA office reaffirmed that our application was categorised as complex and would take ‘at least 2 years … before … (coming up) in the queue’ (Supplementary Appendix C, item 27).

In the interim, we searched the FDA Adverse Event Reporting System for adverse events related to ‘fluoxetine’ or ‘fluoxetine hydrochloride’ for children 12–17 years of age that occurred during 2000–2003, the period TADS was conducted [31]. The system reported 142 adverse events that met these search requirements, plus 857 undated events that were received by the FDA after the year 2000. These data did not yield any useful information, apart from indicating that three TADS cases with SAEs were not reported until 2007.

Although our initial FOIA request was submitted in early 2019, we acknowledge that the FDA would have been inundated with work in 2020, due to COVID-19. In general, the FDA has made some efforts to support collaborative, transparent drug approval processes, including recommending a harmonised system for disclosing clinical study reports [32]. In 2018, it piloted a voluntary program of public access to CSRs related to drug approvals [33] in order to address ambiguity in medical reviewers’ summaries that ‘are packaged in a format that can sometimes make it difficult for external audiences to extract and understand the detailed clinical evidence that supported the FDA’s approval decisions’ [32]. However, the only CSR volunteered was Janssen Biotech’s Erleada® (the prostate cancer drug, apalutamide) that redacted patient-level harm narratives [34], thus limiting external safety review and the FDA’s intention to clarify their drug approval process.

At a time when COVID-19 has accelerated regulatory actions with the attendant risks, it is all the more important to ensure transparency. Yet, the FDA closed its pilot program in March 2020, concluding that inconsistencies in international laws and regulations create oppressive costs to facilitating public access to drug approval data [32]. Its recommendation was for the establishment of a standardised, independently-managed international library system that operates on requests and voluntary provision of data by sponsors [32]. However, these recommendations seem unrealistic, given that the discontinued program clearly demonstrated sponsors’ lack of interest in sharing data.

5.Other regulatory agencies

Other international regulators are unlikely to have had access to TADS SAE data in their approval processes concerning paediatric use of fluoxetine, since it was not conducted by a pharmaceutical company and post-dated the studies used by Lilly to apply for licensing [35]. Regulators typically receive clinical study reports from companies. While it is routine practice for such reports to be created in clinical trials, so far as we can ascertain, none were generated for TADS. Nevertheless, we have made applications to the European Medicines Agency, Health Canada and Australia’s Therapeutic Goods Administration (TGA). The European Medicines Agency and Health Canada only accept Freedom of Information requests from their citizens and we are pursuing these options with assistance from our collaborators.

6.Conclusions

In the interests of patient safety, we have energetically pursued SAE reports from TADS to redress the lack of reporting of harms that might have a significant impact on clinical decision making. Despite the fact that we had executed appropriate data use agreements, both Duke and Lilly have withheld SAE data, and the FDA is not facilitating the timely provision of data. Our own University declined to provide legal support to challenge Duke’s decision to deny us access to the SAE forms. This experience exposes potentially fatal flaws in a regulatory system whereby independent analysis of the harms of drugs is denied, primarily under the guise of purportedly protecting participants’ rights.

There is an imperative, particularly with increasing rates of psychotropic prescribing to children and adolescents [36–40], to ensure access to all information that might bear on the safety of their medication. Duke’s arguments around participant confidentiality do not stand up against this ethical responsibility when it is TADS patients and their peers who have the greatest investment in the outcome from such scrutiny. Given that placebo-controlled trials in children are sometimes considered unethical [41], it is unlikely that similar studies will be able to test the safety of fluoxetine, underscoring the importance of maximising the use of existing scientific data.

Data must be available in a form that facilitates meaningful reanalysis, else public data access is merely tokenistic. The 2010 CONSORT statement emphasises that ‘readers need information about the harms as well as the benefits of interventions to make rational and balanced decisions’ [42, p. e25]. Government organisations must therefore regulate for safety over protecting patents and profits. The availability of regulators’ drug approval data to the broader scientific community is essential for safe and effective medical practice. Patients and their families have a right to full information on the potential harms of fluoxetine.