Published in The British Medical Journal - 19th July 2021

Strengths and limitations of this study

  • This study uses a comprehensive measure for clinical transparency, which assesses the trial registration, results reporting, publication, FDAAA compliance and patient level data sharing practices among pharmaceutical companies, novel drugs and biologics—not merely the usual crude measure of whether companies report results for trials they registered on ClinicalTrials.gov.

  • This study uniquely assesses, for the first time, variations in transparency and data sharing practices by biopharmaceutical company size, location and sponsored product type, and includes a focus on biologics.

  • Companies included in the sample were given the opportunity to validate data associated with their approved products, and a 30-day amendment window to improve their data sharing procedures to meet our measures, as such, generalisability may be limited.

  • Non-large companies are new to the Good Pharma Scorecard and were less responsive to our outreach efforts which may have hindered their ability to improve their procedures and scores.

Introduction

Clinical trial transparency, including trial registration, results dissemination and even data sharing, is becoming the norm in research, with clear benefits for patient care and drug and vaccine development.1 2 Wide access to clinical trial data and results helps clinicians make better prescribing decisions, payers make reimbursement decisions, researchers reproduce, synthesise and build on findings, and funders avoid unnecessary and duplicative research.1–5 Further, human studies are ethically justified largely by their potential to advance generalisable knowledge and the common good but cannot fully realise this goal if results and data are not shared. Finally, transparency can also help build public trust in research findings, a particularly salient consideration today as novel SARS-CoV-2 vaccines reach marketing authorisation and approval and vaccine hesitancy challenges.6–8

Since 2015, the Good Pharma Scorecard (GPS) initiative has published and applied a suite of measures, developed through a multistakeholder deliberative process, to evaluate clinical trial transparency among large pharmaceutical companies with respect to their newly approved drugs.9–11 The Scorecard has proven effective at tracking transparency practices longitudinally and catalysing improvements. For instance, our previous study assessing data sharing practices among large pharmaceutical companies with drugs approved by the US Food and Drug Administration (FDA) in 2015 found moderate initial adherence to our data sharing measure (median score was 63% and 1/4 of companies achieved perfect scores), which improved after companies were offered a 30-day amendment window to meet our GPS measure (median final score rose to 80% and 1/3 of companies had perfect scores).11 Further, our previous study found transparency among large companies is improving; the median proportion of patient trials with publicly available results within 1 year of FDA approval increased from 87% for 2012 FDA approved drugs to 100% for 2015 approved drugs).11 However, variability in practices across large companies and substantial room for improvement persist.9–13

Previous studies have identified associations between research transparency and trial funding type (government vs industry),14–16 trial phase, results significance, sample size17–19 and condition treated.18 20 21 One study, focused on companies’ data sharing policies, found larger companies have more complete policies than smaller ones.13 However, to our knowledge, no study has assessed associations between pharmaceutical company characteristics, such as size, headquarter location (ie, US vs non-US) and sponsored product type (ie, biologics vs drugs) with a comprehensive measure for clinical trial transparency, which includes FDA Amendments Act of 2007 (FDAAA) implementation, data sharing, and trial registration and results reporting.

To address these gaps, we expanded the GPS from evaluating only large companies and their approved novel drugs to include companies of all sizes and biologics. We also newly analyse variations in transparency practices by product type, company size and company headquarter location to help fill gaps in knowledge around the role of these factors in transparency performance. This analysis expansion should help provide a more comprehensive understanding and tracking process of biopharmaceutical companies’ clinical trial transparency performance, given large companies only sponsor about half of all novel drugs approved each year and healthcare now increasingly involves biologics and products sponsored and manufactured by non-US-based companies.9 22 23

Methods

This study assesses the transparency of clinical trials supporting approval of novel drugs and biologics by the FDA in 2016 and 2017, using a series of measures related to trial registration, results reporting, FDAAA implementation and data sharing. We also rank pharmaceutical companies according to their performance on these transparency measures and assess company characteristics associated with better transparency.

Data sources

Following previously published methods,9–11 we gathered data from , a publicly accessible database containing records of FDA regulatory decisions; 39 trial registries including ClinicalTrials.gov, corporate registries and the WHO’s International Clinical Trials Registry Platform (which aggregates 16 country registries); journals indexed in PubMed, Google Scholar and EMBASE; corporate press releases and websites; data repositories (such as clinicalstudydatarequest.com and yoda.yale.edu); and personal communications with product sponsors.

Products and company sample

We included new therapeutic biologics and novel drugs approved by the FDA in 2016 and 2017, identified from Drugs@FDA.24–26 Novel drugs are defined as new molecular entities (NMEs) or new combination drugs containing at least one NME component. New therapeutic biologics exclude biosimilars. For the 2016 sample, we confined our analysis to drugs and biologics sponsored by the 20 largest companies measured by their 2016 market capitalisations.27 28 Companies in the top 20 largest companies by market capitalisations are considered large companies throughout this analysis. All other companies are considered non-large. Subsidiaries were linked with parent companies by searching corporate websites, press releases and SEC filings. As part of our annual scope expansion of the GPS, the 2017 sample also includes new drugs and biologics sponsored by non-large companies.

Trial samples

For each product in our sample, we created three trial samples: (1) ‘all trials,’ (2) ‘patient trials’ and (3) ‘FDAAA applicable trials,’ in keeping with our previous methods. The ‘all trials’ sample contains all trials submitted to the FDA for initial approval of each product (ie, all trials in an approved new drug application (NDA)). The ‘patient trials’ sample contains only trials in the targeted patient population for the approved indication (excluding, eg, trials conducted in healthy volunteers). ‘FDAAA applicable trials’ are those highly likely to be subject to FDAAA trial registration and results reporting requirements, generally phase II and III controlled trials begun after 27 September 2007 or ongoing as of 26 December 2007 that (1) have at least one US site; (2) were conducted under an FDA investigational new drug application; or (3) involved a drug, biologic or device manufactured in the USA and exported for research.29

Data collection

FDA approval packages for each product were reviewed to extract every clinical trial supporting initial approval of each product, along with available trial characteristics, such as identification number, location, enrolled participants, phase, type and condition studied. We then searched ClinicalTrials.gov to determine whether these trials were registered and had reported results, using our previously published search and matching techniques, and extracted further trial characteristics.9–11 If we could not find a trial registered in ClinicalTrials.gov, we searched international and corporate registries registrations. We also reviewed the medical literature for publication of each trial, using at least three trial characteristics for matching along with product names, recording the earliest publication date available. Lastly, we abstracted data sharing policies from each product sponsor’s website. If there was no policy on a company’s website, we also searched its trial repository website (such as www.clinicalstudyreport.com).

At least two research assistants, trained by JM, extracted each data point, working independently, with discrepancies resolved through discussion and consensus. Databases were accessed between January 2017 and March 2019, with data validated and finalised between March 2020 and June 2020.

Patient and public involvement

Patients and other stakeholders were involved in the original development of the transparency measures used in this study, including 10 non-industry data sharing experts (academics, regulators, medical journal editors and trial repository experts), representatives from 11 pharmaceutical companies and 12 patient representatives. As previously published, we identified patient groups based on the relevance of our work to theirs (ie, because the conditions treated by our cohort of drugs were responsive to them) and independence from industry. We provided financial support so funding was not a barrier to participation. Going forward, we aim to convene our semiannual multistakeholder meeting in 2021 with patients, regulators, academics, healthcare professionals, ethicists and industry to disseminate results, in keeping with our methods from the past several years, and discuss priority setting for future iterations of the GPS. Furthermore, we have partnered with Scientific American to disseminate and amplify summaries of these findings for a wider public audience.

Outcome measures

Transparency measures, product level

We examined three outcome measures for the trials supporting each product’s approval. The first pertains to trial registration: we determined whether trials in the ‘all trials’ and ‘patient trials’ samples for each product were registered within 6 months of initial FDA approval of each product. Second, for trials completed by a product’s FDA approval, we determined whether results were reported in a public registry or published in a journal indexed by PubMed, Google Scholar or EMBASE within 6 months of initial FDA approval. Adhering to our previous methods, we excluded expanded access and observational trials from our review of whether results were publicly available for the ‘patient trials’ sample. Third, we examined FDAAA implementation—that is, whether applicable trials were registered within 21 days of their start date and results reported within 30 days of initial FDA approval of each product (we gave sponsors a 7-day grace period).

Data sharing measures, company level

We examined companies’ data sharing practices using five previously developed measures9–11: (1) whether they had a public policy committing to sharing analysis-ready datasets and clinical study reports (CSRs) for applicable studies; (2) whether their policy explained how such data could be requested; (3) whether the policy committed to making data available by 6 months after approval by the FDA or European Medicines Agency or 18 months after a trial’s completion date, whichever was later; (4) whether the company reported the number of data requests received and how each was handled (granted or denied); and (5) the proportion of ‘data sharing applicable’ trials registered in a public registry. For outcome measures 1–4, companies received a score of 0 for a no and 100% for a yes, while measure 5 could range from 0% to 100%. The overall data sharing score for each company is the average of the five component scores.

Scoring companies on their overall transparency

Lastly, we determined an overall company transparency score following our previous methodology.9–11 For companies with only one product approved by the FDA in 2016 and 2017, we averaged their scores on their (1) patient trials analysis, (2) FDAAA compliance and (3) data sharing analysis. Each component was weighted equally for consistency with past GPS analyses, and because each component is essential to achieving the full benefits of transparency.9–11 For companies with multiple products approved, we pooled the trials from all their products into our three trial samples and then applied our outcome measures to the pooled trial samples. We then calculated an overall score by averaging the pooled components (see table 1).

Table 1

Summary of transparency measures

Analysis

Descriptive statistics were calculated for all outcome measures (median and IQR) on both the product and the company level. For each product, we determined the proportion of ‘all trials’ and ‘patient trials’ publicly available and the proportion of ‘FDAAA applicable trials’ that were FDAAA compliant. We also determined the proportion of products and companies scoring 100% on each outcome measure. Companies were ranked based on overall transparency scores, from highest to lowest.

We used Mann-Whitney U tests to examine associations between our outcome measures and the categorical characteristics of company size (large vs non-large), product type (drug vs biologic) and company headquarter location (US vs non-US). Remaining consistent with previous GPS analyses, large companies were defined as those in the 20 largest by market capitalisations; all other companies were categorised as non-large. Results less than 0.05 significance level are described as statistically significant. Analyses were conducted in Microsoft Excel V.15.11 (Redmond, Washington, USA) and R V.3.5.1.

Validation and amendment window

We shared the raw data underpinning our analyses and our findings on the product-level measures with each company for validation purposes. Companies had at least 30 days to amend their procedures to meet our data sharing measures and request error corrections in our data. Error corrections were made if confirmable through public data sources. In the rare case where the company sponsoring a new drug or biologic application to the FDA stated it did not have control over a trial’s data during our study period, we reassigned responsibility to the company named as controlling these data (ie, a trial’s sponsor) if that company confirmed responsibility and data control in writing. Each company was contacted at least twice. We report the number and proportion of companies responding to our data validation requests in total and by company size. We also report the number of companies opting into our 30-day amendment window and specific changes made, if any.

Results

Sample characteristics

We analysed 62 products (40 novel drugs and 22 biologics) treating 56 unique conditions, sponsored by 42 companies (17 large and 25 non-large). Twenty-six companies were headquartered in the USA and 16 elsewhere (table 2).

Table 2

Sample characteristics

Collectively, these products were approved based on 1017 trials involving more than 187 000 participants. Of these trials, 38% (391/1017) were conducted in the targeted patient population (‘patient trials’) for the approved indication and 23% (236/1017) were subject to FDAAA. A median of 13 (IQR 8–21) trials supported FDA approval of each product, with a median of 5 trials (IQR 3–8) per product conducted in the targeted patient population (‘patient trials’) for the approved indication. Each product had a median of 3 (IQR 2–5) FDAAA applicable trials (table 2).

Product-level transparency

We found 26% of products (16/62) had publicly available results for all trials supporting their FDA approval, which rose to 67% (39/58) when we narrowed our sample to just ‘patient trials’, that is, trials conducted in patients for the approved indication. Fifty-eight per cent of products (32/55) were fully FDAAA compliant; all of their applicable trials complied with FDAAA registration and results reporting requirements.

Of note, 11% of products (7/62) had no FDAAA applicable trials subject to results reporting at the time of their approval. Two of these seven products were manufactured by US-based companies but were approved based on ongoing trials not yet subject to results reporting under FDAAA. The other five products were manufactured by non-US-based companies and were approved based on trials conducted entirely outside the USA or ongoing trials.

Further, 6% (4/62) of products had no completed ‘patient trials’ when they were FDA approved, meaning the FDA approved them based on interim analyses from ongoing trials that had not reached their primary completion date. All four of these products were for oncology.

The median product-level transparency score was 62% (IQR 36–95) for the ‘all trials’ sample, 100% (IQR 83–100) for the ‘patient trials’ sample and 100% (IQR 71–100) for FDAAA compliance (table 3).

Table 3

Transparency of novel drugs and biologics approved by the FDA in 2016 and 2017

Supplemental material

Company-level transparency and data sharing

Seven of the 42 companies (17%) scored 100% overall; they had publicly available results for all their patient trials, were fully FDAAA compliant and fully met our data sharing measures (table 4). Examining the component measures, 58% of companies (23/40) had publicly available results for all patient trials, 42% (16/38) were FDAAA compliant and 26% (11/42) fully met our data sharing measure. Median company scores for public availability of results for patient trials, FDAAA implementation and data sharing were 100% (IQR 80–100), 88% (IQR 50–100) and 69% (IQR 20–100), respectively (table 4).

Table 4

Overall transparency scores for companies with novel drugs or biologics FDA approved in 2016 or 2017

Validation and amendment window results

Smaller companies were less responsive to our outreach than large companies, offering an opportunity to correct data errors and improve data sharing practices within our amendment window (21% participation by non-large companies vs 94% by large companies). Four companies (4/42, 10%) improved their data sharing procedures to meet our measures during our amendment window, raising the median data sharing score for companies from 60% (IQR 20–80) to 69% (IQR 20–100) after the amendment window (online supplemental table 1).

Radius added a new policy to its website committing to sharing analysis-ready datasets and CSRs by our deadline and explaining how such information could be requested; initially they did not have a public data sharing policy. Radius’s data sharing score thus improved from 20% to 80%. Takeda newly committed to sharing data by our deadline, instead of only after trial publication, increasing its score from 80% to 100%. Shire newly began reporting the number and outcome of received data requests and added a new commitment to share data by our deadline, raising its data sharing score from 60% to 100%. Merck KgaA/EMD Serono amended its policy to share data by our deadline, improving its data sharing score from 80% to 100%.

Associations between company characteristics and transparency

Company size and location

Large companies had a higher overall median transparency score than non-large companies (median 96%, IQR 91–100 vs 59%, IQR 41–70, p<0.001) (table 5), driven by higher FDAAA compliance (median 100% (IQR 88–100) vs 57% (IQR 0–100), p=0.01) and better data sharing (median 100% (IQR 80–100) vs 20% (IQR 20–40), p<0.001). Only three non-large companies—Takeda, Ultragenyx and Radius—scored above the median company score of 73% (IQR 54–95) (table 4).

Table 5

Bivariate associations of company characteristics with clinical trial transparency measures

There were no statistically significant differences by company size in the public availability of results for the patient trials or all trials samples. There were no significant differences on any of our measures by company headquarter location (US vs non-US) (table 5).

Product type

There was a statistically significant difference between biologics and drugs in the public availability of results for all trials (median 85% (IQR 62–100) for biologics vs 47% (IQR 32–82) for drugs, p=0.005), but not for patient trials or FDAAA compliance (table 5). Notably, most biologics (19/22) were developed by large companies.

Discussion

This study evaluated companies on their clinical trial transparency, assessing results dissemination, FDAAA implementation and data sharing practices for their novel drugs and biologics approved by the FDA in 2016 and 2017. Novel to this analysis, compared with past GPS analyses and other studies, is the addition of biologics and companies of all sizes, important expansions as large companies only sponsor about half of all novel drugs approved annually and the proportion of biologics among new FDA approvals is increasing (up 2.8% in 1995–1997; 14.0% in 2005–2007; and 27.5% in 2015–2017).22 We also analysed differences in transparency performance among US versus non-US-based companies, because FDA-approved products are now often sponsored or manufactured by non-US-based companies.23

We found about one-quarter of reviewed products had publicly available results for all trials supporting their approval within 6 months of FDA approval; this rose to about two-thirds when we focused just on trials conducted in the targeted patient populations for the approved indication. Roughly three in five products fully complied with FDAAA reporting requirements. About one-quarter of companies met all of our transparency measures.

Smaller companies were significantly less likely than larger companies to comply with FDAAA reporting requirements and have public data sharing policies. Within both size groups, there was substantial heterogeneity in practices and room for improvement. We found nearly two in five products in our sample were sponsored by non-US-based companies, with no meaningful differences in transparency performance among US versus non-US-based companies.

Juxtaposing our results to our previous analyses of the public availability of clinical trial results for drugs approved in 2012, 2014 and 2015, which were limited to large companies, we found sustained improvement in practices.9–11 The median proportion of trials in patients, per product, with publicly available results at 12 months after FDA approval increased from 87% for drugs approved by the FDA in 2012 to 100% for drugs approved by the FDA in 2015 and remained at 100% for 2016 and 2017 drug approvals.9 11 Median data sharing scores among large companies rose from 80% for 2015 approvals to 99% for 2016, and 100% for 2017 approvals.9

The finding that large companies are more transparent than smaller ones is not surprising and supports other study findings that larger companies have more complete data sharing policies and that companies sponsoring high volumes of trials are more likely to report trial results within FDAAA timelines.13 30 There are a number of reasons why smaller companies might lag behind larger ones in transparency, such as resource limitations, smaller staffs and less experience with regulatory compliance, all of which suggest problems can be addressed. Our findings suggest large companies may benefit from auditing the transparency of smaller companies and requesting deficiencies be fixed before partnerships, mergers or acquisitions. Transparency deficiencies among large companies were often inherited from collaborating with smaller companies.

The finding that 42% of FDA-approved novel drugs and biologics fail to fully meet FDAAA reporting requirements suggests the FDA may benefit from more aggressive enforcement of this law. To date, the FDA has only issued one public notice of non-compliance, to Acceleron Pharma, around 28 April 2021, for failing to meet FDAAA reporting obligations and to respond to the FDA’s pre-notice of non-compliance sent in July of 2020.31 The FDA is authorised to seek civil money penalties from Acceleron for the FDAAA violation, including additional civil money penalties if it fails to submit the required information within the 30-day period. Despite several studies showing poor FDAAA compliance among drug companies, the FDA has yet to systematically penalise non-compliant companies.9–11 14 32

Further, although the European Medicines Agency and Health Canada release redacted clinical study reports after a drug has been approved, the FDA does not. In 2018, the FDA piloted a programme to release parts of CSRs for pivotal trials.33 However, it ended in March of 2020 with poor sponsor participation (Janssen, part of J&J, was the only sponsor that participated) and the FDA shifted its focus to producing new integrated review templates.34 Experts have argued the new integrated review templates have resulted in an overall net loss of information, rather than enhanced transparency, as they exclude information previously contained in the older approval packages released by the FDA. While the FDA reports exploring other approaches to increase the availability of data supporting approval decisions, concrete progress would better support research transparency and could, in theory, alleviate our need to evaluate and track some of the transparency measures in the GPS.

Lastly, our finding that 11% of products in our sample had no FDAAA applicable trials subject to results reporting at the time of their approval raises questions about whether FDAAA’s scope should be expanded to address the growing number of products approved by the FDA based on ongoing trials and trials conducted entirely outside the USA by non-US-based companies.

There are limitations to this work. First, company size was categorised dichotomously (large vs non-large) by market capitalisation; we did not evaluate associations by other measures of size such as number of employees, years in existence and the like. We selected market capitalisation because it is a simple metric of a company’s total value. This dichotomous categorisation, while practical, does not address differences within non-large companies. Additionally, we ranked the companies that submitted each product for FDA approval; sometimes these sponsors differed from trial sponsors. We made efforts to confirm with all companies that they had control of and could disseminate data, excluding trials from company scores when they did not. It is possible the companies at the bottom of the top 20 largest by market capitalisation are not significantly different than those just outside the top 20. Further, the differences in transparency performance among large and non-large companies may be partly explainable by the fact that this is the first year the GPS includes non-large companies. Perhaps as a result, smaller companies were less responsive to our outreach efforts and large companies have already improved their practices in response to being rated, which may have widened the performance gap between large and smaller companies. Although each company was contacted at least twice, longer-term efforts are needed to engage smaller companies with the GPS and make it a more effective reform tool, which we aim to do. There are a number of other factors that may impact transparency, such as PhRMA membership, company resources, and priority review or orphan drug designations. We did not evaluate the accuracy of shared data or results.

Conclusion

Evaluating pharmaceutical companies and their novel drugs and biologics approved by the FDA in 2016 and 2017 on a series of clinical trial transparency measures, we found substantial room for improvement particularly among non-large companies. Disseminating results and sharing patient-level data in research is critical for gaining the full and essential benefits of clinical research, honouring research participants, and fostering trust in medical research, medicines, vaccines and care. The trajectory over time is promising, but the arc must bend further towards transparency to fully realise the potential benefits of and trust in clinical research.

Data availability statement

Data are available in a public, open access repository. Extra data can be accessed via the Dryad data repository at http://datadryad.org/ with the doi:10.5061/dryad.r2280gbdb.

Ethics statements

Acknowledgments

The authors thank Christina Deutsch, Joshua Kimelman, Kianna Pierson and Rayne Kim for research assistance.

back