The Evolving Definition of Comparative Effectiveness Research: Lessons for Researchers Planning and Designing a Clinical Development Program

Editorial

Among the many challenges that must be overcome in successfully developing a novel drug treatment is to understand and prepare evidence that will ultimately be required by regulators and reimbursement authorities. Payers and other stakeholders often state the need for evidence of the comparative effectiveness of new treatments, in some cases arguing that this evidence should be a standard requirement for market access and reimbursement. Comparative effectiveness research (CER) is a method of developing evidence of the clinical and cost effectiveness of true alternative interventions, potentially including long-term data that assess patient- centered outcomes and evidence that a new treatment actually changes clinical practice. Conceiving of and executing a plan for developing CER to support these evidence requirements is therefore a substantial challenge, one made even more difficult by evolving notions of CER and uncertainty in what constitutes sufficient evidence of comparative effectiveness. In fact, CER as a methodology is rapidly evolving, incorporating new sources of real-world and big data and applying cutting edge analytic techniques including novel statistical methods and machine learning [1]. This review aims to analyze the evolving conception of CER in order to support researchers and others in the life sciences industry in planning and designing a clinical development program to meet potential future requirements for CER that may increasingly determine commercial success.

Early definitions of CER are often associated with the heightened attention and new funding that came with the American Recovery and Reinvestment Act (ARRA) of 2009 and with the establishment of the Patient-Centered Outcomes Research Institute (PCORI). Policy documents that established this view of CER argued for the importance of large prospective studies that could definitively establish the best (most effective) course of treatment [2, 3, 4]. More recently, definitions of CER have evolved to become more operational than theoretical, describing CER as a tool for a specific purpose; namely, to provide the specific evidence needed by clinicians, patients, policymakers, health plans, and other payers to make specific treatment and resource allocation decisions [5, 6]. One example that arguably fits the original conception of CER is the widely-disseminated study of the comparative effectiveness of percutaneous coronary intervention (PCI) versus drug therapy alone in patients with stable angina (COURAGE study) [7, 8]. In this study, a modest symptom benefit of PCI relative to drug therapy was observed, but no evidence was found for improved survival or reduction in the rate of myocardial infarction, an absence of effect that persisted over 15 years of follow up [9]. The observed clinical benefit meant that PCI demonstrated comparative clinical effectiveness, while the small magnitude of the benefit and its incremental cost meant that it was not cost effective at typically accepted thresholds [10]. Moreover, the effect of this new evidence was shown to have a real impact on clinical practice [11]. The full potential of CER is exemplified by studies such as COURAGE (and its successor, ISCHEMIA) [12], and by several others, [13, 14, 15], however, the number of clear-cut success stories for large centralized comparative clinical trials has arguably been modest, especially considering the vast number of treatment and resource allocation decisions where evidence is sorely needed. An emerging definition of CER is quite distinct from the approach taken in these large, broadly inclusive clinical trials. Real-world, observational data often provide evidence of comparative effectiveness, potentially from a range of sources designed to meet the needs of different decision makers, and thus can provide incremental evidence, often supportive or correlative rather than independently definitive. Widely-accepted definitions of real-world evidence are broader than the original scope of CER, including for example electronic health records and medical chart reviews, administrative data, and surveys [16]. While prospective randomized controlled trials are undoubtedly the gold standard for establishing safety and efficacy, observational data sources have the potential to meet pressing needs of decision makers: real-world PubMed Article Counts (2000-2016)

Source: Analysis Group Research. The NCBI PubMed database was searched using the terms “real world” and “comparative effectiveness” in title and abstract fields on 7/25/2017.

The number of comparative effectiveness clinical trials in PubMed has also peaked (Figure 2), with roughly the same number of registered CER studies initiated in 2016 as earlier in the decade. One interpretation of this data is ClinicalTrials.gov Study Counts (2000-2016)

Source: Analysis Group Research. Clinicaltrials.gov was searched using the term "comparative effectiveness" on 7/25/2017. The implication for researchers is that there is a growing reliance on real-world observational studies to supply evidence of comparative effectiveness to meet the needs of decision makers. Powerful, traditional CER study designs (large, broadly inclusive prospective interventional studies comparing two alternative interventions) may find use primarily for “big picture” questions, typically highly prevalent conditions that attract academic interest and substantial government funding. While prospective interventional studies undoubtedly provide valuable data with superior internal validity, the future of CER will reflect the accessibility of observational study designs and the growing power of large databases used for observational studies (aka, big data). The 21st Century Cures Act has been approved with bipartisan support in the lame-duck session of the 114th United States Congress. This legislation includes substantial changes to regulatory processes for drugs and medical devices, including provisions for the use of real- world observational data to support initial FDA approval and to satisfy post-marketing commitments. Clearly, there is a movement towards a more pragmatic application of real-world data to meet the needs of both regulators and of payers. CER, as it was conceived a decade ago, was framed in a context where the availability of real-world data and methods for analysis were relatively limited, and thus the place of prospective clinical trials to assess comparative effectiveness was central. In the next decade, as the availability and interconnectedness of real-world data increases, comparisons of clinical and cost effectiveness using real-world data are likely to become the standard, with growing influence on the commercial success of new health technologies.