Background

Intuitively, the pattern of tumor size changes is indicative of the therapeutic effects of novel anti-cancer regimens. Currently, tumor response and disease progression are based on Response Evaluation Criteria in Solid Tumors (RECIST) criteria, which were initially developed in the mid of 1990s (RECIST 1.0) to standardize and simplifies response criteria and then modified in 2009 (RECIST 1.1) in the context of cytotoxic treatments [1]. Regulatory authorities accept RECIST as an appropriate guideline for these assessments. Tumor response assessments, e.g., tumor shrinkage (response) or growth (progressive disease), are categorized based on pre-specified thresholds and essentially only capture a snapshot from the entire tumor growth kinetics. For example, a partial response (PR) in target lesions requires at least a 30% decrease from baseline in the sum of diameters of target lesions while progressive disease (PD) requires at least a 20% increase in the sum of diameters of target lesions from the smallest sum on study (nadir). However, such use of tumor size measurements is far from efficient and oversimplifies the complex trajectory of tumor size kinetics.

The concerns over the appropriateness of RECIST in measuring treatment benefit and predicting long-term outcomes in the era of immunotherapy have led to some recent efforts in developing alternative efficacy endpoints based on continuous tumor size data collected over time. For example, continuous tumor measurement–based metrics have been explored in terms of the absolute or relative changes in tumor size [2–7], the averaged overall assessments [8], and time to tumor growth using a mechanistic tumor growth model [9]. These efforts mainly focus on evaluating if short-term (6, 8 or 12 weeks) tumor size kinetics are sufficiently predictive of Overall Survival (OS) such that they may be considered for alternative early-stage efficacy endpoints. For traditional chemotherapy and target therapy trials, it has been demonstrated tumor size changes are predictive of OS [4–12], but do not provide any predictive improvement over traditional RECIST response evaluation criteria (e.g. trichotomized response status) [6–8].

Despite the robust progress in the development of immunotherapies, additional intermediate endpoints may be needed to detect signals of early anti-tumor activity, prioritize compound development or combinations, and interpret early phase study results. With an increasingly competitive and dynamic immune-oncology space, advancing successful therapies and terminating unsuccessful therapies earlier is crucial. In early oncology drug development, RECIST-based Objective Response Rate (ORR) and Disease Control Rate (DCR), and Early Tumor Size Changes (ETSC) are commonly used as early efficacy metrics for early decision-making. The predictive value of these metrics has been demonstrated for traditional chemotherapy and target therapy trials [4–12]. However, for immunotherapy trials, it is unclear whether all these early efficacy metrics are still predictive of long-term clinical benefit [13].

In the era of immunotherapy, it is recognized that the tumor size kinetics are not well captured using the same RECIST categories of tumor responses and disease progression. For example, immune checkpoint inhibitors have exhibited unusual response and progression kinetics, such as durable response, durable stable disease, “pseudo”-progression and/or delayed response. All these patterns are associated with favorable survival [14, 15]. Additionally, the immunotherapy treatment effects on Progression-Free Survival (PFS) are not reliable predictor of OS and survival benefit could be observed in the absence of PFS effect [16]. The unique patterns of response, progression, and survival with immune checkpoint inhibitors in melanoma, lung cancer and other cancers lead to renewed interest in exploring novel intermediate endpoints to obtain an early signal of efficacy and assist in go/no-go decision making [16, 17]. Intuitively, given the atypical pattern of response in patients treated with immunotherapies, continuous tumor sizes measurement instead of RECIST categories may be more sensitive to the underlying dynamic patterns of tumor responses, and thus may be more likely to inform down-stream clinically meaningful events such as OS. While ETSC have been explored for chemo and target therapies [18–22], very few analysis results on ETSC are available based on large immunotherapy datasets.

In this paper, we investigated if ETSC as a continuous measurement and RECIST-based response status were predictive of long-term survival benefit in checkpoint blockade based immunotherapy studies in metastatic melanoma, evaluated if tumor size changes provided any predictive improvement over RECIST based efficacy metrics, and explored the optimal cut-off of ETSC with external validation. The goal is to identify appropriate early efficacy metrics that can be predictive of survival for immunotherapy trials.

Discussions

Due to the unique response pattern of immunotherapeutic agents, there is a need to further explore intermediate endpoints that would better serve as a surrogate endpoint for OS in assessing the clinical benefit and accelerate the drug approval process. We explored if ETSC and early response status were predictive of long-term survival benefit based on 3 melanoma studies with over 1800 patients receiving immunotherapeutic agents.

Our analyses showed tumor size changes were predictive of survival and early tumor size changes (e.g., Week 12) appeared to have reasonable predictive value than other later time points for immunotherapy trials. However, in agreement with earlier research in traditional chemotherapy and target therapy trials, these metrics do not provide any predictive improvement over traditional RECIST response evaluation criteria for immunotherapy trials. The application of the tumor size change cut-off derived from training datasets to validation datasets is successful. This provides evidence not only for the feasibility of the application to other future trials but also the importance of the choice of early time points. Approximately 10% tumor shrinkage for target lesions at an earlier time point (e.g., Week 12) may be a reasonable efficacy screening cut-off while RECIST-based efficacy metrics remains the primary tool, especially for trials with patients in later line of therapies (e.g., 2L+) and when an early decision is desirable. In practice, later time point landmark analysis has limited utility for early efficacy signal detection due to longer follow up time requirement, and relatively smaller and selected patient population at later time points. Under the situation when patient enrollment is fast, early landmark analysis is especially useful for early decision-making and may help program planning and speed up development to get to market earlier. The c-index is a measure of accuracy and serves to ascertain the predictive ability of a given model/metric. However, it is important to note the discriminatory ability of one or more factors for a given outcome depends strictly on the clinical/biology context being considered, as well as on the structure of the model being tested.

Alternative cut-offs have been explored in cancers treated with chemo and target therapies [16, 18–22]. Krajewski et al. [18, 19] identified and validated a 10% tumor shrinkage as a reliable early predictor of OS with a total of 133 mRCC patients receiving VEGF-targeted therapies. Sakamaki et al. [20] suggested a time point of 8 weeks and a cut-off value of 20% are optimal criteria for defining early tumor changes based on 67 patients with metastatic colorectal cancer treated with first-line chemotherapy. Luo et al. and Lamarca et al. [21, 22] identified a 10% tumor reduction to predict progression-free survival based on 33 patients with NETs treated with SSA, and 237 patients with advanced pancreatic neuroendocrine tumors treated with sunitinib, respectively. Based on over 1800 melanoma patients receiving immunotherapeutic agents from 3 studies, we separated data into training and validation datasets. Our analysis suggest 10% tumor reduction at Week 12 may be a reasonable cutoff to be predictive of OS and patients with more than 10% reduction at Week 12 tend to have a longer survival, as well as higher chance of achieving RECIST PR/CR. Compared to the traditional 30% reduction cut-off, 10% reduction as cut-off appears to have higher sensitivity. While the PPV of OS seems higher using traditional 30% reduction cut-off, patients with tumor reduction between 10 and 30% still have good chance of achieving PR/CR and longer survival time. Therefore, it may not be necessary to use a stringent cutoff to serve as early efficacy signal detection. Compared to 20% increase as cut-off, 10% reduction as cut-off appears to have a high specificity and high PPV of OS. Ten percent tumor reduction, which is between 30% tumor reduction for PR/CR and 20% tumor increase for PD, may be the sweet spot to be predictive of OS, also maintaining reasonable sensitivity and specificity.

While 10% reduction at week 12 may be a reasonable efficacy screen cut-off from the perspective of early decision-making and program planning in drug development, patients should be encouraged to stay on treatment until disease progression in case of less than 10% reduction after 12 weeks. Because of the possibility of immunotherapy-related flare, patients who show initial radiographic progression, if they are clinically stable, may continue on treatment at the discretion of the investigator.

ETSC and RECIST-based trichotomized response status should be considered in go/no-go decision process for immunotherapy trials. Based on 3 studies we analyzed, 72% of those patents with more than 10% reduction at Week 12 achieved a RECIST PR/CR, while only 6% of those without a 10% tumor reduction at Week 12 had a RECIST PR later. Approximately 10% tumor reduction may help identify more potential responders, critical for early decision-making process, program planning and design of late-stage trials. Advancing promising therapies earlier is crucial especially within the highly competitive immune-oncology space. ORR and DCR are simple, but the predictive value may decrease with the number of prior therapies. By considering certain amount of tumor reduction (e.g., 10 to 30%) also leads to PR/CR later on and is related with longer survival, composite response rate which takes SD as partially positive outcome and utilize different weights (or utility) for PR/CR and SD, may be of value for go/no-go early decision as well. For example, while a PR/CR is worth a full weight of 1, SD may be worth a partial weight of 0.5 given that approximately half of the patients had a PR later based on 3 studies we analyzed. The optimal weighting strategy that best predicts OS is a research topic beyond of the scope of this paper.

Our analysis has strengths such as the fact that all these 3 studies have long-term survival data, in addition to the quality-assurance of these data for drug registration purpose. Our recommended 10% tumor reduction cutoff is based on separate training and validation datasets, and also validated by different statistical classification approaches. One limitation of our study was inclusion of only one tumor type. Whether our findings in melanoma patients can be generalized to other cancer types should also be explored. Second, our findings are associated with checkpoint blockade based immunotherapy (e.g., pembrolizumab, ipilimumab). Other types of immunotherapy (e.g., oncolytic viruses) may have different treatment response pattern. Therefore, our findings specifically apply to checkpoint blockade based immunotherapy in metastatic melanoma and whether they can be generalized to other immunotherapies is unknown. Another limitation of this research (same as other research using continuous tumor size as the early efficacy metric) is that the early efficacy metrics would only include the numerical values from the size of the target lesions but it would not include data such as response of the non-target lesions or the appearance of new lesions. Further research into other methods that potentially address the whole tumor burden including target, non-target lesion and new lesions should be investigated to better account for therapeutic effects on long-term survival that may be observed in immunotherapy trials.