You are here

  • Home
  • Archive
  • Volume 8, Issue 1
  • Comparison of iRECIST versus RECIST V.1.1 in patients treated with an anti-PD-1 or PD-L1 antibody: pooled FDA analysis

Article Text

Article menu

Download PDFPDF

XML
Clinical/translational cancer immunotherapy
Original research
Comparison of iRECIST versus RECIST V.1.1 in patients treated with an anti-PD-1 or PD-L1 antibody: pooled FDA analysis
  1. Flora Mulkey1,
  2. Marc R Theoret2,
  3. Patricia Keegan3,
  4. Richard Pazdur3 and
  5. Rajeshwari Sridhara1
  1. 1 CDER/Office of Translational Sciences, US Food and Drug Administration, Silver Spring, Maryland, USA
  2. 2 Oncology Center of Excellence/CDER Office of Oncologic Diseases, US Food and Drug Administration, Silver Spring, Maryland, USA
  3. 3 Oncology Center of Excellence, US Food and Drug Administration, Silver Spring, Maryland, USA
  1. Correspondence to Dr Marc R Theoret; Marc.Theoret{at}fda.hhs.gov

Abstract

Background Response criteria developed when cytotoxic chemotherapy was the predominant therapeutic modality to treat patients with cancer, do not capture the full spectrum of tumor response patterns observed with anti-PD-1/PD-L1 antibody treatment. iRECIST was developed to capture both typical and atypical response patterns.

Methods Target, non-target, and new lesion measurements for 7920 patients receiving anti-PD-1/PD-L1 antibody (n=4751) or anti-CTLA-4 antibody (n=613) or undergoing chemotherapy (n=2556) from 14 randomized controlled trials submitted to the U.S. Food and Drug Administration were used to calculate the best overall response, objective response rate and progression-free survival (PFS) per iRECIST (iPFS) and Response Evaluation Criteria in Solid Tumours (RECIST). Associations between either PFS or iPFS and overall survival (OS) were evaluated using the method adopted by Oba et al.1

Results Among 4751 anti-PD-1/PD-L1-antibody treated patients, 31.5% (95% CI 30.2% to 32.9%) and 30.5% (95% CI 29.2% to 31.8%) achieved an objective response per iRECIST or RECIST V.1.1, respectively. OS among the 48 patients with objective response by iRECIST only resembled that in patients with responses per RECIST V.1.1. The association between iPFS and OS was R2=0.277 and that between PFS and OS was R2=0.260.

Conclusions Patients treated with anti-PD-1/PD-L1 antibodies with initial progressive disease per RECIST V.1.1 can experience prolonged stability or substantial reductions in tumor burden per iRECIST, atypical response patterns associated with prolonged OS. In the subgroup of patients with atypical responses, the application of iRECIST retrospectively in the evaluation of the objective response durations and the magnitude of PFS results in large differences compared with RECIST V.1.1. For the overall pooled population, the magnitude of these differences was modest, although a large proportion of patients had no further tumor assessments following RECIST V.1.1-defined progressive disease. Prospective studies employing iRECIST will be required to assess whether this response criteria more fully captures the benefit of immune checkpoint inhibitors.

  • oncology
http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See http://creativecommons.org/licenses/by-nc/4.0/.

https://doi.org/10.1136/jitc-2019-000146

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Introduction

    The U.S. Food and Drug Administration (FDA) has considered large durable treatment effects on tumor burden and large effects on progression-free survival (PFS) in randomized controlled trials (RCTs) to predict effects on overall survival (OS), a direct measure of clinical benefit.2 Because these endpoints can be evaluated earlier than OS to characterize a drug’s efficacy, and these endpoints are not confounded by crossover, clinical trials routinely assess treatment effects on PFS and objective response rate (ORR). Identifying the optimal algorithm to detect changes in tumor burden that correlate best with OS is particularly important in the regulatory setting to ensure that new, safe, and effective therapies are accessible to patients as soon as possible.

    The criteria to characterize treatment effects on tumor have evolved to maintain or improve accuracy while limiting administrative costs and other burdens. The first generally accepted bidimensional criteria, the WHO response criteria (1981),3 was replaced by unidimensional Response Evaluation Criteria in Solid Tumours (RECIST) (2000)4; the current standard is RECIST V.1.1,5 a widely used, standardized algorithm for characterizing tumor response and tumor progression in clinical trials. Investigators, the pharmaceutical industry, and regulatory agencies accept ORR, duration of response (DOR), and PFS as assessed by conventional response criteria (eg, RECIST V.1.1) as valid measures of clinically meaningful changes in tumor burden, describing treatment effects supporting some of the new drug approvals.6

    However, RECIST V.1.1 does not capture the atypical patterns of tumor response described with ipilimumab and anti-PD-1/PD-L1 antibodies. These atypical responses include initial increase in tumor size followed by a clinically important reduction in tumor burden (‘tumor flare’/pseudoprogression) or initial reduction in tumor size with appearance of new lesions that subsequently regress. To account for atypical tumor responses, several modified response criteria have been proposed, including the immune-related response criteria,7 immune-related RECIST,8 immune-modified RECIST,9 and immune RECIST.10 These criteria differ in their consideration of new lesions (ie, whether new lesions indicate progression and/or incorporation of new lesions in the measurement of tumor burden), requirements for confirmation of progression, and use of bidimensional or unidimensional measurements of tumor lesions (online supplementary table 1).

    Supplemental material

    Evidence to support whether these novel response criteria better assess outcomes in patients in trials evaluating immunotherapeutics (as compared with RECIST V.1.1) is limited. To provide insight into the relative performance of iRECIST, as immune-based response criteria are increasingly employed in cancer immunotherapy trials, we conducted a retrospective assessment of response (ORR, best overall response (BOR), and PFS) according to iRECIST and RECIST V.1.1.

    Methods

    Selection criteria

    All trials that evaluated the safety and efficacy of an anti-PD-1/PD-L1 antibody and were submitted to FDA between September 2014 and September 201711–24 were assessed for inclusion of a randomized active control and the potential opportunity to treat patients beyond initial RECIST V.1.1-defined progression. We identified 14 multicenter RCTs, 11 open-label and 3 double-blind, in patients with melanoma, squamous/non-squamous non-small cell lung cancer (NSCLC), renal cell carcinoma (RCC) and head and neck squamous cell carcinoma (HNSCC) meeting these criteria.

    Data extraction and analysis

    Patient-level data from investigator-assessed tumor measurements were used to calculate the response status at each time point according to RECIST V.1.1 and iRECIST (online supplementary table 2). Patients for whom there was only a baseline tumor assessment with no postbaseline assessments, who received non-protocol anticancer therapy prior to the first post-baseline assessment, or with all assessments identified as non-evaluable in the datasets were categorized as ‘not-evaluable’ in our analyses of ORR, a subcategory of non-responder.

    Definitions and outcomes

    The analyses of ORR were conducted in the ‘as-treated’ population defined as patients who received at least one dose of protocol-specified therapy. ORR was defined as the proportion of patients achieving a complete response (iCR/CR) or partial response (iPR/PR) per iRECIST or RECIST V.1.1, respectively. iBOR/BOR was defined as the single best response status at any evaluation assessment timepoint prior to receipt of non-protocol therapy or prior to progression of disease (PD) by RECIST V.1.1 and prior to confirmed progression of disease (iCPD) by iRECIST. In accordance with iRECIST, patients with stable disease (iSD) or better after an initial unconfirmed progression of disease (iUPD) were evaluated after iUPD for iCR/iPR/iSD in the determination of iBOR.

    Analyses of OS and PFS were conducted in the intent-to-treat (as randomized) population. OS was measured from the date of randomization until death; in accordance with FDA Guidance,2 data were censored at the data cut-off date specified in the clinical study report for each trial. Our analyses of progression-free survival (ie, iPFS per iRECIST and PFS per RECIST V.1.1) were calculated from the date of randomization to the date of disease progression by iRECIST/RECIST V.1.1 or death, whichever occurred earlier. In the analyses of PFS per RECIST V.1.1, patients not experiencing progression or death were censored as of their last tumor assessment prior to any subsequent non-protocol anticancer therapy. In the analyses of iPFS, patients with iUPD at their last assessment were assigned a date of progression at the earliest timepoint iUPD was consecutively determined for that patient (ie, sequential iUPD determinations without an intervening iSD, iPR, or iCR determination).

    Associations between survival and response status in subgroups of patients where responses per iRECIST and RECIST V.1.1 differed were analyzed and plotted separately.

    Statistical analysis

    Kaplan-Meier25 methods were used to estimate iPFS/PFS and OS. The association between iPFS/PFS and OS was evaluated using a weighted linear regression model with weights equal to the sample size of each comparison. The strength of this association was measured by the coefficient of determination (R2) from the weighted linear regression model, where values close to 1 represent strong association and those close to 0 represent lack of association. Estimated treatment effects of iPFS, PFS and OS were calculated as the log of the HR from an unstratified Cox proportional hazards model with study arm as the covariate. The analysis for this paper was generated using SAS software V.9.4. Survival figures were generated using R V.3.4.3 and RStudio V.1.1.456.

    Results

    Among the 8170 randomized patients, 4802 (59%) were randomized to receive an anti-PD-1/PD-L1 antibody; 642 (8%) were randomized to receive an anti-CTLA-4 antibody; and 2728 (33%) were randomized to receive chemotherapy. The as-treated population included 7920 patients receiving at least one dose of study-specified therapy, comprising 4751 (60%) anti-PD-1/PD-L1 antibody-treated patients, 613 (8%) anti-CTLA-4 antibody-treated patients, and 2556 (32%) chemotherapy-treated patients (figure 1).

    Figure 1
    Figure 1

    Consort diagram of analysis populations. OS, overall survival PFS, progression-free survival.

    ORR was 31.5% by iRECIST and 30.5% by RECIST V.1.1 for anti-PD-1/PD-L1 antibody-treated patients, for a difference in ORR of 1%. The differences in ORR by iRECIST and RECIST were even smaller for those treated with an anti-CTLA-4 antibody (19.7% vs 19.2%) or chemotherapy (15.2% vs 15%) (table 1). There were 232 (4.9%) of the 4751 anti-PD-1/PD-L1 antibody-treated patients who achieved an iCR or iPR (n=133 (2.8%)) or iSD (n=99 (2.1%)) after RECIST V.1.1-defined progression (table 2). Of these 133 patients with iPR/iCR, 85 (64%) also achieved a CR or PR prior to their RECIST V.1.1-defined progression, whereas 48 (36%) patients had stable (n=11) or progressive disease (n=37) as their best response by RECIST V.1.1 (ie, iRECIST-only responses) (table 3). The median DOR among all responding patients by iRECIST was 10.1 months (range: 0–33.4) and that by RECIST V.1.1 was 9.2 months (range: 0–33.3) (table 1). Among the 138 patients with a DOR that differed based on response criteria, the median iDOR was 14.1 months (range: 0–33.4) and the median DOR was 6.1 months (range: 0.9–24.9) (online supplementary figure 1).

    View this table:
    Table 1

    Food and Drug Administration-calculated ORR and DOR by iRECIST and RECIST V.1.1 by treatment group

    View this table:
    Table 2

    Best response pre-RECIST V.1.1/post-RECIST V.1.1-defined progression

    View this table:
    Table 3

    Concordance in best overall response achieved prior to use of non-protocol-specified anticancer therapy per iRECIST and RECIST V.1.1 in the anti-PD-1/PD-L1 antibody-treated population

    Among the 37 patients with iRECIST responses whose BOR by RECIST V.1.1 was PD, evidence of PD by RECIST V.1.1 was based on an increase in existing lesions in 57% of patients (target lesions increase in 11 patients (30%); increase in non-target lesions in 9 patients (24%), and increase in non-target plus new lesions in 1 patient (3%)) and on an appearance of new lesions in the other 43%. The spider plots summarizing the change in tumor burden over time by type of progression are presented in figure 2A–C. As illustrated in the spider plots, all PD events occurred within 14 weeks of initiation of treatment with a median time to iUPD of 8 weeks (min, max: 1, 14).

    Figure 2
    Figure 2

    Spider plots depicting change in tumor burden in 37 patients with PD by RECIST V.1.1 and iCR or iPR by iRECIST. (A) n=11 responders by iRECIST; black circle represents timing of progression based on increase in target lesions by RECIST V.1.1. (B) n=10 responders by iRECIST; black circle represents timing of progression based on non-target lesions by RECIST V.1.1. (C) n=16 responders by iRECIST; black circle represents timing of progression based on new lesions by RECIST V.1.1. iCR, complete response; iPR, partial response; RECIST, Response Evaluation Criteria in Solid Tumours.

    In the analyses of PFS, 62% of those randomized to anti-PD-1/PD-L1 antibodies were identified as having PD per RECIST V.1.1 and 59% per iRECIST. Of the 2832 patients with PD per iRECIST, 61% had iUPD assigned at the time of PD per RECIST and had no further disease assessments; 18% had iUPD with subsequent disease assessments that did not confirm PD (ie, iCPD not assigned); and 21% were subsequently identified as having iCPD. In total, there were 232 patients with PD by RECIST V.1.1 who had a longer duration of iPFS based on subsequent determination of iSD/iPR/iCR at a later timepoint, resulting in either censoring at the last assessment if ongoing (n=116, figure 3A) or in a later PFS event (n=116, figure 3B) based on iCPD (n=12), iUPD at the last assessment (n=93), or death (n=11); specifically, the respective median iPFS and median PFS durations were 18.7 (min, max: 1.3, 36.1) and 5.4 (95% CI 4.0 to 6.2) months in the iRECIST-censored subgroup and were 10.1 (95% CI 8.4 to 10.9) and 2.8 (95% CI 2.6 to 4.1) months in the subgroup with PFS events documented later by iRECIST. In the overall PFS analyses, the estimated median iPFS was 4.2 months (95% CI 4.0 to 4.4) compared with an estimated median PFS of 3.9 months (95% CI 3.6 to 4.1).

    Figure 3
    Figure 3

    Swimmer plot of response by n=232 patients with iRECIST status (stable disease or better) post RECIST V.1.1 PD. (A) n=116 patients with on-going response post RECIST V.1.1 PD. (B) n=116 patients having subsequent progression/death post-RECIST V.1.1 PD. iCPD, confirmed progression of disease; iCR/CR, complete response; iPFS, progression-free survival per iRECIST; iPR/PR, partial response; iUPD, unconfirmed progression of disease; PFS, progression-free survival; RECIST, Response Evaluation Criteria in Solid Tumours.

    In the analyses of OS by response status in all randomized patients and in anti-PD-1/PD-L1-treated patients (figure 4A,B), patients with responses only per iRECIST appear to have similar survival initially compared with those achieving PR or CR under RECIST V.1.1. Among the anti-PD-1/PD-L1-treated group, patients who achieved iSD after RECIST V.1.1 progression appeared to have similar survival initially to those achieving SD by RECIST with divergence of the curves at later timepoints. The estimated median OS time was 12.1 months (95% CI 8.9 to NR) for those with iSD only and 16.2 months (95% CI 14.9 to 18.2) for those with SD per RECIST V.1.1.

    Figure 4
    Figure 4

    (A) Overall survival by BOR among all 8170 randomized patients. (B) Overall Survival by BOR among anti-PD-1/PD-L1-treated patients. BOR, best overall response. iCR/CR, complete response; iPR/PR, partial response; iSD/SD, stable disease; NE, not evaluable; PD, progression of disease.

    The correlations of PFS per RECIST V.1.1 and per iRECIST with OS were analyzed in the evaluation of PFS (RECIST V.1.1) and iPFS (iRECIST) as a surrogate endpoint for OS. There was a minimal increase in R2 value for the association of iPFS and OS compared with that for PFS and OS (R2=0.277 vs R2=0.260) (online supplementary figure 2 a, b).

    Discussion

    Trials intended to demonstrate the efficacy of anticancer drugs and biologics commonly evaluate tumor measurement based endpoints—such as ORR with prolonged durations of response and PFS—as the primary outcome measure and, in the context of the clinical setting of the disease, the magnitude of the treatment effect, and the risk–benefit profile, may support an accelerated or regular approval.2 There is intense interest in the oncology community for use of immune-based response criteria rather than the conventional criteria, given the observed pattern of atypical responses and that conventional criteria are reported to underestimate the ORR for cancer immunotherapeutics by up to 15%.26

    Notably, the incremental percentage of immune-based responders by iRECIST is lower than that in previous reports using other immune-based criteria27 as our analysis did not consider patients with iSD following RECIST V.1.1 PD as responders, since this is not clear evidence of a drug effect and may represent the natural history of the disease in that patient.26 These analyses conducted across 14 clinical trials provides a large database in which ORR per iRECIST was similar to that per RECIST V.1.1 (31.5% vs 30.5%), a finding that was consistent across tumor types, with 2.8% of anti-PD-1/PD-L1 antibody-treated patients achieving an iCR or iPR after progression based on RECIST V.1.1. This includes the approximately 1% of patients with early RECIST V.1.1 PD who appear to derive benefit based on subsequent durable tumor responses as illustrated in online supplementary figure 1 and Figure 2 a-c. Additionally, some patients with RECIST V.1.1 responses experience iUPD followed by iSD, thereby increasing the durability of the response. Taken together, durability of response assessed by iRECIST led to a longer median DOR, a difference that was modest in the overall analysis population (~1 month) but potentially meaningful within the evaluation of specific therapeutic categories, for example, the anti-CTLA-4 antibody subgroup (~3 months increase in median DOR with iRECIST). Thus, while RECIST V.1.1 appears to capture most of the treatment effect based on ORR for anti-PD-1/PD-L1 antibodies, the use of this response criteria may result in early termination of effective treatment in a limited number of patients who may also exhibit prolonged survival similar to that observed in patients with RECIST V.1.1 response.

    Patients with prolonged iSD, while not considered in the assessment of ORR, could substantially impact the assessment of treatment effects on PFS. Some patients with initial disease progression followed by disease stability are considered early progressors by RECIST but prolonged SD by iRECIST, thus improving the correlation between PFS and survival. Among anti-PD-1/PD-L1 antibody-treated patients, the estimated median iPFS was 4.2 months (95% CI 4.0 to 4.4), and the estimated median PFS was 3.9 months (95% CI 3.6 to 4.1) by RECIST V.1.1. Ultimately, PFS is a time-to-event tumor measurement-based endpoint that requires a control arm, typically in a randomized trial, to interpret the meaningfulness of a treatment effect on this outcome. The associations between iPFS and OS and between PFS and OS were weak and similar. Thus, on a trial-level basis, both tumor response criteria yield similar results. However, conclusions regarding the minimal differences in R2 value when evaluating the utility of iPFS as a surrogate for OS in immunotherapy trials compared with PFS are limited by the fact that this analysis included multiple cancer subtypes (melanoma, NSCLC, RCC, and HNSCC), and PFS per RECIST V.1.1 has not been validated as a surrogate for survival in all of these cancer types. Additionally, not all studies had mature OS follow-up, potentially affecting these results.

    Although this analysis includes a large number of studies and patient data, it is limited by its retrospective nature and the fact that none of the trials were conducted according to iRECIST. Collection of non-target lesion tumor data in case report forms does not provide adequate information for characterization of response status by iRECIST since subsequent increase in size, which is used to confirm disease progression, was not collected after first designation of ‘unequivocal progression’. In our analyses, we took a conservative approach, assigning confirmation if unequivocal progression was indicated at the next assessment. Additionally, measurements for new lesions were not always recorded, thus confirmation based on an increase in lesion size could not always be made. Finally, iRECIST requires that if a patient discontinues prior to confirmation (iCPD), the reason for discontinuation should be captured in case report forms, as discontinuation is medically appropriate in some settings. The number of patients without confirmation of disease progression in our analysis is considerable; 79% of patients with iUPD were not confirmed as of the data cut-off date for the study or prior to leaving the trial, including the 61% of patients with PD per RECIST V.1.1 and no further disease assessments. The effects of missing data regarding confirmation of iUPD on determination of iPFS have not been assessed in this retrospective analysis.

    Conclusions

    The use of immune-based tumor response criteria as the primary assessment of tumor measurement-based endpoints has been hampered in part by the multiple response criteria that have been proposed and variations therein, complexity of individual criteria, differential application in randomized trials where immune-based response criteria are applied only to the experimental arm (immunotherapy) and not the control arm (eg, a chemotherapeutic regimen), and increasing reliance on single-arm trials to identify clinically meaningful treatment effects on ORR and DOR. Employing novel response criteria is particularly challenging for single-arm trials as the assessment of treatment effects relies on historic controls with cross-study comparisons confounded by use of disparate response criteria. While adoption of iRECIST in immunotherapeutic trials may address some of the limitations of RECIST V.1.1 with respect to patient management—limitations that are often overcome on a case-by-case basis in current protocols—the question remains whether the additional burden in data collection (eg, measurements of new lesions, assessment of non-target growth from the previous assessment, and confirmation of progression) and data evaluation (eg, programming patient response post initial RECIST V.1.1 progression and a more complicated algorithm regarding what constitutes a confirmation of progression) is outweighed by any impact that prospective use of iRECIST will have on the interpretation of tumor measurement-based endpoints such as ORR and PFS. While the magnitudes of treatment effects on ORR and PFS, as well as associations of PFS with OS, as calculated by iRECIST or RECIST V.1.1 appear similar, the substantial proportion of missing data following RECIST V.1.1 progression in this retrospective analysis limits firm conclusions of the utility of iRECIST for evaluation of immunotherapeutics in trials intended to support marketing authorization.

    In prospective trials proposing the use of an immune-based tumor response criteria such as iRECIST for the determination of the primary endpoint, the trial design, trial conduct, and analysis plan would need to minimize and address potential bias introduced with the use of such criteria. Given its widespread acceptance, iRECIST has the potential to unify how data are collected and evaluated across both single-arm and randomized trials, providing a new standard for tumor response assessment across trials.

    References

    1. ↵
      2. Oba K ,
      3. Paoletti X ,
      4. Alberts S , et al
      . Disease-Free survival as a surrogate for overall survival in adjuvant trials of gastric cancer: a meta-analysis. J Natl Cancer Inst 2013;105:1600–7.doi:10.1093/jnci/djt270
      OpenUrlCrossRefPubMed
    2. ↵
      1. FDA
      . Clinical trial endpoints for the approval of cancer drugs and biologics guidance for industry 2018.
    3. ↵
      2. Miller AB ,
      3. Hoogstraten B ,
      4. Staquet M , et al
      . Reporting results of cancer treatment. Cancer 1981;47:207–14.doi:10.1002/1097-0142(19810101)47:1<207::AID-CNCR2820470134>3.0.CO;2-6
      OpenUrlCrossRefPubMedWeb of Science
    4. ↵
      2. Therasse P ,
      3. Arbuck SG ,
      4. Eisenhauer EA , et al
      . New guidelines to evaluate the response to treatment in solid tumors. European organization for research and treatment of cancer, National cancer Institute of the United States, National cancer Institute of Canada. J Natl Cancer Inst 2000;92:205–16.doi:10.1093/jnci/92.3.205
      OpenUrlCrossRefPubMedWeb of Science
    5. ↵
      2. Eisenhauer EA ,
      3. Therasse P ,
      4. Bogaerts J , et al
      . New response evaluation criteria in solid tumours: revised RECIST guideline (version 1.1). Eur J Cancer 2009;45:228–47.doi:10.1016/j.ejca.2008.10.026
      OpenUrlCrossRefPubMedWeb of Science
    6. ↵
      2. Fojo AT ,
      3. Noonan A
      . Why RECIST works and why it should stay--counterpoint. Cancer Res 2012;72:5151–7. Discussion 8.doi:10.1158/0008-5472.CAN-12-0733
      OpenUrlCrossRefPubMed
    7. ↵
      2. Wolchok JD ,
      3. Hoos A ,
      4. O'Day S , et al
      . Guidelines for the evaluation of immune therapy activity in solid tumors: immune-related response criteria. Clin Cancer Res 2009;15:7412–20.doi:10.1158/1078-0432.CCR-09-1624
      OpenUrlAbstract/FREE Full Text
    8. ↵
      2. Bohnsack O ,
      3. Hoos A ,
      4. Ludajic K , et al
      . Adaptation and modification of the immune related response criteria (IRRC): IrRECIST. J Clin Oncol 2014;32:e22121.doi:10.1200/jco.2014.32.15_suppl.e22121
    9. ↵
      2. Hodi FS ,
      3. Ballinger M ,
      4. Lyons B , et al
      . Immune-Modified response evaluation criteria in solid tumors (imRECIST): refining guidelines to assess the clinical benefit of cancer immunotherapy. J Clin Oncol 2018;36:850–8.doi:10.1200/JCO.2017.75.1644
      OpenUrlPubMed
    10. ↵
      2. Seymour L ,
      3. Bogaerts J ,
      4. Perrone A , et al
      . iRECIST: guidelines for response criteria for use in trials testing immunotherapeutics. Lancet Oncol 2017;18:e143–52.doi:10.1016/S1470-2045(17)30074-8
      OpenUrlPubMed
    11. ↵
      2. Borghaei H ,
      3. Paz-Ares L ,
      4. Horn L , et al
      . Nivolumab versus docetaxel in advanced Nonsquamous non-small-cell lung cancer. N Engl J Med 2015;373:1627–39.doi:10.1056/NEJMoa1507643
      OpenUrlCrossRefPubMed
    12. ↵
      2. Ferris RL ,
      3. Blumenschein G ,
      4. Fayette J , et al
      . Nivolumab for recurrent squamous-cell carcinoma of the head and neck. N Engl J Med 2016;375:1856–67.doi:10.1056/NEJMoa1602252
      OpenUrlCrossRefPubMed
    13. ↵
      2. Motzer RJ ,
      3. Escudier B ,
      4. McDermott DF , et al
      . Nivolumab versus everolimus in advanced renal-cell carcinoma. N Engl J Med 2015;373:1803–13.doi:10.1056/NEJMoa1510665
      OpenUrlCrossRefPubMed
    14. ↵
      2. Postow MA ,
      3. Chesney J ,
      4. Pavlick AC , et al
      . Nivolumab and ipilimumab versus ipilimumab in untreated melanoma. N Engl J Med 2015;372:2006–17.doi:10.1056/NEJMoa1414428
      OpenUrlCrossRefPubMed
    15. ↵
      2. Ribas A ,
      3. Puzanov I ,
      4. Dummer R , et al
      . Pembrolizumab versus investigator-choice chemotherapy for ipilimumab-refractory melanoma (KEYNOTE-002): a randomised, controlled, phase 2 trial. Lancet Oncol 2015;16:908–18.doi:10.1016/S1470-2045(15)00083-2
      OpenUrlCrossRefPubMed
    16. ↵
      2. Robert C ,
      3. Long GV ,
      4. Brady B , et al
      . Nivolumab in previously untreated melanoma without BRAF mutation. N Engl J Med 2015;372:320–30.doi:10.1056/NEJMoa1412082
      OpenUrlCrossRefPubMedWeb of Science
    17. ↵
      2. Robert C ,
      3. Schachter J ,
      4. Long GV , et al
      . Pembrolizumab versus ipilimumab in advanced melanoma. N Engl J Med 2015;372:2521–32.doi:10.1056/NEJMoa1503093
      OpenUrlCrossRefPubMed
    18. ↵
      2. Brahmer J ,
      3. Reckamp KL ,
      4. Baas P , et al
      . Nivolumab versus docetaxel in advanced squamous-cell non-small-cell lung cancer. N Engl J Med 2015;373:123–35.doi:10.1056/NEJMoa1504627
      OpenUrlCrossRefPubMed
    19. ↵
      2. Reck M ,
      3. Rodríguez-Abreu D ,
      4. Robinson AG , et al
      . Pembrolizumab versus chemotherapy for PD-L1-positive non-small-cell lung cancer. N Engl J Med 2016;375:1823–33.doi:10.1056/NEJMoa1606774
      OpenUrlCrossRefPubMed
    20. ↵
      2. Fehrenbacher L ,
      3. Spira A ,
      4. Ballinger M , et al
      . Atezolizumab versus docetaxel for patients with previously treated non-small-cell lung cancer (poplar): a multicentre, open-label, phase 2 randomised controlled trial. Lancet 2016;387:1837–46.doi:10.1016/S0140-6736(16)00587-0
      OpenUrlCrossRefPubMed
    21. ↵
      2. Larkin J ,
      3. Chiarion-Sileni V ,
      4. Gonzalez R , et al
      . Combined nivolumab and ipilimumab or monotherapy in untreated melanoma. N Engl J Med 2015;373:23–34.doi:10.1056/NEJMoa1504030
      OpenUrlCrossRefPubMed
    22. ↵
      2. Rittmeyer A ,
      3. Barlesi F ,
      4. Waterkamp D , et al
      . Atezolizumab versus docetaxel in patients with previously treated non-small-cell lung cancer (oak): a phase 3, open-label, multicentre randomised controlled trial. Lancet 2017;389:255–65.doi:10.1016/S0140-6736(16)32517-X
      OpenUrlCrossRefPubMed
    23. ↵
      2. Weber JS ,
      3. D'Angelo SP ,
      4. Minor D , et al
      . Nivolumab versus chemotherapy in patients with advanced melanoma who progressed after anti-CTLA-4 treatment (CheckMate 037): a randomised, controlled, open-label, phase 3 trial. Lancet Oncol 2015;16:375–84.doi:10.1016/S1470-2045(15)70076-8
      OpenUrlCrossRefPubMed
    24. ↵
      2. Herbst RS ,
      3. Baas P ,
      4. Kim D-W , et al
      . Pembrolizumab versus docetaxel for previously treated, PD-L1-positive, advanced non-small-cell lung cancer (KEYNOTE-010): a randomised controlled trial. Lancet 2016;387:1540–50.doi:10.1016/S0140-6736(15)01281-7
      OpenUrlCrossRefPubMed
    25. ↵
      2. Kaplan EL ,
      3. Meier P
      . Nonparametric estimation from incomplete observations. J Am Stat Assoc 1958;53:457–81.doi:10.1080/01621459.1958.10501452
      OpenUrlCrossRefWeb of Science
    26. ↵
      2. Queirolo P ,
      3. Spagnolo F
      . Atypical responses in patients with advanced melanoma, lung cancer, renal-cell carcinoma and other solid tumors treated with anti-PD-1 drugs: a systematic review. Cancer Treat Rev 2017;59:71–8.doi:10.1016/j.ctrv.2017.07.002
      OpenUrl
    27. ↵
      2. Blumenthal GM ,
      3. Theoret MR ,
      4. Pazdur R
      . Treatment beyond progression with immune checkpoint Inhibitors-Known unknowns. JAMA Oncol 2017;3:1473–4.doi:10.1001/jamaoncol.2017.1819
      OpenUrl

    Footnotes

    • FM and MRT contributed equally.

    • Contributors FM, PK, MRT, RP, and RS conceived and designed the study, FM performed the data analysis and produced the figures and tables. All authors interpreted the data and contributed to writing the report.

    • Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

    • Competing interests None declared.

    • Patient consent for publication Not required.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Data availability statement No data are available. Trial data for this analysis was pulled from submissions evaluating the safety and efficacy of an anti-PD-1/PD-L1 antibody submitted to FDA between Sept. 2014 and Sept. 2017.