You are here

Article Text

Article menu
Immunotherapy biomarkers
Original research
Harmonization of tumor mutation burden testing with comprehensive genomic profiling assays: an IQN Path initiative
  1. Riziero Esposito Abate1,
  2. Raffaella Pasquale2,
  3. Alessandra Sacco1,
  4. Vittorio Simeon3,
  5. Monica Rosaria Maiello1,
  6. Daniela Frezzetti1,
  7. Paolo Chiodini3 and
  8. Nicola Normanno1
  1. 1Cell Biology and Biotherapy Unit, Istituto Nazionale Tumori IRCCS Fondazione G.Pascale, Napoli, Italy
  2. 2Medical Genetics Laboratory, AORN S.G. Moscati, Avellino, Italy
  3. 3Medical Statistics Unit, Department of Mental Health and Public Medicine, Università degli Studi della Campania Luigi Vanvitelli, Napoli, Italy
  1. Correspondence to Dr Nicola Normanno; nicnorm{at}yahoo.com

Abstract

Background Although conflicting results emerged from different studies, the tumor mutational burden (TMB) appears as one of most reliable biomarkers of sensitivity to immune checkpoint inhibitors. Several laboratories are reporting TMB values when performing comprehensive genomic profiling (CGP) without providing a clinical interpretation, due to the lack of validated cut-off values. The International Quality Network for Pathology launched an initiative to harmonize TMB testing with CGP assay and favor the clinical implementation of this biomarker.

Methods TMB evaluation was performed with three commercially available CGP panels, TruSight Oncology 500 (TSO500), Oncomine Comprehensive Plus Assay (OCA) and QIAseq Multimodal Panel (QIA), versus the reference assay FoundationOne CDx (F1CDx). Archived clinical samples derived from 60 patients with non-small cell lung cancer were used for TMB assessment. Adjusted cut-off values for each panel were calculated.

Results Testing was successful for 91.7%, 100%, 96.7% and 100% of cases using F1CDx, TSO500, OCA and QIA, respectively. The matrix comparison analysis, between the F1CDx and CGP assays, showed a linear correlation for all three panels, with a higher correlation between F1CDx and TSO500 (rho=0.88) than in the other two comparisons (rho=0.77 for QIA; 0.72 for OCA). The TSO500 showed the best area under the curve (AUC, value 0.96), with a statistically significant difference when compared with the AUC of OCA (0.83, p value=0.01) and QIA (0.88, p value=0.028). The Youden Index calculation allowed us to extrapolate TMB cut-offs of the different panels corresponding to the 10 mutations/megabase (muts/Mb) cut-off of F1CDx: 10.19, 10.4 and 12.37 muts/Mb for TSO500, OCA and QIA, respectively. Using these values, we calculated the relative accuracy measures for the three panels. TSO500 showed 86% specificity and 96% sensitivity, while OCA and QIA had lower yet similar values of specificity and sensitivity (73% and 88%, respectively).

Conclusion This study estimated TMB cut-off values for commercially available CGP panels. The results showed a good performance of all panels on clinical samples and the calculated cut-offs support better accuracy measures for TSO500. The validated cut-off values can drive clinical interpretation of TMB testing in clinical research and clinical practice.

  • Tumor Biomarkers
  • Immune Checkpoint Inhibitors
  • Immunotherapy
  • Non-Small Cell Lung Cancer

Data availability statement

Data are available in a public, open access repository. The data that support the findings of this study are in Zenodo, June 8, 2023 Version v1 at https://doi.org/10.5281/zenodo.8016300.

http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See http://creativecommons.org/licenses/by-nc/4.0/.

https://doi.org/10.1136/jitc-2023-007800

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    WHAT IS ALREADY KNOWN ON THIS TOPIC

    • Tumor mutational burden (TMB) is a possible predictive biomarker for immune checkpoint inhibitors. Several different assays are commercially available for TMB estimation from tumor tissue samples and are widely used in clinical research and clinical practice. However, there is no information available on the best cut-off values for these assays.

    WHAT THIS STUDY ADDS

    • This study provides for the first time a TMB reference value for three widely used comprehensive genomic profiling assays.

    HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY

    • The calculated cut-offs will allow laboratories to provide a correct clinical interpretation of testing results. In this respect, the availability of validated cut-off values will permit a better stratification of patients based on their probability to benefit immune checkpoint inhibitor treatment, thus improving the use of these agents in clinical research and clinical practice.

    Background

    The advent of immune checkpoint inhibitors (ICIs) targeting programmed cell death 1, programmed death ligand 1 and cytotoxic T lymphocyte antigen 4 proteins represented a major breakthrough in cancer therapy, leading to durable responses in a subset of patients with different cancer types.1 2 Although ICIs received several approvals from regulatory agencies to treat patients with cancer as single agent or in combination treatments, reliable biomarkers to select patients with higher probability to respond to ICIs are still missing.3 This selection is crucial to guarantee the appropriateness of the treatment and the sustainability of cancer treatments, due to the high cost of these agents and their possible side effects.

    The tumor mutational burden (TMB) has emerged as a candidate biomarker for ICIs activity in different tumor types, including non-small cell lung cancer (NSCLC), melanoma and bladder cancer.4 TMB is defined as the number of somatic mutations per megabase (muts/Mb) in the coding area of the genome. TMB has been approved as tumor agnostic biomarker for pembrolizumab treatment in previously treated patients with cancer by the Food & Drug Administration (FDA) based on the results of a phase II KEYNOTE-158 clinical trial.5 However, this biomarker has not been approved by the European Medicine Agency (EMA), thus limiting its uptake at least in Europe.

    TMB has been initially measured by whole exome sequencing that is able to provide the overall mutational count over the entire coding genome.6 However, this approach is not feasible to translate into clinical routine and multiple companies have developed target sequencing (TS) assays for TMB estimate.7 8 In the clinical trial leading to the approval of TMB as biomarker, the FoundationOne CDx (F1CDx) (Foundation Medicine) panel was used, with a cut-off of 10 muts/Mb to define a high TMB value, based on the results of the KEYNOTE-158 study.5 More recently, the MSK-IMPACT TS panel has also been cleared by the FDA for TMB testing.9 10

    However, the majority of academic centers, at least in Europe, tests samples in house rather than sending to commercial vendors. Sequencing tumor samples in house allows the academic centers to reduce the costs and, more importantly, to accumulate raw data that can be used for different purposes, including the development of new bioinformatics pipelines and the exploration of additional biomarkers. Several TS panels are commercially available and previous studies demonstrated that they might lead to significantly different estimates of TMB values in the same samples. In fact, different factors might affect TMB testing with TS. In particular, the formalin fixation can significantly affect the quality and the quantity of the DNA extracted and can also lead to the creation of deamination artifacts.11 Moreover, the estimation of TMB can be affected by genes covered and bioinformatics platform used to analyze raw data, which might overestimate TMB if germline and known driver mutations are not filtered out.11 12 ,13

    It is crucial, due to the high number of available TS panels, to establish validated cut-off values to correctly select patients.14 Several groups launched initiatives for the harmonization of TMB testing. In particular, the Friends of Cancer Research (FOCR) TMB Harmonization Project recently reported a high panel-based variability in TMB estimation.15–18 Similar results were obtained by the International Quality Network of Pathology (IQN Path), a network of quality assessment associations with an interest in cancer biomarker testing. The External Quality Assessment (EQA) pilot scheme for TMB testing organized by IQN Path revealed a low interlaboratory variability among centers that used the same panel for TMB assessment, but a significant difference between laboratories that employed different TMB assays.19 Importantly, most of the laboratories offering TMB testing declared not providing a clinical interpretation of the result due to lack of validation data of specific cut-off values for the commercially available TMB tests.

    Although TMB has not been approved as biomarker in several countries, the increasing use of large Next Generation Sequencing (NGS) panels for comprehensive genomic profiling (CGP) is leading many laboratories to report the TMB value without a clinical interpretation. In order to address this issue, IQN Path performed a comparison of TMB testing with three commercially available NGS panels used for CGP, in order to create a reference table that maps the score of one assay to another.

    Methods

    Samples

    Sample size was calculated considering a linear regression model with mutational burden values evaluated using gold standard procedure as independent variable and mutational load values evaluated with one of the new methods proposed as dependent variable. The effect size for linear regression is usually measured by Cohen’s f2 calculated as R2/(1−R2) and a Cohen f2 of 0.15 is considered a medium effect size. A sample size of 55 subjects achieves 80% power to detect a change in effect size from zero under the null hypothesis to 0.15 under the alternative hypothesis with a significance level of 0.05. In a simple linear regression, a Cohen f2 of 0.15 corresponds to a R2 of 0.13 and to a Pearson correlation coefficient of 0.36. Based on these estimates, the study included 60 formalin fixed paraffin embedded (FFPE) tissue blocks from patients with NSCLC, which were acquired from the France Tissue Bank. All the samples were primary tumor and had a tumor content>40% of neoplastic cells.

    DNA extraction

    Genomic DNA (gDNA) was isolated from two 10 µm-thick FFPE tissue sections using the GeneRead DNA FFPE kit (QIAGEN) according to the manufacturer’s protocol. The gDNA quantity was assessed with the Qubit dsDNA HS assay kit (Thermo Fisher Scientific) using the Qubit 2.0 Fluorometer (Thermo Fisher Scientific).

    Oncomine Comprehensive Plus Assay

    Oncomine Comprehensive Plus Assay (OCA) covers a region of 1.50M total bases, of which 1.06M exonic bases that include more than 500 genes with known cancer associations (Thermo Fisher Scientific). Libraries were prepared using Ion AmpliSeq Library Kit Plus (Thermo Fisher Scientific) starting from 20 ng of gDNA. Fifty picomoles of each library were multiplexed and clonally amplified by emulsion PCR, performed and enriched on the Ion Chef instrument (Thermo Fisher Scientific). Finally, the template was loaded on an Ion 550 Chip and sequenced on an Ion S5 XL sequencer (Thermo Fisher Scientific) according to the manufacturer’s instructions. Each sequenced chip contained four samples. The analysis was performed on Ion Reporter v5.18 with Oncomine Comprehensive Plus -w2.3 -DNA- Single Sample. All the sample had at least 22M of mapped reads, mean read length >85 bp, uniformity >90% and coverage >800. For TMB calculation, we used the TMB (Non-germline Mutations) workflow. In particular, this pipeline excluded germline variants and used only somatic mutations with allelic frequency >5% and coverage >60, not included in the UCSC Common SNP. Moreover, all the samples with deamination score >60 were excluded from further analysis.

    TruSight Oncology 500 Assay

    The TruSight Oncology 500 Assay (TSO500) (Illumina) analyzes 523 cancer-related genes in a coding region of 1.3 Mb and 1.90M total bases. The DNA was checked before fragmentation with Infinium HD FFPE QC Assay Protocol using manufacturer’s instructions. Only samples that passed FFPE quality control (QC) were fragmented with a M-220 Focused-ultrasonicator (Covaris) using 80 ng of gDNA. To evaluate the peak size of the generated fragments, the samples were run on the 2100 Bioanalyzer Instrument (Agilent) using the High Sensitivity DNA Kit (Agilent). Libraries were prepared following the manufacturer’s instructions and were quantified with the Qubit dsDNA HS Assay Kit (Thermo Fisher Scientific). A library concentration of at least 3 ng/µL was required to achieve efficient bead-based library normalization. Sequencing was performed on NextSeq 500 platform using High Output reagents, and 200 cycles of sequencing in paired-end (Illumina). Furthermore, PhiX control at 1% (Illumina) was used as a sequencing control. TMB was reported using the TruSight Oncology 500 Local App version 1 (Illumina). The manufacturer’s QC criteria were used to determine percentage of passing filter reads >80%, percentage of Q30_R1 >80%, percentage of Q30_R2 >80%, median insert size ≥70 bp, median exon coverage ≥150 count, percentage of exons with coverage of at least 50 count ≥90%, and usable MSI sites >40.

    QIAseq Multimodal Panel

    The QIAseq Multimodal Panel (QIA) targets relevant mutations in genes related to tumor and covers a total 1.44 Mb of DNA with Single Primer Extension technology (QIAGEN). The generated libraries were run for QC of size on 2100 Bioanalyzer Instrument (Agilent) using the High Sensitivity DNA Kit. The sequencing run was performed on NextSeq 500 using High Output reagents, and 300 cycles of sequencing in paired-end (Illumina) and custom primers for read1 (QIAGEN). The PhiX control at 1% (Illumina) was used as a sequencing control. The bioinformatics analysis was performed on the QIAGEN CLC Genomics Workbench using Multimodal TMB/ Microsatellite Instability workflow. In particular, the pipeline used only non-synonymous mutations with an allelic frequency >5% and <95% with a coverage >100, an average quality >25 and minimum QUAL >150.

    FoundationOne CDx

    The FoundationOne CDx (F1CDx) (Foundation Medicine, Cambridge, Massachusetts, USA) assay was used as the reference standard method in our EQA scheme, based on the results of the KEYNOTE-158 trial. The assay covers the exonic regions of 324 cancer-related genes and selected introns from 51 genes commonly rearranged in cancer for a total coverage of 0.8 Mb of DNA. Two 10 µm slides from all samples were sent to Roche FMI (Penzberg, Germany) for the analysis.

    Statistical analysis

    Statistical analyses were performed on the complete case dataset, removing missing samples for one or more NGS targeted panels. TMB measures for each panel were described with mean and SD and median value with IQR to better evaluate the different distributions. Spearman’s R correlation values were calculated, and scatterplots were created to assess linearity of the relationship between each pair of panel. Measurements of agreement between F1CDx, as gold standard, and the other panels were evaluated using the concordance correlation coefficient (CCC) with the relative 95% CI. This metric combines measures of both precision and accuracy to determine how far the observed data deviate from the line of perfect concordance (line at 45° on a square scatterplot). Furthermore, the bias between two panels (F1CDx as gold standard) was evaluated with a Bland and Altman plot, using the mean of the difference between the two measurements per subject and the 95% limits of agreement (based on SD of the differences).

    Next, the complete case samples were categorized according to F1CDx TMB value ≥10 muts/Mb and further investigated. The discriminatory ability of each panel, with respect to F1CDx method as reference, was studied using the area under the receiver operating characteristic (ROC) curve, with relative 95% CI. The decision thresholds were displayed on the ROC plot and, using the Youden Index, a cut-point was estimated for each method. The sensitivity, specificity, positive and negative predictive values and accuracy were estimated to characterize the different cut-off points. Data were analyzed using R software V.4.3.1 (R Foundation for Statistical Computing, Vienna, Austria), using ggplot2 package for graphical elaboration.

    Results

    Sixty NSCLC FFPE samples were tested with three different commercially available TS panels commonly employed for CGP, the OCA (Thermo Fisher Scientific), the TSO500 (Illumina) and the QIA (QIAGEN) and with the reference F1CDx assay. The clinical and pathological features of the cases tested in this study are summarized in online supplemental table 1. The success rate of TMB assessment was 91.7%, 100%, 96.7% and 100% for F1CDx, TSO500, OCA and QIA, respectively. The test failures led us to reduce the analysis to 53 samples with available TMB values for the four panels, given that two analyses failed with OCA evaluation due to high deamination score and five with F1CDx because of low tumor content. A STARD diagram summarizing samples and results is shown in online supplemental figure 1.

    Supplemental material

    Supplemental material

    Sequencing with F1CDx revealed the presence of at least one genomic alteration (GA) in all cases (53/53, 100%). In depth, the analysis detected in 99 genes, 298 GAs (110 SNVs, 36 indels, 80 gene amplifications, 37 truncations, 33 homozygous deletions and 1 rearrangement) with an average of 5.6 GAs/patient. All tumors were microsatellite stable. The most commonly mutated genes were TP53, EGFR, KRAS and PIK3CA (online supplemental figure 2). These data are in line with other series of NSCLC tested with the F1CDx panel.20

    Supplemental material

    The TMB measured with F1CDx had a mean value of 10.1 (SD 8.6) muts/Mb and a median value of 7.6 muts/Mb (IQR 10). We explored the association between the most frequently mutated genes and TMB values. In this respect, we found no associations with TP53, PIK3CA, KRAS, RBM10 and PTEN mutations, although the low numbers might have affected such correlations. Indeed, in the nine KRAS mutant cases the median TMB was 10.09 vs 7.57 of the KRAS wild type samples, while the 11 EGFR mutant showed a median TMB of 3.78 vs 8.83 of the EGFR wild type tumors. The latter correlation showed a trend (EGFR wild-type vs mutant p value=0.0543) in agreement with previous reports21 (figure 1).

    Figure 1
    Figure 1

    Evaluation of the association between TMB and mutational status of TP53, EGFR, PIK3CA, KRAS, RBM10 and PTEN in patients with NSCLC. The horizontal line indicates the median. NSCLC, non-small cell lung cancer; TMB, tumor mutational burden.

    Next, TMB values were calculated for the three commercially available panels following manufacturers’ instructions. The sample requirements and QC metrics for the different panels are shown in online supplemental table 2. In particular, only non-synonymous substitutions were included in the TMB assessment by OCA, TSO500 and QIA. TSO500, OCA and QIA showed a mean TMB value of 10.7 (SD 9), 9.7 (SD 5.6) and 11.6 (SD 8) muts/Mb, and a median TMB value of 8.6 (IQR 7.8), 8.5 (IQR 5.7) and 9.6 (IQR 8.1) muts/Mb, respectively (table 1). All the measurements had a skewed distribution (online supplemental figure 3).

    Supplemental material

    View this table:
    Table 1

    Statistical analysis on 53 valid NSCLC samples

    Matrix comparison with scatter plot and the value of Spearman’s rho, respectively for each crossing, showed that there was a higher correlation between F1CDx and TSO500 TMB values (rho 0.88) than in the other two comparisons (rho 0.73 for F1CDx vs OCA and 0.77 for F1CDx vs QIA). TSO500 and QIA also showed a strong correlation value when compared (rho 0.87). The analysis of the scatter plots clearly demonstrate that the correlation was linear (figure 2A). The CCC value of the F1CDx–TSO500 comparison was 0.95 (95% CI 0.91 to 0.97), whereas both F1CDx–OCA and F1CDx–QIA comparison showed a lower CCC value, 0.76 (95% CI 0.66 to 0.84) and 0.86 (95% CI 0.78 to 0.92), respectively (figure 2B). In the Bland and Altman diagram, considering low and medium difference values for the comparison of TSO500 with F1CDx, we found the same scatter and variance (homoscedasticity) which seems to turn towards different variance distribution (heteroscedasticity) for higher mean values. On the other hand, a strong heteroscedasticity is visible in the OCA/F1CDx comparison with negative difference values (higher for OCA) to positive values (higher for F1CDx) as their mean value increases; for QIA, the graph shows a clear homoscedasticity (online supplemental figure 4).

    Supplemental material

    Figure 2
    Figure 2

    (A) Scatter plot and relative Spearman’s rho values for each comparison. (B) Concordance correlation coefficient (CCC) for the three assays. F1CDx, FoundationOne CDx; OCA, Oncomine Comprehensive Plus Assay; QIA, QIAseq Multimodal Panel; TSO500, TruSight Oncology 500.

    Categorizing the samples by the cut-off value defined by F1CDx, the analyzed cohort of samples consisted of 31 cases (58.5%) with TMB values <10 muts/Mb, and 22 (41.5%) cases with TMB ≥10. ROC curve and relative area under the curve (AUC) (with 95% CI) were performed. The AUC for TSO500 was 0.96 (95% CI 0.91 to 0.99), for OCA 0.83 (95% CI 0.71 to 0.94) and for QIA 0.88 (95% CI 0.78 to 0.96). TSO500 showed a higher AUC, and this difference is statistically significant (bootstrap test for two correlated ROC curves: p value=0.01 vs OCA; p value=0.03 vs QIA) (figure 3). OCA and QIA showed an overlapping AUC.

    Figure 3
    Figure 3

    ROC curve and relative area under the curve for each TMB test used in the assessment. OCA, Oncomine Comprehensive Plus Assay; QIA, QIAseq Multimodal Panel; ROC, receiver operating characteristic; TMB, tumor mutational burden; TSO500, TruSight Oncology 500.

    The Youden Index calculation allowed to extrapolate a TMB threshold value with higher sensitivity and specificity for TSO500, OCA and QIA measurements. This value was 10.19 for TSO, 10.4 for OCA and 12.37 for QIA, respectively. Considering these cut-offs and compared with the gold standard F1CDx test, we calculated specificity sensitivity, accuracy, positive and negative predictive values (PPV, NPV) for each panel. In particular, the TSO500 panel showed the best accuracy measures in terms of sensitivity (86%), specificity (94%), accuracy (91%), PPV (90%) and NPV (91%) when compared with OCA and QIA (table 2). The OCA and QIA panels showed similar results of performance with similar values of all measures of concordance (table 2).

    View this table:
    Table 2

    Concordance of TMB classification using the calculated cut-offs for each assay

    With the aim of comparing our results with previous data, we used the public dataset of the TSO500/F1CDx comparison from the manuscript of Ramos-Paradas and colleagues. The dataset, composed of 96 subjects, was analyzed by categorizing the samples based on the results of our cut-off, obtaining a sensitivity of 77.6%, a specificity of 91.5%, and positive and negative predictive values of 90.5% and 79.6%, respectively. When compared with the results published by Ramos-Paradas et al who identified the value 7.847 muts/Mb as cut-off for TSO500, a clear difference is noted in terms of better sensitivity with the Ramos-Paradas cut-off and better specificity with our cut-off, with a very similar overall accuracy between the two analyses (82% for the Ramos-Paradas cut-off, 84% for our cut-off) (table 3).

    View this table:
    Table 3

    Concordance of TMB classification using our cut-off for TSO500 results on Ramos-Paradas dataset

    We next used the cut-off proposed by Ramos-Paradas and colleagues in our cohort, obtaining a sensitivity of 100%, a specificity of 74.2%, and PPV and NPV of 73.3% and 100.0%, respectively (table 4).

    View this table:
    Table 4

    Concordance of TMB classification using Ramos-Paradas cut-off for TSO500 results on our dataset

    There is a clear difference when compared with our results, of better sensitivity with the Ramos-Paradas cut-off and better specificity with our cut-off, with a very similar overall accuracy between the two analyses (87% for the Ramos-Paradas cut-off, 90% for our cut-off).

    Discussion

    The role of TMB as a possible biomarker of response to ICI is still debated. Clinical validation of this biomarker has failed in some clinical trials, while in others it appears to significantly correlate with ICI activity.5 22 23 Several studies suggest that response to ICI can be better predicted using a combination of biomarkers.24–28 In particular, the response to immunotherapy is conditioned by tumor-related factors, including the presence of neoantigens for which TMB is a surrogate, and the activation of the immune microenvironment. Indeed, the combination of TMB with parameters related to the activation of the immune response in the tumor microenvironment seems to better correlate with the response to immunotherapy.24–27 However, it must be emphasized that in these studies aimed at identifying biomarkers for ICI, TMB has consistently emerged as one of the most relevant biomarkers predictive of ICI activity. In addition, high TMB levels are associated with increased CD8 positive T-cell infiltration and activation of immune signatures in NSCLC,26 thus confirming the ability of high TMB to induce an immune response. It has been hypothesized that this phenomenon might be restricted only to selected cancer types, including lung adenocarcinoma.23 However, recent real-world data suggests the predictive value of high TMB values in patients with different advanced solid cancers.29 30 In particular, high TMB values seem to better predict activity of single agent ICI.30 Contradictory results have been also reported on the role of TMB as predictive biomarker of response in neoadjuvant clinical trials of ICI in NSCLC, which used the same assay(s) and the same cut-off values employed in the metastatic setting.31

    TMB is a complex biomarker and, therefore, the application of TMB in clinical practice requires careful validation. Several initiatives have helped to improve the TMB test. In particular, the FOCR TMB Harmonization Project initiated a program that helped highlight the limitations of some TS panels and improved the performance of the TMB test, by identifying a minimum size of CGP panels and highlighting the need for germline and driver mutation filtering.16 18 However, this program, like other studies in the field,32 evaluated the agreement of various panels aimed at assessing TMB with a reference assay, but did not help identify cut-off values for the different panels available for assessing TMB.

    In the TMB EQA pilot program organized by IQN Path, we have noticed that many laboratories described the TMB value in the report without providing a clinical interpretation, due to the absence of reference values. To address this issue, we conducted this study in which we compared three widely used panels for CGP, TSO500, QIA and OCA, to the reference assay F1CDx. Our study provides for the first time a TMB reference value for these panels that can be used in clinical research and clinical practice.

    We acknowledge that the use of only three panels in this study is a limitation, while previous studies assessed a larger number of assays.33 However, we chose these assays based on the results of a survey on TMB testing and the EQA scheme organized by IQN Path.19 34 In particular, EQA providers under the umbrella of IQN Path distributed the survey among all the laboratories participating to their schemes. Overall, 127 laboratories replied to the survey, of which 117 used NGS for tumor molecular profiling. Among the 69 laboratories that were already testing TMB, 21 used the Oncomine Tumor Mutation Load (OTML) panel, 18 custom panels, 6 the TSO500 and 2 the QIA.34 In the TMB EQA organized by IQN Path, the use of the OTML was in part replaced by the OCA panel, which was released more recently. Indeed, we observed a transition from the use of TMB specific panels to CGP assays, which are increasingly adopted by laboratories in order to limit the number of panels needed to cover the growing number of biomarkers to analyze in different tumor types. In this respect, the three tests selected for this study were the most widely used in Europe for CGP, at least when the project was planned. We decided to use F1CDx as reference method since this assay was employed in the clinical trials that led to the approval of TMB as biomarker by the FDA.

    An earlier study by Ramos-Paradas compared the F1CDx assay with the OTML (not adequate for CGP) and the TSO500 assay. For the latter assay, the authors proposed a cut-off value of 7.847 muts/Mb, much lower than the 10.19 muts/Mb identified in our study. A comparison of the two cut-offs demonstrates that the one proposed by Ramos-Paradas has higher sensitivity but also lower specificity when compared with the cut-off identified in this study. The difference between the results of these two studies may be due to a number of factors. In both cases, the number of samples tested is relatively small. The distribution of TMB values also appears to be different. In fact, in our study the median value of TMB with F1CDx is 7.6 muts/Mb with a range of 0–35.3 muts/Mb, against a median of 10 muts/Mb, spanning a range of TMB values from 0 muts/Mb to 74 muts/Mb, in the Ramos-Paradas study. Moreover, our cohort included 11 cases carrying an EGFR mutation, and the previously reported correlation between these variants and TMB values was confirmed in our series.21 Finally, Ramos-Paradas and collaborators used samples from clinical practice, while we analyzed highly selected samples obtained from a centralized biobank in order to limit bias due to sample quality. Data suggest high TMB values are associated with increased clinical benefit from immunotherapy.26 Therefore, we expect that the TMB cut-offs proposed by us can improve patient selection for immunotherapy and therefore lead to better clinical results, in particular when ICIs are used as single agent.24

    The interlaboratory reproducibility of TMB testing using different TS panels appears good based on the results of the pilot EQA program that we previously described.19 However, it appears evident that the classification of samples in ‘high’ vs ‘low’ based on a single test with a numerical value that could also be influenced by numerous technical and biological variables such as tumor heterogeneity,35 is probably not a suitable option. Cases with values close to the cut-offs should always be discussed within multidisciplinary groups or molecular tumor boards for correct data interpretation.

    The NSCLC tissues used in this study are derived from primary tumors in order to have sufficient amount of gDNA to perform all the CGP tests from the same blocks. Previous data suggest that TMB levels are higher in lung cancer metastases as compared with the primary tumor tissue, with the highest levels found in brain metastases.35 For this reason, the TMB calculated on primary tumor might be underestimated in patients with metastatic disease. However, the current FDA approval of TMB as biomarker and the currently used clinical cut-off do not take into account the source of the tissue tested (primary or metastases). In this context, we hypothesize that the cut-off values that we identified could be used in both curative and recurrent/metastatic settings.

    The 10 muts/Mb cut-off was initially established in clinical trials of immune checkpoint inhibitors in NSCLC.36 The same assay(s) and the same cut-off have been used for tumor agnostic approaches. Based on this background, the results of our study could be applied to other tumor types. However, extensive validation of TMB assays in tumor types other than NSCLC is missing. Even more importantly, the clinical cut-off could be different for different tumor types as suggested by previous reports.3

    Although TMB is not recognized as a predictive biomarker for ICI by EMA, European Society for Medical Oncology (ESMO) recommended TMB testing in selected tumor types (ie, cervical cancers, well-differentiated and moderately differentiated neuroendocrine tumors, salivary cancers, thyroid cancers and vulvar cancers) that were adequately represented in the clinical trial that led FDA to approve TMB as an agnostic biomarker.37 The ESMO recommendations were prepared in 2020 based on the available clinical evidence and might change over the time if new data are generated. In addition, ESMO recommends to test a biomarker if the matched drug is available. We must acknowledge that the uptake of NGS in Europe is significantly limited,38 39 as well as the approval of agnostic biomarkers has been restricted by EMA only to Neurotrophic Tropomyosin Receptor Kinase (NTRK) fusions. However, several genomic initiatives are ongoing in Europe that might favor the uptake of NGS testing and the implementation of precision medicine.40 Interestingly, a recent study of prospective genomic profiling in patients with advanced cancer conducted in Italy reported that 57% of patients with high TMB received immunotherapy regardless of the tumor histology, suggesting that the oncologists take into account this biomarker for therapeutic decisions.20 In this scenario, harmonization of CGP assays is fundamental to ensure adequate stratification of patients and to improve the reproducibility of data among laboratories using different assays. This latter point is particularly relevant since there is an increasing interest of academic institutions and regulatory bodies in the use of real-world data, which will be comparable only if the assays used to stratify patients have been harmonized.41 Indeed, real-world data from large clinical and genomic datasets seem to confirm the correlation between high TMB and benefit from immunotherapy in advanced cancer types with different histology, including tumor types not included in clinical trials.29 30 However, the use of real-world data for regulatory purposes is still debated. In conclusion, our study identified cut-offs for TMB assessment with CGP panels widely used in clinical research and clinical practice. These findings may contribute to the uptake of TMB as a possible biomarker for the selection of patients who have a better chance of benefiting immunotherapy.

    Data availability statement

    Data are available in a public, open access repository. The data that support the findings of this study are in Zenodo, June 8, 2023 Version v1 at https://doi.org/10.5281/zenodo.8016300.

    Ethics statements

    Patient consent for publication

    Ethics approval

    Not applicable.

    References

    1. ↵
      1. Pardoll DM
      . The blockade of immune checkpoints in cancer immunotherapy. Nat Rev Cancer 2012;12:252–64. doi:10.1038/nrc3239
      OpenUrlCrossRefPubMedWeb of Science
    2. ↵
      1. Vaddepally RK,
      2. Kharel P,
      3. Pandey R, et al
      . Review of indications of FDA-approved immune checkpoint inhibitors per NCCN guidelines with the level of evidence. Cancers (Basel) 2020;12:738. doi:10.3390/cancers12030738
    3. ↵
      1. Samstein RM,
      2. Lee C-H,
      3. Shoushtari AN, et al
      . Tumor mutational load predicts survival after immunotherapy across multiple cancer types. Nat Genet 2019;51:202–6. doi:10.1038/s41588-018-0312-8
      OpenUrlCrossRefPubMed
    4. ↵
      1. Chan TA,
      2. Yarchoan M,
      3. Jaffee E, et al
      . Development of tumor Mutation burden as an immunotherapy biomarker: utility for the oncology clinic. Ann Oncol 2019;30:44–56. doi:10.1093/annonc/mdy495
      OpenUrlPubMed
    5. ↵
      1. Marabelle A,
      2. Fakih M,
      3. Lopez J, et al
      . Association of tumour mutational burden with outcomes in patients with advanced solid tumours treated with pembrolizumab: prospective biomarker analysis of the multicohort, open-label, phase 2 KEYNOTE-158 study. Lancet Oncol 2020;21:1353–65. doi:10.1016/S1470-2045(20)30445-9
      OpenUrlCrossRefPubMed
    6. ↵
      1. Jardim DL,
      2. Goodman A,
      3. de Melo Gagliato D, et al
      . The challenges of tumor mutational burden as an immunotherapy biomarker. Cancer Cell 2021;39:154–73. doi:10.1016/j.ccell.2020.10.001
      OpenUrlCrossRefPubMed
    7. ↵
      1. Stenzinger A,
      2. Allen JD,
      3. Maas J, et al
      . Tumor mutational burden standardization initiatives: recommendations for consistent tumor mutational burden assessment in clinical samples to guide Immunotherapy treatment decisions. Genes Chromosomes Cancer 2019;58:578–88. doi:10.1002/gcc.22733
      OpenUrlCrossRefPubMed
    8. ↵
      1. Rizvi H,
      2. Sanchez-Vega F,
      3. La K, et al
      . Molecular determinants of response to anti-programmed cell death (PD)-1 and anti-programmed death-ligand 1 (PD-L1) blockade in patients with non-small-cell lung cancer profiled with targeted next-generation sequencing. J Clin Oncol 2018;36:633–41. doi:10.1200/JCO.2017.75.3384
      OpenUrlCrossRefPubMed
    9. ↵
      1. Marcus L,
      2. Fashoyin-Aje LA,
      3. Donoghue M, et al
      . FDA approval summary: pembrolizumab for the treatment of tumor mutational burden-high solid tumors. Clin Cancer Res 2021;27:4685–9. doi:10.1158/1078-0432.CCR-21-0327
      OpenUrlAbstract/FREE Full Text
    10. ↵
      1. Zehir A,
      2. Benayed R,
      3. Shah RH, et al
      . Mutational landscape of metastatic cancer revealed from prospective clinical sequencing of 10,000 patients. Nat Med 2017;23:703–13. doi:10.1038/nm.4333
      OpenUrlCrossRefPubMed
    11. ↵
      1. Jennings LJ,
      2. Arcila ME,
      3. Corless C, et al
      . Guidelines for validation of next-generation sequencing–based oncology panels: a joint consensus recommendation of the association for molecular pathology and college of American pathologists. J Mol Diagn 2017;19:341–65. doi:10.1016/j.jmoldx.2017.01.011
      OpenUrlCrossRefPubMed
    12. ↵
      1. Sung MT,
      2. Wang YH,
      3. Li CF
      . Open the technical black box of tumor mutational burden (TMB): factors affecting harmonization and standardization of panel-based TMB. Int J Mol Sci 2022;23:5097. doi:10.3390/ijms23095097
    13. ↵
      1. Do H,
      2. Dobrovic A
      . Sequence artifacts in DNA from formalin-fixed tissues: causes and strategies for minimization. Clin Chem 2015;61:64–71. doi:10.1373/clinchem.2014.223040
      OpenUrlAbstract/FREE Full Text
    14. ↵
      1. Sha D,
      2. Jin Z,
      3. Budczies J, et al
      . Tumor mutational burden as a predictive biomarker in solid tumors. Cancer Discov 2020;10:1808–25. doi:10.1158/2159-8290.CD-20-0522
      OpenUrlAbstract/FREE Full Text
    15. ↵
      1. Stenzinger A,
      2. Endris V,
      3. Budczies J, et al
      . Harmonization and standardization of panel-based tumor mutational burden measurement: real-world results and recommendations of the quality in pathology study. J Thorac Oncol 2020;15:1177–89. doi:10.1016/j.jtho.2020.01.023
      OpenUrlCrossRefPubMed
    16. ↵
      1. Merino DM,
      2. McShane LM,
      3. Fabrizio D, et al
      . Establishing guidelines to harmonize tumor mutational burden (TMB): in silico assessment of variation in TMB quantification across diagnostic platforms: phase I of the friends of cancer research TMB harmonization project. J Immunother Cancer 2020;8:e000147. doi:10.1136/jitc-2019-000147
    17. ↵
      1. Ramos-Paradas J,
      2. Hernández-Prieto S,
      3. Lora D, et al
      . Tumor mutational burden assessment in non-small-cell lung cancer samples: results from the TMB 2 harmonization project comparing three NGS panels. J Immunother Cancer 2021;9:e001904. doi:10.1136/jitc-2020-001904
    18. ↵
      1. Vega DM,
      2. Yee LM,
      3. McShane LM, et al
      . Aligning tumor mutational burden (TMB) quantification across diagnostic platforms: phase II of the friends of cancer research TMB harmonization project. Ann Oncol 2021;32:1626–36. doi:10.1016/j.annonc.2021.09.016
      OpenUrlCrossRefPubMed
    19. ↵
      1. Esposito Abate R,
      2. Cheetham MH,
      3. Fairley JA, et al
      . External quality assessment (EQA) for tumor mutational burden: results of an international IQN path feasibility pilot scheme. Virchows Arch 2023;482:347–55. doi:10.1007/s00428-022-03444-y
      OpenUrl
    20. ↵
      1. Normanno N,
      2. De Luca A,
      3. Abate RE, et al
      . Current practice of Genomic profiling of patients with advanced solid tumours in Italy: the Italian register of actionable mutations (RATIONAL) study. Eur J Cancer 2023;187:174–84. doi:10.1016/j.ejca.2023.03.027
      OpenUrl
    21. ↵
      1. Offin M,
      2. Rizvi H,
      3. Tenet M, et al
      . Tumor mutation burden and efficacy of EGFR-tyrosine kinase inhibitors in patients with EGFR-mutant lung cancers. Clin Cancer Res 2019;25:1063–9. doi:10.1158/1078-0432.CCR-18-1102
      OpenUrlAbstract/FREE Full Text
    22. ↵
      1. Cristescu R,
      2. Aurora-Garg D,
      3. Albright A, et al
      . Tumor mutational burden predicts the efficacy of Pembrolizumab monotherapy: a pan-tumor retrospective analysis of participants with advanced solid tumors. J Immunother Cancer 2022;10:e003091. doi:10.1136/jitc-2021-003091
    23. ↵
      1. McGrail DJ,
      2. Pilié PG,
      3. Rashid NU, et al
      . High tumor mutation burden fails to predict immune checkpoint blockade response across all cancer types. Ann Oncol 2021;32:661–72. doi:10.1016/j.annonc.2021.02.006
      OpenUrlCrossRefPubMed
    24. ↵
      1. Cristescu R,
      2. Mogg R,
      3. Ayers M, et al
      . Pan-tumor genomic biomarkers for PD-1 checkpoint blockade-based Immunotherapy. Science 2018;362. doi:10.1126/science.aar3593
    25. ↵
      1. Litchfield K,
      2. Reading JL,
      3. Puttick C, et al
      . Meta-analysis of tumor- and T cell-intrinsic mechanisms of sensitization to checkpoint inhibition. Cell 2021;184:596–614. doi:10.1016/j.cell.2021.01.002
      OpenUrlCrossRefPubMed
    26. ↵
      1. Ricciuti B,
      2. Wang X,
      3. Alessi JV, et al
      . Association of high tumor mutation burden in non-small cell lung cancers with increased immune infiltration and improved clinical outcomes of PD-L1 blockade across PD-L1 expression levels. JAMA Oncol 2022;8:1160–8. doi:10.1001/jamaoncol.2022.1981
      OpenUrl
    27. ↵
      1. Lee JS,
      2. Ruppin E
      . Multiomics prediction of response rates to therapies to inhibit programmed cell death 1 and programmed cell death 1 ligand 1. JAMA Oncol 2019;5:1614–8. doi:10.1001/jamaoncol.2019.2311
      OpenUrl
    28. ↵
      1. Cuppens K,
      2. Baas P,
      3. Geerdens E, et al
      . HLA-I diversity and tumor mutational burden by comprehensive next-generation sequencing as predictive biomarkers for the treatment of non-small cell lung cancer with PD-(L)1 inhibitors. Lung Cancer 2022;170:1–10. doi:10.1016/j.lungcan.2022.05.019
      OpenUrl
    29. ↵
      1. Aggarwal C,
      2. Ben-Shachar R,
      3. Gao Y, et al
      . Assessment of tumor mutational burden and outcomes in patients with diverse advanced cancers treated with immunotherapy. JAMA Netw Open 2023;6:e2311181. doi:10.1001/jamanetworkopen.2023.11181
    30. ↵
      1. Gandara DR,
      2. Agarwal N,
      3. Gupta S, et al
      . Tumor mutational burden (TMB) measurement from an FDA-approved assay and real-world overall survival (rwOS) on single-agent immune checkpoint inhibitors (ICI) in over 8,000 patients across 24 cancer types. JCO 2023;41:2503. doi:10.1200/JCO.2023.41.16_suppl.2503
      OpenUrl
    31. ↵
      1. Provencio M,
      2. Serna-Blasco R,
      3. Nadal E, et al
      . Overall survival and biomarker analysis of neoadjuvant nivolumab plus chemotherapy in operable stage IIIA non-small-cell lung cancer (NADIM phase II trial). J Clin Oncol 2022;40:2924–33. doi:10.1200/JCO.21.02660
      OpenUrl
    32. ↵
      1. Heeke S,
      2. Benzaquen J,
      3. Long-Mira E, et al
      . In-house implementation of tumor mutational burden testing to predict durable clinical benefit in non-small cell lung cancer and melanoma patients. Cancers (Basel) 2019;11:1271. doi:10.3390/cancers11091271
    33. ↵
      1. Mankor JM,
      2. Paats MS,
      3. Groenendijk FH, et al
      . Impact of panel design and cut-off on tumour mutational burden assessment in metastatic solid tumour samples. Br J Cancer 2020;122:953–6. doi:10.1038/s41416-020-0762-5
      OpenUrlCrossRefPubMed
    34. ↵
      1. Fenizia F,
      2. Wolstenholme N,
      3. Fairley JA, et al
      . Tumor mutation burden testing: a survey of the international quality network for pathology (IQN path). Virchows Arch 2021;479:1067–72. doi:10.1007/s00428-021-03093-7
      OpenUrl
    35. ↵
      1. Stein MK,
      2. Pandey M,
      3. Xiu J, et al
      . Tumor mutational burden is site specific in non–small-cell lung cancer and is highest in lung adenocarcinoma brain metastases. JCO Precis Oncol 2019;3:1–13. doi:10.1200/PO.18.00376
      OpenUrl
    36. ↵
      1. Ready N,
      2. Hellmann MD,
      3. Awad MM, et al
      . First-line nivolumab plus Ipilimumab in advanced non-small-cell lung cancer (checkmate 568): outcomes by programmed death ligand 1 and tumor mutational burden as biomarkers. J Clin Oncol 2019;37:992–1000. doi:10.1200/JCO.18.01042
      OpenUrlPubMed
    37. ↵
      1. Mosele F,
      2. Remon J,
      3. Mateo J, et al
      . Recommendations for the use of next-generation sequencing (NGS) for patients with metastatic cancers: a report from the Esmo precision medicine working group. Ann Oncol 2020;31:1491–505. doi:10.1016/j.annonc.2020.07.014
      OpenUrlCrossRefPubMed
    38. ↵
      1. Bayle A,
      2. Bonastre J,
      3. Chaltiel D, et al
      . ESMO study on the availability and accessibility of biomolecular technologies in oncology in Europe. Ann Oncol 2023;34:934–45. doi:10.1016/j.annonc.2023.06.011
      OpenUrl
    39. ↵
      1. Normanno N,
      2. Apostolidis K,
      3. Wolf A, et al
      . Access and quality of biomarker testing for precision oncology in Europe. Eur J Cancer 2022;176:70–7. doi:10.1016/j.ejca.2022.09.005
      OpenUrlPubMed
    40. ↵
      1. Stenzinger A,
      2. Moltzen EK,
      3. Winkler E, et al
      . Implementation of precision medicine in Healthcare—A European perspective. J Intern Med 2023;294:437–54. doi:10.1111/joim.13698
      OpenUrl
    41. ↵
      1. Raphael MJ,
      2. Gyawali B,
      3. Booth CM
      . Real-world evidence and regulatory drug approval. Nat Rev Clin Oncol 2020;17:271–2. doi:10.1038/s41571-020-0345-7
      OpenUrl

    Supplementary materials

    Footnotes

    • Contributors NN contributed to conceptualization. RP, AS, MRM, DF and VS contributed to methodology. RP, AS, VS and PC contributed to formal analysis and investigation. NN, REA and VS contributed to writing-original draft preparation. RP, AS, VS, MRM, DF and PC contributed to writing-review and editing. NN contributed to funding acquisition. NN contributed to supervision. NN is the guarantor. All authors read and approved the final manuscript.

    • Funding IQN Path is a non-profit organization. Thermo Fisher Scientific, QIAGEN, Roche, Merck KGaA, Illumina, Genentech, Bristol Myers Squibb (BMS) and AstraZeneca supported the study through an unrestricted grant that included the provision of the assays for the four diagnostic companies. TMB testing (with the exception of F1CDx) was performed in an independent, academic laboratory (Cell Biology and Biotherapy Unit, Istituto Nazionale Tumori "Fondazione G. Pascale"-IRCCS, Naples, Italy). Statistical analysis, interpretation of the results and writing of the manuscript were handled by the scientific committee of the project identified by IQN Path and composed by the authors of the manuscript. The study is also supported by a grant from the Italian Ministero della Salute (Ricerca Finalizzata; grant no. GR-2018-12366829).

    • Competing interests None declared.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.