Article Text

Download PDFPDF

Imaging approaches and radiomics: toward a new era of ultraprecision radioimmunotherapy?
  1. Roger Sun1,2,
  2. Théophraste Henry2,3,
  3. Adrien Laville2,
  4. Alexandre Carré2,
  5. Anthony Hamaoui2,
  6. Sophie Bockel1,2,
  7. Ines Chaffai2,
  8. Antonin Levy1,2,
  9. Cyrus Chargari2,4,
  10. Charlotte Robert1,2 and
  11. Eric Deutsch1,2,5
  1. 1 Department of Radiation Oncology, Gustave Roussy, Villejuif, France
  2. 2 Radiothérapie Moléculaire et Innovation Thérapeutique, Université Paris-Saclay, Institut Gustave Roussy, Inserm, Villejuif, France
  3. 3 Department of Nuclear Medicine, Gustave Roussy, Villejuif, France
  4. 4 Department of Radiation Oncology, Brachytherapy Unit, Gustave Roussy, Villejuif, France
  5. 5 INSERM U1030, Gustave Roussy, Villejuif, France
  1. Correspondence to Prof. Eric Deutsch; eric.deutsch{at}


Strong rationale and a growing number of preclinical and clinical studies support combining radiotherapy and immunotherapy to improve patient outcomes. However, several critical questions remain, such as the identification of patients who will benefit from immunotherapy and the identification of the best modalities of treatment to optimize patient response. Imaging biomarkers and radiomics have recently emerged as promising tools for the non-invasive assessment of the whole disease of the patient, allowing comprehensive analysis of the tumor microenvironment, the spatial heterogeneity of the disease and its temporal changes. This review presents the potential applications of medical imaging and the challenges to address, in order to help clinicians choose the optimal modalities of both radiotherapy and immunotherapy, to predict patient’s outcomes and to assess response to these promising combinations.

  • radioimmunotherapy
  • radiotherapy
  • immunotherapy
  • tumor biomarkers

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See

Statistics from

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.


Immunotherapy has revolutionized cancer management in recent years. Most advanced patients receiving frontline immune checkpoint inhibitors (ICIs) will, however, develop disease progression. In order to improve the response rates of immunotherapy, a growing number of studies are evaluating different treatment combinations or strategies, one of which is the combination of systemic immunotherapy and local radiotherapy.1–7 Indeed, there is substantial evidence that radiotherapy induces immune stimulation with the release of tumor antigens and proinflammatory cytokines at the local and systemic levels and thus, may act synergistically with immunotherapy in the systemic control of advanced metastatic cancers.1 8–12 Radiotherapy activates immune response by triggering immunogenic cell death (with extracellular release of release of high-mobility group box 1 protein and adenosine-5′-triphosphate, and cell surface translocation of calreticulin) and production of type I interferon (IFN) via the activation of the cGAS-STING pathway, leading to dendritic cells and cytotoxic T cells activation, which may eventually lead to an out-of-field ‘abscopal’ response.13 14 Inversely, it has been shown that radiotherapy also promotes immunosuppressive effects, with notably a PD-L1 upregulation via different pathway such as IFN-dependent pathway and IL-6/JAK/STAT pathway,15 or DNA damages through ATM/ATR/Chk1 kinase activation,16 along with the attraction of immunosuppressive cells (ie, tumor-associated macrophages, myeloid-derived suppressor cells, and regulatory T cells) and the release of immunosuppressive cytokines such as TGF-β and IL-10.13 These results support the potential of immunotherapy as an adjunct to radiotherapy to counteract its immunosuppressive effects, as illustrated by the phase III PACIFIC trial showing a benefit of a consolidation therapy with the anti-PD-L1 drug durvalumab in unresectable locally advanced non-small cell lung cancer (NSCLC) after chemoradiotherapy.17

However, only a small number of controlled randomized trials evaluating the benefit of adding radiotherapy to immunotherapy exist with mostly negative results.3 4 7 18–20 For instance, in advanced NSCLC patients, a pooled analysis of two clinical trials evaluating pembrolizumab (anti-PD1)±radiotherapy showed that adding radiotherapy to pembrolizumab improved outcomes. However, a randomized phase 2 trial was not able to identify a benefit of low-dose or high-dose radiation therapy with durvalumab (anti-PD-L1)+tremelimumab (anti-CTLA4) in advanced immunotherapy-pretreated NSCLC patients.4

Irradiation modalities to optimize the achievement of a systemic antitumor immune response are then subject to numerous studies.10 21–23

Fine tuning delivered dose and fractionation (dose per session) may be crucial for achieving an out-of-field abscopal response.24–27 For example, an abscopal response was obtained in immunocompetent mice bearing syngeneic mammary and colorectal tumor cell transplants after hypofractionated radiotherapy (three sessions of 8 Gy delivered 3 days in a row) but was not obtained with a single fraction radiotherapy of 20 Gy.24 Doses above 12 Gy increase the activity of 3′ repair exonuclease 1, a DNA exonuclease that degrades DNA accumulating in the cytosol of irradiated cells, decreasing STING activation and Batf3-dependent dendritic cell recruitment.28 These studies would suggest that the optimal dose in radioimmunotherapy may be lower than the maximum tolerated doses used in radiotherapy alone.22 However, low dose radiotherapy (1 Gy/fraction) has also been shown to potentially stimulate immune responses by activating and stimulating helper T cells, promoting a generation of immune memory and modulating the tumor stroma with the reduction of TGF-β induced Tregs in the tumor microenvironment.29 30

Open questions such as the number and choice of lesions to irradiate remain unanswered.31 32 Indeed, the tumor heterogeneity of extensive disease could limit the likelihood of a systemic (abscopal) antitumor immune response induced by radiotherapy, especially in the case of single-site irradiation.21 Thus, a wide range of possibilities in the choice of doses and volumes to irradiate makes possible combinations more complex.

In this context where both disease and treatment are characterized by heterogeneity, medical imaging may express its full potential as it allow analysis of the entire tumor mass and individual lesions, unlike traditional biopsies.33 Indeed, each lesion may have a different microenvironment and may receive different local and/or systemic treatment effects, leading to heterogeneous response profiles.34 Imaging-derived biomarkers especially may help to better understand the relationship between radiotherapy and immunotherapy and to eventually guide the clinician in the choice of treatment and its modalities.

Here, we will present the potential applications and challenges of imaging to guide immunotherapeutic combinations with a particular focus on radiomics. We will also discuss the particularities to be taken into account in imaging for the evaluation of response in these new combinations.

Imaging to increase knowledge, to predict responses, and to guide therapeutics

A brief introduction to computational medical imaging principles

Artificial intelligence approaches applied to medical imaging have been particularly developed in recent decades with great results for detection tasks, image reconstruction, and generation of imaging biomarkers for diagnostic support, prognostic assessment, or treatment response prediction.33 35 36 The term ‘radiomics’ was introduced by Gillies et al 37 in 2010 to define the process of computational translation of medical imaging into high-dimensional quantitative data (similar to other ‘omics’) and the use of machine learning for the development of imaging biomarkers. A typical pipeline of analysis is usually constituted by the following steps: (1) definition of the volumes of interest (VOIs) to be analyzed, (2) extraction of the quantitative information represented by the radiomics features, and (3) application of machine learning approaches and evaluation of the validity and generalizability of the model.35 38 While machine learning methods may seem complex at first glance to the clinician, we will outline some general and relatively simple concepts that allow for a better understanding of this discipline.

Computational translation of the image into numbers relies either on the calculation of ‘classical’ radiomic features (in particular from the histogram of the image intensities, texture matrices, or the shape of the VOI, defined a priori and which can be extracted by different software such as LIFEx39 or Pyradiomics40), or on deep learning approaches that allow ‘discovering’ the relevant information directly from the raw image (‘deep features’) using neural networks (especially conventional neural networks).41 While classical radiomic features are mostly used for biomarker development, neural networks can also be used for image detection, segmentation and reconstruction tasks. This data extraction can be performed in one or more volume of interest to be analyzed (eg, a tumor lesion), or on the whole image (especially when using deep learning techniques). It is important to note that the different acquisition protocols and reconstruction parameters of the imaging process (CT, MRI, or positron emission tomography scan (PET-CT)) directly affect the image and may be a confounding factor. This can particularly impact the performance of algorithms in the case of heterogeneous cohorts. Thus, image preprocessing and normalization steps are performed to make the images of different patients with different acquisition protocols comparable.42 43

Once the quantitative data have been extracted from the image, machine learning algorithms aim to train a model for a particular task. Classically, three distinct data sets are used in machine learning approaches: a training set to learn a specific task, a validation set to optimize the algorithm and an independent test set for the final validation and evaluation of the generalizability of the algorithm. Assessment of the model performance in an independent data set is essential to ensure the external validity of an algorithm. Indeed, overfitting may happen during the training, especially when the number of features extracted is higher than the number of patients, leading to overoptimistic performance estimation and subpar performance when used in real clinical applications.

A growing number of initiatives and recommendations are being issued to ensure the quality, the reproducibility and the methodological validity of artificial intelligence (AI) approaches applied on medical images.44–46 Harmonization of imaging data and/or radiomic features a posteriori is essential to enable robust development of generalisable imaging biomarkers in a multicenter setting.47 Although standardization of image acquisition protocols is a simple way to obtain consistent images across different centers,48 it can be difficult to implement on a large scale and does not take into account the potential batch effect of acquisitions from multiple devices from different manufacturers. Image processing methods, including at least isotropic spatial resampling of voxels and, for example, skull stripping for image intensity normalization (using, eg, the Z-score method), and bias field correction for brain MRI images, have been shown to be effective for medical imaging normalization.49 50 Adjusting radiomic features after extraction to correct for batch effect using methods such as the ComBat approach has also shown interesting results in reducing discrepancies between different imaging protocols and manufacturers.51 Standardization of radiomic feature definitions have also been proposed by the image biomarker standardization initiative to increase the reproducibility and comparability of software used for feature extraction.52 Regarding deep learning studies, some recommendations to ensure their proper conduct have also recently been published.53 The Radiomic Quality Score calculated from a 16-point checklist allows the assessment of the various technical and statistical aspects and the clinical validity of the radiomic analysis method.54

Radiomics to predict response to immunotherapy and radiotherapy combinations: toward a selective destruction of predicted immune refractory lesions and immunogenic low-dose radiotherapy for abscopal effect friendly lesions?

Since the first studies evaluating radiomics to predict the immune microenvironment in 2017,55 an increasing number of radiomic studies have confirmed the potential of imaging biomarkers for use in immuno-oncology (table 1). Promising results have been obtained using a wide variety of approaches on many different tumors types, such as lung cancers,56–61 head and neck cancers,62 melanoma,56 63 64 urothelial65–67 and renal cancers,68 with either CT scans,56–58 69 PET-CTs,60 70 or MRIs71 72 (figure 1).

Figure 1

Overview of the different methodologies and strategies applicable for developing imaging biomarkers. Adapted from Sun et al Artificial intelligence, radiomics and pathomics to predict response and survival of patients treated with radiations. Cancer/Radiothérapie 2021; Volume 25 (Issues 6–7): 630–637. Copyright 2021 Elsevier Masson SAS. All rights reserved.

Table 1

Summary of radiomics study evaluating response to radioimmunotherapy and immunotherapy

One of the major considerations for implementation of imaging biomarkers in radiotherapy and immunotherapy combinations in a metastatic context is whether they can be used to assess the spatial heterogeneity of the disease and more specifically the heterogeneity between lesions of a same patient to potentially guide radiotherapy on specific lesions (figure 2).

Figure 2

Potential clinical interest of imaging biomarkers for radiotherapy-immunotherapy combinations. Lesion-level analyses may help to identify potential immune-refractory lesions (high-risk lesions) needing focal destruction with ablative SBRT (high dose RT) or immunogenic lesions for which low dose RT may improve systemic response. Patient-level analyses allow overall response prediction. RT, radiotherapy; SBRT, stereotactic body radiation therapy.

To the best of our knowledge, only two studies have evaluated radiomics to predict response of metastatic cancer patients treated with immunotherapy combined with radiotherapy.73 74 Interestingly, each of these independently evaluated the previously published radiomics signature predicting CD8-T cells (CD8-Rscore) developed by our team.69 This CD8-Rscore was initially developed using contrast enhanced CT scans of 135 patients for whom RNA sequencing data of a biopsied lesion were available and allowed estimation of the abundance of CD8 T cells. This signature, based on eight variables including five radiomics features extracted from the lesion analyzed and a peripheral ring around the lesion, two variables of VOI location and one technical variable (the kiloVoltage Peak), was validated using a Cancer Genome Atlas dataset of 119 patients (area under the curve (AUC)=0.67; 95% CI 0.57 to 0.77; p=0.0019)69 and was also associated with clinical response and overall survival of 137 patients treated with immunotherapy (HR=0.58, 95% CI 0.39 to 0.87; p=0.0081). However, in the primary study, only one lesion per patient was analyzed for predicting outcomes.69 In ref 73, we applied the CD8-Rscore on multiple lesions on the baseline CT scan of 94 solid cancer patients treated with immunotherapy and radiotherapy. Almost all patients underwent hypofractionated radiotherapy (n=91 (96.9%)). The main radiotherapy dose-fractionation schedule was 3×8 Gy delivered on a single lesion. Seven patients (7.4%) had more than one irradiated lesion. The CD8-Rscore was applied on a total of 100 irradiated lesions and 189 non-irradiated lesions. At a lesion level, the CD8-Rscore at baseline of responding lesions was higher than those of stable or progressive lesions (AUC=0.63, 95% CI 0.56 to 0.71), p=0.0020). To assess whether the CD8-Rscore of multiple lesions may be aggregated to predict outcomes at a patient level, several metrics consisting of the mean value of the CD8-Rscores, its maximal and the minimal values, the SD, and the entropy of the distribution of the CD8-Rscores have been evaluated, using median value to dichotomize two groups of patients. In this study, the entropy of the distribution of the CD8-Rscores was associated with patient progression-free survival (PFS) (HR=1.67, p=0.040), out-of-field abscopal response (AUC=0.70, p=0.014) and overall survival (HR=2.08, p=0.023). The minimal value of the CD8-Rscores, representing the lesion considered to be the least infiltrated by CD8 T cells by the radiomics signature, was also interesting with a statistical trend identified with PFS (p=0.07). Significance was reached when looking at the extreme values (first vs third quartile, HR=0.34,95% CI 0.17 to 0.68, p=0.0021). This last metric has the advantage to allow direct designation of the most pejorative lesion in a non-invasive manner, which are more likely to be non-responsive to immunotherapy and eventually may benefit from ablative targeted radiotherapy (high dose radiotherapy).

Korpics et al 74 also evaluated the CD8-Rscore in a cohort of 68 patients with advanced solid cancer patients treated with multisite stereotactic body radiation therapy (SBRT) and pembrolizumab. Two to four metastases per patient were irradiated, with a prescription dose of 30–50 Gy in 3–5 fractions. A total of 139 metastases were treated with SBRT and evaluated using the CD8-Rscore. Low pretreatment CD8-Rscore was defined as a CD8-Rscore lower than the 25% percentile in their study. The average of the CD8-Rscores of irradiated lesions of each patient was used to evaluate the clinical outcomes at a patient-level. High CD8-Rscore was associated with improved lesion control (HR=0.18; 95% CI 0.04 to 0.74; p=0.018). At a patient-level, high scores were correlated with better PFS (HR 0.47; 95% CI 0.26 to 0.85; p=0.013) and OS (HR=0.39; 95% CI 0.20 to 0.75; p=0.005). As a continuous variable, higher scores were also associated with improved PFS (HR=0.12; 95% CI 0.03 to 0.51; p=0.004) but did not reach significance for overall survival and local control.74 Overall, although these two studies evaluating the CD8-Rscore were retrospective and need prospective validation, they illustrate the potential of radiomics for lesion response and patient outcomes prediction in the context of radiotherapy and immunotherapy combinations. However, it remains unclear whether this type of signature, developed in pan-cancer cohorts, can be as effective in the case of homogeneous cohort of patients with specific tumor type. Indeed, tumor type might be a confounding factor since tumor immune infiltrate may vary according to tumor types.75 The development of tumor-specific signatures could therefore improve the performance of these models.

Regarding the prediction of out-of-field ‘abscopal’ response, a preclinical study evaluated CT and MRI radiomics on Lewis Lung Carcinoma in a syngeneic, subcutaneous murine model.76 Tumors were implanted on both flanks of 19 mice. Fifteen mice received radiotherapy on the right flank (3×8 Gy) followed by immunotherapy (anti PD-1), and four mice presented abscopal response on the left flank. A total of 92 CT and 92 MRI pretreatment radiomics features were extracted from each lesion (right and left flanks). Using six mice (12 tumors) for training and nine mice (18 tumors for validation), the authors identified some pretreatment CT and MRI features and a biological variable, the neutrophil-to-lymphocyte ratio, that were associated with abscopal response. This proof-of-concept study, although very promising, must be interpreted with caution due to the low number of mice and events (occurrence of abscopal response).76 However, it suggests that radiomics may help to identify lesions favorable to the occurrence of an abscopal response. Such biomarkers would be of particular interest to guide potential low-dose immunogenic radiotherapy to improve systemic responses.29 30

It is interesting to note that, vice versa, these imaging tools may help to indicate lesions—and patients—with a high chance of response under immunotherapy and not requiring radiotherapy. This is of particular interest in order to optimize and limit radiotherapy as parsimoniously as possible. Indeed, side effects of radiotherapy have to be taken into account, in particular lymphopenia, which may decrease the response to immunotherapy.77 In the same spirit, analyses of intratumor heterogeneity could also be of interest for the partial irradiation of tumors, which is also an approach being explored in the combination of radiotherapy and immunotherapy. Moreover, computational imaging approaches have also shown promising results for intratumor heterogeneity assessment.78 79 This could be key for guiding partial tumor irradiation, which is also an approach being explored in radioimmunotherapy with the concept of ‘adscopal effect’, the immune mediated effect of radiotherapy close to the irradiated field.80

Spatial heterogeneity assessment: the key to allow characterization and targeting of lesions

Aggregating spatial heterogeneity from predictions made at a lesion level to perform predictions at a patient level is particularly challenging, and no consensual method is clearly defined (figure 3). Indeed, such models need to take into account the possibility for each patient to have different numbers of lesions and to be robust to variation in lesion selection if exhaustive delineation is not performed. This may explain why the majority of radiomics studies only analyzed a single lesion to predict outcomes of patients (table 2). Some relatively simple ways to analyze several lesions are the aggregation of either all the lesions delineated into one volume corresponding to the tumor burden before the extraction of features,81 or the aggregation of the features extracted from several lesions using metrics such as the average value, the minimal or the maximal value of the features, irrespective of the lesion from which the value was extracted.65 82 However, although they have shown very interesting results to predict response or survival of patients treated with immunotherapy, these methods do not allow evaluation of the aggressiveness or the immunogenicity of a particular lesion, which is determinant in order to identify a possible target for destruction in radiotherapy (cooperative effect of radiotherapy) or to promote an abscopal response (synergistic effect of radiotherapy). Similarly, some authors have trained a lesion-wise radiomics model to predict patient response by assigning the same patient response to each different lesions.66 This, however, does not take into account the possibility of dissociated response, with presence of both responding and progressive lesions.

Figure 3

Summary of different aggregating methods for multiple lesion analyses. (A) Defining a fixed number of a lesion to analyze for each patient. (B) Aggregating lesions into one volume (tumor burden). (C) Aggregating features extracted from several lesions (ie, using metrics such as the average value or the minimal value). (D) Predicting lesion-level response then aggregating the predictions to assess patient outcomes. (E and F) Assigning for each lesion the patient outcome to predict then using predefined aggregation metrics (E) or learned aggregation methods (attention) in multiple-instance learning approaches (F).

Trebeschi et al 56 have applied a very interesting method to address these challenges. In a study analyzing 1055 primary and metastatic lesions from 203 patients with advanced melanoma and NSCLC treated with anti-PD1, they trained a lesion-wide model predicting lesion progression. Best performances were obtained with NSCLC pulmonary and nodal metastases (0.83 AUC, Mann-Whitney U test p<0.001 and 0.78 AUC, p<0.001, respectively). By combining predictions made on individual lesions, this method allowed identification of patients with at least one predicted progressive lesion. The patients experiencing either a uniform progressive disease or a dissociated response had lower overall survival than patients with a uniform response (0.76 AUC for all patients).56 This study highlighted the potential of imaging to identify high-risk lesions for predicting survival of patient treated with immunotherapy.

Only few studies have compared the performance of different aggregation methods for multiple lesions.73 82 83 Liu et al compared the predictive value of radiomic features extracted from the largest lesion (LL model) with the average of features extracted from several target lesions defined according to RECIST V.1.184 (TL model) with a maximum of five lesions analyzed, at baseline (pretreatment), as well as the relative change of these features on the follow-up CT (delta-radiomics), in a cohort of 197 NSCLC patients (and 322 target lesions) treated with anti-PD1 immunotherapy. LL and TL models seemed comparable in their study, with good performance when using delta-radiomics to predict responders at 6 months; however, none of the methods showed significant predictive value when using only the baseline features.82 Chang et al evaluated six different aggregating methods of radiomics features extracted from brain metastases: ‘Unweighted average’ of radiomic features (except for the size and shape features which were summed), ‘Weighted average’ of radiomic features according to the total volume of all metastases, ‘Weighted average of largest three metastases’, ‘Largest+number of metastases’, and ‘Largest metastasis and Smallest metastasis’.83 The ‘volume-weighted average of the largest three tumors’ model had the best performance for predicting patient survival with C-index of 0.627 (0.595–0.661), 0.628 (0.591–0.666), and 0.652 (0.565–0.727) for Cox proportional hazards model, LASSO and random forest models, respectively. In our study evaluating the CD8-Rscores in patients treated with radiotherapy and immunotherapy, we have shown that entropy of the distribution of the score and the least score were metrics to assess patient outcomes, while average or maximal score were not.73 Interestingly, Van den Eynde et al have shown similar results with an immunohistochemistry and digital pathology-based scoring system of lymphocyte populations (CD3 and CD8), the Immunoscore, in metastatic colorectal cancer patients.85 86 They evaluated the Immunoscore in 603 resected metastases from 222 patients. The value of the least infiltrated lesion had a stronger association with patient outcomes than the most infiltrated lesion or the average value of the different lesions of a patient.

Evaluating spatial heterogeneity also raises questions of the effect of the number of lesions chosen for analysis, as well as the minimal number of lesions to analyze, in order to obtain maximum information regarding disease heterogeneity, and whether a comprehensive analysis of all lesions is mandatory. These questions have been studied particularly for the number of lesions to be measured in radiology87 but remains pending for radiomic analyses.

Finally, advanced machine learning approaches such as deep learning and multiple instance learning (MIL) may be key for spatial heterogeneity assessment.88–90 Unlike the methods described previously where one class label is assigned to each observation or ‘instance’, in MIL, class labels are assigned to a ‘bag of instance’, which is a collection of observations. Here, the bag of instance could be the patient global response, where the instances would be the different lesions. The challenge of the MIL approach is to determine which instance(s) influence the overall bag label. This approach is particularly interesting since obtaining the annotation and labels of each lesion is difficult and time-consuming, while patient response label is easier to obtain.

Other promising immunotherapy radiomics biomarker

Radiomics biomarkers have shown promising results to predict several biological and histological features that can be used as a rationale for characterizing the patient’s disease and developing radiomic scores predictive of response to high or low doses of radiotherapy. A study used a deep learning method in a cohort of 327 patients with NSCLC to predict tumor mutational burden (TMB) (AUC=0.81, 95% CI 0.77 to 0.85) in the test cohort).91 This radiomic signature of TMB was associated with overall survival, in an independent cohort of NSCLC patients treated with immunotherapy (OS: HR=0.54; 95% CI 0.31 to 0.95; p=0.030, and PFS: HR=1.78; 95% CI 1.07 to 2.95; p=0.023).91

Tunali et al 92 have developed a radiomic signature predicting overall survival of NSCLC patients treated with immunotherapy using a large cohort of 180 patients for training, and two cohorts of 90 and 62 patients for validation. Interestingly, the signature they identified, allowing discrimination of four risk groups was based on one radiomic feature: gray level co-occurrence matrix inverse difference, which has been associated with the expression of CAIX, an hypoxia-related gene in their analyses.92

Other radiomic studies, although conducted without immunotherapy, have also shown interesting results in large cohorts for the prediction of lymphocyte infiltration according to the Immunoscore on MRI images,71 93 PD-L1 on either CT94 95 or PET images,70 or of microsatellite instability on CT or MRI images96 97 and may also show promise for the prediction of response to immunotherapy. The diversity of biological mechanisms that can be assessed by radiomics is of definite interest for the development of new immunotherapy combinations such as myeloid-derived suppressor cells (MDSC)-targeting immunotherapy.98

While both biologically driven radiomic signatures (ie, predicting TILs,69 PD-L199 or TMB91) and radiomic signatures directly designed to predict patient outcomes from images seem to achieve good performance to predict response to immunotherapy (table 1), both strategies have their advantages and disadvantages. Using radiomics to estimate a biological intermediate biomarker allows to bring a rational to the interpretation of the patient’s outcomes predictions, which makes the signature appearing less like a ‘black box’. However, the final performance on the output of clinical interest, for example, OS or PFS, may be limited by this two-step process and the associated accumulation of errors. Radiomic signatures trained without any biological assumptions are less interpretable but may help to discover new mechanisms and to generate new hypotheses. Thus, some radiomic signatures have shown to be independent from known biological biomarkers, which paves the way for potential multimodal signatures integrating genomics and TMB, pathomics, biology and/or metabolomics.95 100 101 Trebeschi et al 102 have developed an AI-derived score predicting 1-year survival of advanced NSCLC patients treated with nivolumab from morphological changes on chest CT acquired during patient’s follow-up (73 patients in the training set, 79 in the test set). This showed that their tool outperformed PD-L1 expression (>1%) and volumetric changes of tumor burden in a multivariate analysis; however, it was only evaluable for 22 patients. Khorrami et al 58 have shown in a similar population that a delta-radiomic feature combined with PD-L1 was better than PD-L1 alone to predict patient overall survival. Finally, Luo et al have shown in a cohort of 247 NSCLC patients treated with immunotherapy that a multimodal signature integrating radiology, pathology and genomic data was able to predict response to anti-PD-1 (AUC=0.80, 95% CI 0.74 to 0.86) and outperformed unimodal radiomic model (0.65, 95% CI [0.57 to 0.73), PD-L1 (AUC=0.73, 95% CI 0.65 to 0.81), and TMB (AUC=0.61, 95% CI 0.52 to 0.70).103

Despite this, a low level of evidence for radiomic studies remains, with many studies often performed in small numbers and some without independent testing cohorts.60 61 72 104 To date, only one study has a prospective testing cohort,59 and only the CD8-Rscore has had testing by an external team.74 Thus, further research still remains, prior to implementing strategies based on radiomics into clinical practice. Standardization of imaging protocols and prospective clinical trials will be key.

Contribution of functional imaging

Many functional and molecular imaging techniques have been developed in recent years with promising applications for radiotherapy and immunotherapy combinations.105 106

Functional images have the advantage of directly evaluating cellular functions or metabolic pathways, while machine learning approaches can only seek statistical associations of these processes with imaging patterns.

Immuno-PET is the combination of ultrasensitive functional imaging, PET imaging and the high affinity and specificity of monoclonal antibodies or ligands based on monoclonal antibodies. This approach allows detection, quantification and longitudinal monitoring of specific immune receptors or cells. Several studies have shown promising PET-imaging tracers allowing detection of PD1 expression such as the 64Cu-DOTA-PD-1,107 or PD-L1 expression such as the 89Zr-C4.108 Combined with radiotherapy, these approaches have helped increase knowledge by allowing evaluation of changes induced after radiation.109–112 For instance, several immune-PET studies have illustrated radiotherapy-induced PD-L1 upregulation on tumor cells.110–112 Christensen et al evaluated an 89Zr radiolabeled anti-PD-L1 immuno-PET tracer, the 89Zr-DFO-6E11, in tumor-bearing mice treated with radiotherapy with and without antimouse PD-L1 immunotherapy or with antimouse PD-L1 immunotherapy alone. The PET-CT imaging was performed in all mice and 72 hours after the selective irradiation of the tumor (2 Gy × 3) for irradiated mice. In their study, radiotherapy+anti-PD-L1 and ant-PD-L1 alone resulted in significant tumor control compared with the control group and RT alone. They reported that the 89Zr-DFO-6E11 allowed objective quantification of PD-L1 in tumors and spleens of irradiated mice. Moreover, the maximum 89Zr-DFO-6E11 tumor-to-muscle ratio was associated with treatment response (p=0.025). Several early-phase clinical trials have confirmed the potential of immuno-PET targeting PD-L1 approaches for immunotherapy with different promising tracers113–115: the 89Zr-atezolizumab,98 the 89Zr-CX-072, which is a PD-L1 targeting probody therapeutic, which is engineered to be activated in the tumor microenvironment by tumor-associated proteases,114 and the 89Zr-durvalumab.115

Immuno-PET approaches have also shown interesting results to monitor tumor-infiltrating CD8 T cells.116 117 Farwell et al 117 evaluated an anti-CD8 radiolabeled minibody, 89Zr-Df-IAB22M2C, in a phase 1 trial including 15 patients with metastatic solid tumors treated with immunotherapy (n=8), targeted therapy (n=2) and 5 treatment-naïve patients. Radiotracer uptake in tumors occurred in 10 patients, seven of whom were receiving immunotherapy. A post-treatment increase of the 89Zr-Df-IAB22M2C uptake in tumor lesions occurred in three patients treated with immunotherapy and was correlated with response, which is very promising for immunotherapy early response prediction. Whether the 89Zr-Df-IAB22M2C may characterize the impact of radiotherapy on CD8 T cell infiltration within irradiated and non-irradiated tumor lesions in patients with polymetastatic solid cancers is evaluated in an ongoing clinical trial (ABSCOTEP trial, EudraCT: 2016-000665-23).

Overall, these functional imaging approaches are very promising for immunotherapy-radiotherapy combinations and will surely help to better understand determinant of abscopal response and to identify the best targets for radiotherapy. However, these approaches do have limitations such as the difficult access to the tracers that may impact the repeatability of these exams, the need for standardization to allow reproducibility, and the need to take into account the tracer biological distribution, especially in certain normal lymphoid tissues such as spleen and nodes.110 112

Imaging to assess response to radioimmunotherapy

Addition of radiotherapy to immunotherapy has raised specific questions for response assessment.

The term ‘abscopal effect’ has been introduced to describe an immune-mediated response of lesions occurring after radiotherapy outside the radiation field.118 While rare abscopal responses have been described with radiotherapy alone, immunotherapy has been used to try to enhance the systemic effect of radiotherapy. However, the proper systemic benefit of radiotherapy added to the direct effect of immunotherapy is difficult to evaluate outside of a controlled trial. Moreover, definition of abscopal effect varies between studies. Some have considered as abscopal response, a reduction in the size of one non-irradiated metastasis by ≥30%,119–121 while others have considered out-of-field response using aggregate diameter of non-irradiated target lesions, in accordance with RECIST 1.1 criteria (‘best out-of-field (abscopal) response rate’).6 122 123 To the best of our knowledge, only few data comparing both definitions are published. In our cohort of 94 patients treated with immunotherapy and radiotherapy, 15 patients (16.0%) presented an out-of-field response according to RECIST 1.1 criteria, while nine patients had at least one non-irradiated responding lesion but the aggregated change in diameter of all the non-irradiated lesions did not reach the 30% threshold. Overall survival of patients with ‘RECIST-negative’ out-of-field response was not significantly different than survival of patients with no responding out-of-field lesion, while ‘RECIST-compliant’ out-of-field patients had significantly better overall survival (p=0.039).73 This result therefore encourages staying with the RECIST 1.1 criterion for out-of-field response assessment, which considers the evaluation of patient response on non-irradiated targets.124 To note, immunotherapy has also brought new patterns of response, such as pseudoprogression, dissociated response, and hyperprogression,125 126 which led to the introduction of additional response assessment criteria such as Immune related Response Criteria,127 immune RECIST,128 immune modified RECIST,129 and immune-related RECIST.130

Interestingly, the concern of evaluating treated and untreated lesions has also been raised with intratumoral immunotherapy injections. The itRECIST criteria proposed to assess injected and non-injected lesions response separately, with the possible definition of target and non-target lesions in both injected and non-injected lesions.131 Sum of diameters (SOD) is performed on target-injected lesions and target non-injected lesions separately, then a combined SOD includes all target lesions (injected and non-injected).131 The relevance of such strategies for radiotherapy and immunotherapy combination does not seem clear yet might be worth investigating.

Use of machine learning and functional imaging for early response assessment might also be useful to predict patient outcomes and discrimination of progression and pseudoprogression.81 82 117 132 Delta-radiomics, which considers the relative changes of radiomic features between two time points, might help to improve the assessment of treatment response. Dercle et al 81 have shown in a study analyzing 575 melanoma patients from the KEYNOTE-002 and KEYNOTE-006 studies that a radiomics signature taking into account changes in tumor imaging phenotype at month 3, performed better than RECIST 1.1 criteria for discriminating patients overall survival (AUC=0.92, 95% CI 0.89 to 0.95 vs 0.80, 95% CI 0.75 to 0.84).81

Radiomics analysis of follow-up images have also shown interesting results for the assessment of radiotherapy-immunotherapy toxicity.133 Cheng et al 113 have developed a radiomics signature to differentiate immune checkpoint inhibitor-related pneumonitis (CIP) and radiation pneumonitis (RP) on CT scan images. They trained their model using CT images of patients with pneumonitis while treated with ICI (n=28) or radiotherapy only (n=31) and tested their model in a cohort of 14 patients treated with ICI+radiotherapy. Performance of the best-trained model in the test set was promising with an AUC=0.896, although further prospective validation is needed.

Finally, another challenge will be to assess whether imaging biomarkers may help to monitor patients undergoing treatment. Indeed, the development of imaging biomarkers has particularly focused on the prediction of response using pretreatment scans, with or without the use of first evaluation images (ie, with delta-radiomics). However, no study has assessed whether these tools are still effective in predicting patient responses or variations in the tumor microenvironment during treatment over time. Development of automatic segmentation tools and deep learning will probably be key to automate such analyses.


While radioimmunotherapy combinations seem very promising for improving patients’ outcomes, novel performant biomarkers are needed to identify patients who are most likely to benefit from these treatments and to help clinicians choose the optimal modalities of both radiotherapy and immunotherapy. Imaging biomarkers have shown promising results to characterize the whole disease with its spatial heterogeneity, which eventually may guide different radioimmunotherapy strategies such as a synergistic immunogenic low-dose radiotherapy to improve systemic effects, or a complementary preemptive ablative high-dose radiotherapy to destroy selective lesions at high risk of progression under immunotherapy. Further efforts and randomized prospective studies are needed to validate such strategies, but imaging biomarkers will surely be key for development of precision medicine in immunotherapy and radioimmunotherapy combinations and monitoring of patients.

Ethics statements

Patient consent for publication



  • Twitter @rgrsun, @lescientifiik

  • Contributors Conception and drafting of the manuscript: RS and ED; critical revision of the manuscript: all authors. Final approval of manuscript: all authors.

  • Funding Fondation pour la Recherche Médicale (grant DIC20161236437), Fondation ARC pour la recherche contre le cancer (grant SIGN'IT20181007805), SIRIC-SOCRATE 2.0 (grant INCa-DGOS-INSERM_12551) and Amazon AWS grant provided funding to INSERM U1030 Radiomics team - Gustave Roussy. Fondation ARC pour la recherche contre le cancer and Paris-Saclay University provided support to RS with a grant for international mobility. RS received support from Fondation BETTENCOURT-SCHUELLER and Ecole INSERM.

  • Competing interests ED has declared consulting fees and support from Roche, BMS, Boehringer, Astrazeneca, Lilly Amgen, and Merck-Serono. The remaining authors have no conflicts of interest to declare that are relevant to the content of this article.

  • Provenance and peer review Not commissioned; externally peer reviewed.