Immunotherapy and assessment of the tumor microenvironment (TME)

The application of both dynamic and static imaging methods over the past two decades has revealed a fundamental aspect of immune function, namely, the key role for the spatial organization of innate and adaptive immune cells within a tissue in mediating effective host defense or producing pathology.1 It is thus reasonable to assume that these principles apply to the immune response to cancer, whether within lymphoid tissues in which adaptive immunity develops or within the tumor microenvironment (TME) in which inhibition of tumor growth or killing of malignant cells takes place. Consistent with this view, the application of simple immunohistochemistry led to the development of the ‘immunoscore’,2 a measure of T cell infiltration of the TME, which provided striking correlates with the survival of patients with colorectal cancer irrespective of treatment.3 This concept of spatial organization of the TME gained special cogency with the emergence of checkpoint inhibitors as immunotherapeutic tools against diverse types of cancers—it became apparent that the extension of the ‘immunoscore’ concept that tumors that were ‘immune deserts’ showed ‘immune exclusion’ or were immunologically ‘hot’ (T cell invaded) tumors had real relevance to immunotherapy treatment outcome, with most responses in the third group.4 The advances in immuno-oncology using such therapies have changed the landscape of cancer treatment and provided potential long-term control (or ‘cure’) to many deadly malignancies, especially metastatic disease.5 However, a significant proportion of patients do not respond to immunotherapy and many responders suffer from disease progression, leading immuno-oncology researchers to try to (1) identify biomarkers better able to predict the patients likely to show good responses to the current agents; and (2) discover new targets to overcome current treatment limitations.6 Multiplex immunoimaging has emerged as one important step in this direction and this review succinctly describes contemporary study design considerations and experimental approaches associated with such imaging. This overview is specifically curated for the benefit of oncologists and translational researchers seeking to use this methodology to further their understanding of the TME and how to manipulate the immune system for benefit.

Single cell versus tissue-level studies

Multiple measurement modalities have been applied to achieve these ends of better categorizing patients and developing new immune-therapeutic approaches. These range from non-invasive technologies such as CT, PET, and MRI, to minimally invasive methods involving profiling of peripheral blood, to invasive assessments based on tumor biopsies or resected material. Radiological measurements provide information on tumor mass but lack detailed cell-level information and indeed, in the early trials of anti-CTLA-4, such measurements led to an initial notion of tumor progression promoted by this agent because increases in radiological signal could not distinguish growing tumor from local inflammation.7 8 While assays involving peripheral blood, such as cell-free DNA and flow cytometry, can provide some information on tumor burden and overall immune status, these methods lack precision in assessing the regional immune response in lymph nodes or within the TME.

It is the direct sampling of the tumor using biopsy or resection material that provides the most direct assessment of tumor state and how the immune system is engaged with the malignancy. Such samples can be explored in two very distinct ways: one involving retention of tissue architecture and spatial organization as routinely done using microscopic examination in a surgical pathology laboratory, and the other via tissue dissociation for single-cell studies, employing flow cytometry or single-cell RNA sequencing. Although the latter methods using dissociated cells have high throughput and multiplexity, they sacrifice information that can reveal critical cell-to-cell interactions or the local distribution of both antitumor and immune suppressive cell populations within the TME, information that can be vital to understanding why immunotherapy succeeds or fails in a patient.9

Spatial analysis via microscopy

Spatial analysis of malignant tissue using microscopy is of course not new, but rather a well-established and widely used technology that plays a crucial role in cancer diagnosis, staging, and in some cases, treatment assessment. H&E staining with examination by wide field microscopy is the most common method. It provides morphological descriptions that can be used to identify and quantify major cell types, such as tumor cells, lymphocytes, and macrophages, information that contributes significantly to characterizing the TME and enabling studies that relate such features to treatment outcome, as noted above with respect to the ‘immunoscore’. However, H&E staining is insufficient to distinguish the increasingly complex array of immune cell subsets revealed by flow cytometry and single cell RNA sequencing. Such categorization requires incorporation of multiple markers to identify each cell subpopulation.10 For example, regulatory T cells are almost indistinguishable from other T cell subsets examined by H&E staining but can easily be identified based on positive staining for CD3, CD4, and Foxp3 proteins. Therefore, immunohistochemical (IHC) and immunofluorescence (IF) staining methods were introduced to detect proteins via antibody-driven recognition. IHC is widely used in clinical practice because the deposited dye can be detected using a regular light microscope. However, IHC is largely limited to two to three markers because of limitations in visualizing color differences among the enzymatic products. This can be overcome using elaborate bleaching and re-probing methods,11 12 but this method also suffers from limited linearity of signal due to the use of enzymatic amplification. So, while it is very sensitive, IHC masks differences in the expression of molecules whose quantitative levels are likely to be important in biological processes, for example, the extent of expression of PD-L1 (Programmed Death-Ligand 1)or PD-1 (Programmed cell death protein 1). For this reason, most new approaches to multiplex imaging of the TME rely on one of two other methods—either IF or mass imaging, methods that do not rely on dye deposition.

Two-dimensional versus three-dimensional

While adding dynamic information to what can be obtained in multiplex static imaging could be valuable for probing the interplay between immune cells and tumors and tumor stroma, such live imaging methods are not easily applicable to humans and even ex vivo studies with isolated tissues have serious limitations. Hence, the major emphasis is on multiplex static imaging, which can be two-dimensional (2D) or three-dimensional (3D)13 and that provides a snapshot of a tumor sample at a given time point. The definition of 3D imaging may vary depending on the thickness (depth at the Z-axis) of the tissue being imaged. In general, thin tissue sections (5–10 micron thickness) usually contain a single layer of cells, allowing light to pass through without special treatment. Although there are overlapping cells at the Z axis within cell-dense regions on thin tissue sections, images obtained from such samples are commonly processed as 2D images (single Z section) to reduce data size and assure reasonable downstream processing time. In certain circumstances, 3D imaging of thin sections can further assist the assessment of cell-cell contacts.14 Otherwise, 3D imaging usually refers to obtaining imaging data in tissue more than 10 micron thick, with multiple layers of cells along the Z axis. This requires optical clearing to transform intact tissue into transparent samples so light can pass through. Multiple clearing methods have been developed, such as clearing-enhanced 3D (Ce3D),15 Clear Unobstructed Brain Imaging Cocktails and computational analysis,16 iDISCO(immunolabeling-enabled three-dimensional imaging of solvent-cleared organs),17 clear lipid-exchanged anatomically rigid imaging/immunostaining-compatible tissue hydrogel,18 system-wide control of interaction time and kinetics of chemicals,19 magnified analysis of proteome (MAP),20 stabilization under harsh conditions via intramolecular epoxide linkages to prevent degradation,21 and protein retention expansion microscopy.22 These methods preserve the original tissue structure and epitopes, and allow antibody penetration to varying degrees. The detailed comparison of tissue clearing methods is available.23 Multiplexed 3D images can identify rare cells24 or complex tissue structures (blood vessels,25 lymphatic ducts,26 and nerves27) in the tissue. However, the application of 3D multiplex imaging is significantly limited due to its prolonged antibody staining, clearing, and imaging time compared with 2D multiplex imaging and the need for advanced microscopes. Currently, most multiplex imaging studies in immuno-oncology research employ static 2D multiplex imaging methods, which is what we will focus in the remainder of this review.

Considerations in multiplex 2D analysis

An ideal method for multiplex analysis of the TME would enable detection all proteins, RNAs, and metabolites at a subcellular resolution within a reasonable imaging acquisition time. Unfortunately, this requirement is far beyond current methods. Even co-detection of protein and RNA in the same tissue section has been challenging because RNA detection requires protein digestion to expose RNAs for complementary strand binding, a process that can destroy protein epitopes to which antibodies bind. Although careful adjustment of the digestion and staining conditions can enable measurement of both types of molecules at once,28 most current multiplex imaging methods only permit investigation of either protein or RNA in a single tissue section, with investigators using adjacent sections to infer correspondence between the two read-outs.

Antibody-based multiplex imaging depends on the availability of specific antibodies to selected target molecules. Compared with flow cytometry analysis, the use of antibodies for tissue analysis is more challenging due to non-specific binding in the more complex tissue microenvironment, loss of epitopes due to fixation, and a lower signal to noise ratio since only a fraction of the molecules of a given type expressed by a cell are captured in a thin section passing through a plane of the cell volume, as compared with the capture of total cellular fluorescence in a flow cytometer.29

A particular issue in multiplex imaging of patient material has to do with the way the tissue samples are preserved. This can involve either snap freezing (fresh frozen material) or fixation to avoid protein degradation and tissue deformation. Frozen samples are best for preserving antigenic epitopes for staining30 but require immediate processing and low-temperature storage, something not always easy to arrange in the clinic. More commonly, the tissue is placed in formalin fixative soon after harvest and embedded in paraffin (formalin-fixed paraffin-embedded, FFPE). The extensive aldehyde cross-linking caused by formalin fixation hides many antibody epitopes and tissue sections from FFPE samples typically require harsh chemical and heat treatment to reverse these cross-links and reveal the antibody epitopes (antigen retrieval). In research settings, fixation methods can be optimized to allow adequate fixation with minimal artifacts or epitope loss, typically by employing paraformaldehyde at less than 4%.31 For samples that are only available in FFPE format, antigen retrieval poses a significant limitation. Different antigens may require distinct retrieval conditions (pH, temperature, or pressure), which can be problematic when trying to undertake highly multiplex imaging as not all epitopes can be re-exposed at the same time. In addition, the fixation process usually increases tissue autofluorescence, a major challenge in image acquisition and analysis using IF methods.32

After antigen retrieval, all multiplex imaging methods detect target antigens by using antibodies (most often monoclonal, though polyclonal reagents are sometimes employed). Two major schemas are used to reach high plexity. Mass spectrometry (MS)-based methods, such as multiplex ion beam imaging (MIBI)33 and imaging mass cytometry (IMC),34 use MS to detect metal-conjugated primary antibodies. All antibodies can be applied to the tissue at once and detected in a single ‘imaging’ cycle by the MS. This enables examination of more than 40 biomarkers in a tissue section without iterative staining and imaging cycles. This approach also has less background because there are few contaminating ions matching the metal chelate reporters attached to the antibodies, as opposed to autofluorescence that can confound IF methods. However, these methods require special conjugated antibodies and instruments that are more expensive, not as commonly available, and not as stable as the light or confocal microscopes used for IF imaging. Furthermore, MS-based methods typically have lower resolution (~1 μm vs <300 nm for IF) and have a lower imaging acquisition speed (200 pixels/s or 1 mm²/2 hours).9 35

Photon-based methods visualize antigens recognized by antibodies via chromogenic immunohistochemistry (antibodies are conjugated with an enzyme that catalyzes a color-producing chemical reaction—IHC) or fluorophore-conjugated with antibodies (IF). Signals are usually amplified by enzymatic reaction (horseradish peroxidase11) or tyramide signal amplification (TSA).36 As noted above, IHC dye-deposition methods have limitations in color number per cycle, non-linearity of signal strength, and often substantial background from enzymes in macrophages abundant in the TME. IF methods can make use of a very large set of fluorochromes that emit across the color spectrum. Unfortunately, all organic dyes used for antibody conjugation have rather broad emission spectra and overlap between the fluorophores thus limits the number of that can be imaged and distinguished at one time. Using modern methods of spectral deconvolution and other methods that combine narrow band spectral capture with computational techniques, one can reach more than a dozen markers in one cycle, but on a practical basis, a typical IF multiplex imaging cycle will consist of 4–8 makers/fluorochromes, with linear unmixing employed to separate signals where emission overlaps still occur.37

Given this limitation of markers in a single cycle, highly multiplex IF imaging is typically based on iterative cycles of staining, imaging, and signal removal achieved by eluting antibodies, dissociating antibody bound-reporter oligos, or using light or chemicals to inactivate fluorophores. Iterative multiplex imaging methods using such approaches, such as iterative bleaching extends multiplexity (IBEX),31 tissue-based cyclic immunofluorescence (t-CyCIF),38 multiepitope-ligand cartography,39 and multiplexed immunofluorescence (MxIF, Cell DIVE, PhenoCycler (formerly CODEX))40 41 can detect more than 60 markers in a single tissue section (table 1). A major concern with such cycling methods is tissue damage/loss caused by the harsh chemical or physical events involved in these repetitive steps. This significantly limits the number of staining cycles that can be done with confidence without loss of information, an issue of critical importance with precious clinical material. A second issue is image alignment, as there are again physical and chemical effects on the shape of the tissue during these treatments; however, computational alignment methods have been developed that largely overcome this problem for thin sections properly adhered to the imaging substrate.42–44

The other class of multiplex spatial analytical methods involve RNA rather than protein epitope detection.28 45 RNA detection methods include both sequence-based and hybridization-based technologies, with MERFISH and RNAscope being just two examples of many evolving approaches, respectively. One issue with RNA method is determining cell boundaries—when the probes detect transcripts in widely spaced regions of the tissue and include prototypic transcripts of known cell types, the combination of mRNAs detected in a small area can be reasonably interpreted as being co-expressed in a single cell, but in densely packed tissue settings, such interpretations are fraught and limit the utility of this approach in the absence of concurrent staining-based imaging data. Method using hybridization probes is also subject to artifact due to undesired binding of probes to matching sequences in DNA or via charge to matrix or other elements in the TME.

Several spatial transcriptomic platforms have been commercialized and are accessible to investigators without expertise in microfluidics. In general, spatial transcriptomic platforms can be divided into imaging-based and NGS(Next-generation sequencing)-sequencing-based platforms.45 The imaging-based systems have a much higher spatial resolution (higher than 1 micron) and can be further categorized by how transcripts are detected: in situ sequencing versus in situ hybridization. In situ hybridization systems depend on probes binding to preselected sequences,46 thus limiting the experiments to selected transcripts. On the other hand, NGS-sequencing-based platforms capture mRNA from tissues via bar-coded microbeads, and the mRNA data are processed via library preparation followed by NGS sequencing. However, the spatial resolution of RNA capture and the diffusion of RNA molecules from the source cell during the capture process impose limits on this technology. Therefore, current single-cell resolution spatial transcriptomic systems are mainly based on imaging-based platforms.

Proper design of multiplex TME imaging studies

Study design is critical for multiplex imaging studies (figure 1). Imaging methods, including antibody-based protein detection and most spatial transcriptomics approaches, are limited by how many targets can be selected for analysis and by the acquisition time, which increases with target plexity. Thus, in comparison to single-cell RNA sequencing methods, imaging remains a relatively low-throughput approach. In addition, for antibody methods, only targets to which such reagents have been prepared and validated can be studied, making it essential that the investigators use prior knowledge to carefully chose a relevant and informative set of probe targets for study. This unfortunately limits the likelihood of completely novel discoveries at the molecular level but does not prevent gaining critical new insights into biology in a complex tissue setting.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

Design for multiplex tumor microenvironment (TME) imaging studies. Essential steps in designing multiplex TME imaging studies are illustrated above. Major considerations for each step are listed in black below text boxes.

After selecting targets, the next step is tissue preparation. As discussed above, the fixation process affects autofluorescence, the preservation or loss of epitopes, and the quality of the tissue preparation, so optimizing fixation conditions is essential before proceeding with the study of precious patient material. Another vital question is selecting the physical size of sections to be imaged. Although larger sections provide more information and thus limit misinterpretations due to under sampling in a heterogeneous tumor environment, needle biopsy samples are usually more accessible and allow placing samples from the same patients or treatment conditions on the same slide to minimize batch effects that can have major impacts on complex multiplex imaging studies. Multiple existing statistical methods are able to estimate the number of subjects needed for a defined study.47 However, it is challenging to identify regions of interest (ROIs) or fields of view that capture all essential morphological information in a whole tissue section, no less throughout a large tumor sample. Traditionally, this process is assisted by pathologists whose experience is employed to identify presentative regions. Recently, artificial intelligence algorithms have been developed to address this question in whole slide image analysis process.48 49 Meanwhile, tissue microarrays (TMAs) are widely used in the research setting for easy access and relatively low cost based on the sample numbers, but the way in which these TMAs are created and the extent to which there can be biased sampling by the pathologist selecting regions from a larger tissue sample need to be considered when taken this route.

The next step is to choose a staining panel with appropriate fluorophore combinations. Although sharing the same principle as high-plex flow cytometry analysis, multiplex imaging has unique challenges because of tissue autofluorescence, variations in antibody penetration into thicker tissues sections, and reduced signal to noise ratio as explained earlier. Fixation effects on epitope availability, variations in which epitopes can be simultaneously recovered using antigen retrieval methods, and spurious binding of antibodies to structural components in the TME all confound multiplex imaging studies, the more so as the number of targets analyzed increases. Finding the correct antibody clone for a given target molecule can be highly challenging. It is important to use prior literature3 and expert knowledge as a reference and perform vigorous validation.29 50 Best practices in this regard are emerging from consortia such as the HuBMAP.51 Besides finding the correct clone, it is also essential to identify appropriate fluorophores. This selection depends on the anticipated density of the target molecule to be detected, with better emitters used for less abundant targets, as well as on the configuration of the microscope to be used with respect to excitation laser wavelengths available and the detector properties. In general, the commonly used fluorophores have an emission ranging from 420 to 800 nm. An ideal fluorophore should be bright (high quantum yield) and have minimal overlap with other fluorophores (narrow emission spectrum). Separation of signals is aided when using microscope-based methods by morphological features. For example, transcription factor staining is typically in the nucleus and can be resolved from surface membrane staining. Therefore, it is a common practice to make sure fluorochrome-antibody combinations with neighboring or overlapping emission spectra react with different cell types and cellular structures to minimize artifacts caused by channel bleed through. However, it is still important to perform linear channel unmixing to get the highest resolution among probes, which requires single color-stained tissue samples as references for creating the unmixing matrix. In addition, arranging the sequence of iterative imaging should be based on the expression level and the signal-to-noise ratios of each target. Targets with low expression levels and low signal-to-noise ratios should be prioritized.

Besides designing an antibody panel, another consideration is selecting a data recording format to balance data size and image content. Commonly used data formats are 8, 16, and 32-bit. High bit depth gives more details (wider signal dynamic range) but requires greater storage space. Also, it is important to avoid signal saturation on target cells and structures when deciding laser strength, exposure time, and signal gain on the imaging platforms. Information will be lost once signals reach the upper limit of the detection range.

Data analysis

Computational data analysis is critical for making use of multiplex imaging information and usually takes a significant amount of time, often exceeding that involved in image acquisition itself.

There are multiple steps involved in extracting cell-associated data from raw images (figure 2). A typical raw image consists of many small segments imaged at high resolution and stitched together.52 Processing before analysis involves selecting an ROI, removing low-quality areas from that selected region, then applying stitching correction, shading correction, chromatic aberration correction, and applying channel unmixing.53–55 Deconvolution can be used to improve image quality, but a key measurement needed to do this properly, the point spread function, is not routinely available and the calculations can be slow and require high-end computing systems.56 Chromatic aberration correction is mainly applied when very high-resolution images are needed.53 Shading correction is primarily used for camera-based image-acquisition methods.54

• Download figure
• Open in new tab
• Download powerpoint

Figure 2

Imaging analysis workflow. Each box is a step in imaging analysis workflow. Red boxes represent critical steps shared by all imaging analysis flows. Green boxes are optional steps enhancing image quality. The step in the blue box can be omitted in whole section image analysis. ROI, region of interest

After generating high-quality images, the next step is identifying choosing between processing at the cell level or pixel-level analysis. Most current methods for spatial analysis of tissue sections use cellular objects,57 but the value of pixel-level methods is apparent in that cell identification algorithms cannot handle acellular stroma and other features of the TME and, as noted below, have substantial limitations in accuracy. Software for conducting pixel-level interrogation of images is now emerging in the literature.58–60

For conducting processing and spatial analysis at the cell level, a process called ‘segmentation’ is employed.61 In the computer vision field, segmentation can be divided into semantic and instance segmentation. In general, semantic segmentation will help identify different types of subjects, such as tumors from blood vessels but would not be able to locate each cell individually. In comparison, instance segmentation aims to identify each subject separately, which is more commonly used in multiplex image analysis (we use segmentation refers to instance segmentation below). Because cell shape varies dramatically and cell membrane staining can be very variable depending on the density of the selected target molecules and the reagents used, segmentation based on nucleus identification and expansion radially out from that region has been the most common method for cell identification. This approach takes advantage of the relatively round shape of the nucleus in most cells. It can capture protein expression information via expansion when the distance between the nucleus and cell membrane is constant.62 However, the actual nuclear shape as well as position within the cell, the distance from the bounding membrane, and the nucleus-to-cytoplasm ratio all vary among cells, making it technically challenging to determine an accurate way to perform the nuclear expansion for all cells. Recent advances in computer vision now approach segmentation by combining nuclear and membrane staining, then using deep learning methods such as Cellpose,63 Mesmer,64 and RAPID60 to more accurately segment the diverse cell types within a tissue cell. However, pretrained models are limited by their training datasets and their performance can be further enhanced after adaptation to the target dataset.65

After segmenting cells, target molecule information on each pixel (voxel in 3D) will be assigned to segmented cells. Unfortunately, because of the resolution limit even of the best current confocal microscopes, there can be pixel overlap at the membrane boundaries of neighboring cells. This is more obvious when analyzing tissues composed of densely packed cells like lymph nodes and spleens, but such cell-cell contact is also common in the TME. The ‘spill-over’ of signal can result in classifying cells inaccurately based on markers they do not actually express but are assigned to them during segmentation due to expression on neighbors. Recently, a variety of computational methods have been introduced to address this issue, but to date, these are only approximate solutions to the problem.60 66 67 This means that careful curation of the segmentation data is needed at an expert level to filter out cell phenotypes that are artifacts of such pixel overlap.

After segmentation, cells can be analyzed based on their markers as well as morphological parameters, such as area, eccentricity, etc.68 69 The general analysis approach is similar to methods used in flow cytometry data analysis by selecting a series of positive and negative markers to identify target cell populations (‘gating’).37 In addition, unbiased clustering methods, such as UMAP(Uniform Manifold Approximation and Projection) or t-SNE(t-distributed stochastic neighbor embedding), have been successfully implemented in image data analysis. In combination, these methods can generate the frequency and absolute number of different cell types in target tissues. Cell density can be calculated based on tissue area and/or volume information. Information on cell frequency and density is commonly used to test hypotheses, but it is evident that additional information on functional cell state is critical to gaining deeper insight into how the immune system and tumor interact and the effects of immunotherapies on those interactions. This is where highly multiplex methods provide an ability to go beyond counting and through proper choice of staining targets, address issues such as cytokine production, signaling status, apoptosis, and other bioactivities key to drawing inferences about the state of immunity within a tumor.

Beyond cell densities and individual cell states, cell-cell interactions, such as tumor-immune interactions and T cell-myeloid cell interactions, are crucial for understanding antitumor immunity and perturbation resulting from various treatments and computational tools for identifying statistically robust non-random arrangements of cells within the TME are needed for this purpose. Cell-cell interactions happen at different speeds, frequencies, and for different durations.70 Intravital imaging has provided basic information on interactions among immune cells or between immune cells and tumors under specific experimental conditions.67 71 72 As discussed above, intravital imaging is still mainly limited to animal models. Therefore, investigators depend on statistical analysis to search for meaningful cell arrangements and interactions in the microscopic datasets.73 Commonly addressed questions are (1) spatial correlations among selected cells and (2) neighborhood relationships (patterns). Fortunately, spatial statistics is well established in geographical science, with multiple tools developed as part of the geographical information systems,74 and progress in digitalized microscopy has led to the use of such existing tools to explore biological questions. However, spatial statistics questions in biology have unique properties compared with those in geographical studies: cells and tissue structure are not randomly distributed, so the null hypothesis should not be based on randomization. For example, most of the immune cells in the lungs are restricted to the narrow space in lung parenchymal due to the unique structure of the lung. Directly comparing the distribution of immune cell populations against a random distribution will generate false positive results, suggesting non-existent correlations. Multiple static methods, such as Monte Carlo techniques,75 76 have been successfully used to address this problem. The detailed statistical consideration can be found in prior reviews.60 73 In addition, multiple open-source packages and methods are or are becoming available to facilitate this process, such as SPACE,60 Giotto,76 ImaCytE,77 Squidpy,78 SpatialDE,79 histoCAT,80 and SPARK.81

Applications of multiplex imaging to studies of cancer

These various multiplex imaging platforms and statistical methods are providing powerful tools for investigation of the TME in human malignancies. The data have been used to discover new subsets of cellular phenotypes in tumors considered to be of the same type, to gain insight into the mechanisms of either antitumor response or immune evasion, or to search for robust patterns that predict response to immunotherapy.69 82 To better understand which patients respond to checkpoint therapy, Philips et al conducted a multiplex imaging using the PhenoCycler platform with a panel of 56 markers. They analyzed 70 tumor regions from 14 patients with advanced cutaneous T cell lymphoma who received pembrolizumab.83 After examining 117,170 cells, they found no differences in the frequency of major immune or tumor cells between responders and non-responders. However, there were topographic differences among effector PD1⁺CD4⁺ T cells, tumor cells and T regs among the patient samples. They used the distance of CD4⁺ T cells to Treg and tumor cells measured separately to assess what they called T cell effector activity and suppressive activity, respectively, creating a ‘SpatialScore’. In general, PD1⁺CD4⁺ T cells in responders had a lower SpatialScore (higher effector function) as compared with non-responders, both at baseline and after treatment. To test in a larger cohort, SpatialScore was implemented with multiplex IHC for higher throughput.

Besides searching for markers and spatial patterns identifying responders and non-responders to immunotherapy, multiplex imaging has also provided insight into tumor-immune interactions and tumor evolution. Nirmal et al used CyCIF for 30 markers along with spatial statistic measurements like latent Dirichlet allocation modeling to study immune evasion and immunoediting during disease progression in 13 patients with primary melanoma.14 Recurrent cellular neighborhoods consisting of tumor cells, immune cells, and stroma cells evolved from melanoma in situ to invasive tumor. Signatures of immunosuppression, including PD-L1-expressing myeloid cells and Tregs, were found even at the precursor stage. Tumors further became consolidated and spatially restricted with a highly suppressive environment along the tumor-stromal boundary involving expression of MHC-II and IDO1 (indoleamine 2,3-dioxygenase 1), together with PD1-PD-L1-mediated cell interactions involving macrophages, dendritic cells, and T cells. Another major study employed multimodal analysis that combined multiplex imaging and transcriptomic experiments to achieve a more holistic view of tumor-immune interactions,84 which used MIBI, spatial transcriptomics, and single-cell RNA sequencing with a collection of cutaneous squamous cell carcinoma and matched normal skin samples.84 Tumor-specific keratinocytes acted as a hub for intercellular communication and immunosuppressive features such as Treg colocalization with CD8 T cells in compartmentalized tumor stroma. Moving beyond tumor and immune cells, Risom et al successfully addressed the technical challenges in segmenting stromal cells in the TME via pixel-based analysis methods. Using multiplexed ion beam imaging by time of flight (MIBI-TOF) and a 37-plex antibody staining panel, they were able to interrogate the TME of 79 clinically annotated surgical resections from patients with ductal carcinoma in situ (DCIS). Machine learning tools for cell segmentation, pixel-based clustering, and object morphometrics revealed myoepithelial disruption in patient with DCIS whose diseases did not progress to invasive breast cancer, suggesting a potential protective role of this process.85

These studies and others now appearing in the literature reveal the power of multiplex imaging for developing new insights into tumor-immune interactions, tumor evolution, and patient stratification with respect to outcome following immunotherapy. Despite the technical challenges mentioned above, advances in the hardware, software, and reagents needed to conduct such high-plex imaging work are now permitting such analyses, initially conducted by only a few dedicated research laboratories, to begin to move into clinical practice. We can expect a constant increase in the application of these methods to interrogation of more samples and tumor types, leading to new insights relevant to clinical practice using current therapies as well as with novel information that can aid development of new agents. The power of this approach and the advances already being made suggest it is not unreasonable to rephase the popular ‘Seeing is believing’ as ‘Seeing is understanding’.