Research paperEvaluation of Elispot assays: influence of method and operator on variability of results
Introduction
The Elispot technique has gained much attention since its first description in 1983 (Czerkinsky et al., 1983). The number of publications using the Elispot technique has quintupled just over the last 4 years. Due to its sensitivity that allows the identification of antigen-specific T cells in frequencies as low as 1/106 peripheral blood mononuclear cells (PBMC) without elaborate manipulation or expansion of the cells, the Elispot assay has become a standard method for monitoring cellular immune responses (Clay et al., 2001). The number of clinical trials employing the Elispot assay has increased over the past 5 years (Wang et al., 1999, Hoerig et al., 2000, Janetzki et al., 2000a, Lewis et al., 2000, Meidenbauer et al., 2000, Rahman et al., 2000, Moss et al., 2000, MacGregor et al., 2002, Schaed et al., 2002, Peterson et al., 2003). New strategies for antigen-presentation have been developed to optimize the assay (Janetzki et al., 2000b, Schmittel et al., 2001, Britten et al., 2002, Atanackovic et al., 2003, Thumann et al., 2003).
Standardization and validation procedures allow accurate and reliable Elispot performance with reproducible results (Smith et al., 2001, Mwau et al., 2002, Lathey, 2003, Janetzki et al., in press).
However, a comparison of methods to evaluate Elispot assays has been given little attention. More than half of the relevant publications in 2002 describe their evaluation effort as counting spots with a stereomicroscope or by eye. The average size of a spot is 30–150 μm, although this varies depending on microtiter plates, antibody source and concentration, enzyme activity, substrates, and other materials used (Janetzki, unpublished data), as well as on the functional state of the cytokine-secreting cells (Lewis et al., 1991, Karulin et al., 2000, Hesse et al., 2001). Therefore, the counting-by-eye approach should not be an acceptable evaluation method. The use of a stereomicroscope is time- and labor-intensive. High spot numbers are difficult or impossible to enumerate, and in wells with high spot numbers, typically only a part of the well is counted and the total amount of spots is estimated. Furthermore, due to limits of ocular resolution, it is difficult to recognize and exclude artifacts or to count small and faint spots or spots located close together. Finally, one of the most important setbacks is the bias of the reader. The investigator evaluating the plate has to essentially reset parameters for spot evaluation at each well. A bias toward “loosening” of these parameters for expected positive response wells and “tightening” the same for negative control wells, whether consciously or subconsciously, is hard to control.
The introduction of automated Elispot readers in 1997 (Cui and Chang, 1997, Herr et al., 1997, Vaquerano et al., 1998) presented some solutions to these drawbacks. The pros and cons of the different evaluation methods have been discussed and limited comparisons between spot counts have been published (Asai et al., 2000, Smith et al., 2001). However, reading methods have never been directly compared in a more comprehensive manner, and the operator-dependent influence on quantitative output by automated reader systems has not been analyzed. This is especially important in the context of recent validation and standardization approaches (Lathey, 2003, Janetzki et al., in press) and comparative studies of Elispot assay performance among different laboratories (Scheibenbogen et al., 2000, Cox et al., in press). In this report, the results of manual and automated evaluation methods of Elispot plates are compared, including automated evaluation with operator-dependent and operator-independent evaluation parameters. We compare the variability and reproducibility of the different methods, including the operator-dependent limitations of each evaluation method. We finish by presenting recommendations for Elispot validation and standardization procedures.
Section snippets
Elispot assay
Human interferon-gamma (IFNγ) Elispot assays were performed as described elsewhere (Lewis et al., 2000). Human CD8+ T cells were tested for reactivity against various HLA-A2-restricted peptides using T2 cells as antigen-presenting cells. One hundred thousand CD8+ T cells/well were plated into nitrocellulose microtiter plates (Millipore, Bedford, MA) coated with 1 μg/well anti-IFNγ (Mabtech, Sweden). T2 cells were pulsed with 10 μg/ml peptide, irradiated at 10,000 rad, and added to the effector
Reproducibility of Elispot reading methods
To examine the reproducibility of all three methods, an analysis of variance was performed with the number of spots as the dependent variable and the specific test well as the independent variable (Table 1). Data for the three methods were analyzed separately. The variability introduced by each investigator was accounted for. Four wells each had spot counts >200 with one well having an average of >1000 spots. These values were incorporated in the overall analysis of variance (“all”). Not
Discussion
In this report, different evaluation procedures of Elispot plates are compared. For this study, three investigators had to enumerate the spot number in a total of 50 randomly selected wells with average spot counts from 0 to >1000 spots by manual evaluation using a stereomicroscope, by automated evaluation using investigator-determined parameters, and by preset reading parameters. Not unexpectedly, the manual evaluation method exhibited high variability, which increased with higher spot counts
References (32)
- et al.
Monitoring CD4+ T cell responses against viral and tumor antigens using T cells as novel target APC
J. Immunol. Methods
(2003) - et al.
The use of HLA-A*0201-transfected K562 as standard antigen-presenting cells for CD8+ T lymphocytes in IFN-gamma Elispot assays
J. Immunol. Methods
(2002) - et al.
A panel of MHC class I restricted viral peptides for use as a quality control for vaccine trial Elispot assays
J. Immunol. Methods
(2002) - et al.
A solid-phase enzyme-linked immunospot (ELISPOT) assay for enumeration of specific antibody-secreting cells
J. Immunol. Methods
(1983) - et al.
Reverse ELISPOT assay for clonal analysis of cytokine production: I. Enumeration of gamma-interferon-secreting cells
J. Immunol. Methods
(1988) - et al.
The use of computer-assisted video image analysis for the quantification of CD8+ T lymphocytes producing tumor necrosis factor alpha in response to peptide antigens
J. Immunol. Methods
(1997) - et al.
Insect cells as HLA-restricted antigen-presenting cells for the IFN-gamma Elispot assay
J. Immunol. Methods
(2000) - et al.
Quantification of antigen-reactive T cells in peripheral blood by IFN-gamma-ELISPOT assay and chromium-release assay: a four-centre comparative trial
J. Immunol. Methods
(2000) - et al.
Application of the IFN-gamma ELISPOT assay to quantify T cell responses against proteins
J. Immunol. Methods
(2001) - et al.
Antigen loading of dendritic cells with whole tumor cell preparations
J. Immunol. Methods
(2003)
Evaluation of the modified ELISPOT assay for gamma interferon production in cancer patients receiving antitumor vaccines
Clin. Diag. Lab. Immunol.
Assays for monitoring cellular immune responses to active immunotherapy of cancer
Clin. Cancer Res.
Results of an ELISPOT Proficiency Panel conducted in 11 laboratories participating in international human immunodeficiency virus Type 1 vaccine trials
AIDS Res. & Hu Retroviruses
Computer-assisted, quantitative cytokine enzyme-linked immunospot analysis of human immune effector cell function
Biotechniques
A T cell clone's avidity is a function of its activation state
J. Immunol.
PhaseI clinical trial of a recombinant canarypoxvirus (ALVAC) vaccine expressing human carcinoembryonic antigen and the B7.1 costimulatory molecule
Cancer Immunol. Immunother.
Cited by (41)
Distinct Immune Reconstitution Profiles Captured by Immune Functional Assays at 6 Months Post Allogeneic Hematopoietic Stem Cell Transplantation
2023, Transplantation and Cellular TherapyDefining ELISpot cut-offs from unreplicated test and control wells
2013, Journal of Immunological MethodsCitation Excerpt :Since it was first described in 1983, the enzyme-linked immunospot (ELISpot) assay has become a widely used method for the detection of antigen-specific cytokine-secreting T cells (Czerkinsky et al., 1983; Versteegen et al., 1988), and is now a standard assay for measuring the cell-mediated immune response to vaccines in clinical trials. The requirement for immunological assays used in vaccine trials to be rigorously validated has resulted in much work to maximize the sensitivity and specificity of ELISpot assays, ensure their reproducibility, minimize inter-laboratory and inter-operator variability and to automate and standardize the counting of the spot forming units (SFU) (Vaquerano et al., 1998; Schmittel et al., 2000; Mwau et al., 2002; Janetzki et al., 2004, 2005, 2008; Cox et al., 2005; Lehmann, 2005; Samri et al., 2006; Maecker et al., 2008). However, criteria for defining a positive response have been subject to considerable debate and controversy (Mwau et al., 2002; Hudgens et al., 2004; Jamieson et al., 2006; Jeffries et al., 2006; Moodie et al., 2010; Slota et al., 2011).
Systems analysis of adaptive immunity by utilization of high-throughput technologies
2011, Current Opinion in BiotechnologyCitation Excerpt :Finally, the anatomical compartments (e.g., lymph nodes, blood, bone marrow) in which lymphocyte activation, homing, and trafficking take place add to the complex interconnected nature of adaptive immunity. Conventional approaches for measuring adaptive immune responses have so far relied on cursory biological measurements, such as antibody titers to an antigen or lymphocyte assays that measure a single (ELISPOT) or multiple cytokines (intracellular staining, ICS) expressed by individual cells [2,3]. However, these techniques are surrogate measures and provide only a partial picture of the state of the immune system; for example, antibody titers do not provide specific molecular information on the monoclonal antibody components that comprise polyclonal responses in serum or secretory fluids.
Methods for detection of antigen-specific T cells by enzyme-linked immunospot assay (Elispot)
2022, Manual of Molecular and Clinical Laboratory Immunology: 7th editionRecommendations on ELISpot assay validation by the GCC
2022, BioanalysisImproving rigor and reproducibility in nonhuman primate research
2021, American Journal of Primatology