Spontaneous antitumor T cell responses in cancer patients are strongly controlled by Tregs, and increased numbers of tumor-infiltrating Tregs correlate with reduced survival. However, the tumor antigens recognized by Tregs in cancer patients and the impact of these cells on tumor-specific T cell responses have not been systematically characterized. Here we used a broad panel of long synthetic peptides of defined tumor antigens and normal tissue antigens to exploit a newly developed method to identify and compare ex vivo the antigen specificities of Tregs with those of effector/memory T cells in peripheral blood of colorectal cancer patients and healthy subjects. Tregs in tumor patients were highly specific for a distinct set of only a few tumor antigens, suggesting that Tregs exert T cell suppression in an antigen-selective manner. Tumor-specific effector T cells were detectable in the majority of colorectal cancer patients but not in healthy individuals. We detected differences in the repertoires of antigens recognized by Tregs and effector/memory T cells in the majority of colorectal cancer patients. In addition, only effector/memory T cell responses against antigens recognized by Tregs strongly increased after Treg depletion. The selection of antigens according to preexisting T cell responses may improve the efficacy of future immunotherapies for cancer and autoimmune disease.