Background Multiplexed immunofluorescence imaging is a powerful spatial biology tool that can produce rich marker expression data at single-cell resolution and whole-slide scales. High parameter images have been particularly useful for characterizing the composition and arrangement of cellular microenvironments and cell interactions in native tissue contexts. These spatial features have been shown to have predictive or prognostic value in clinical tasks, including risk-based cohort stratification, biomarker discovery, therapy response, and survival prediction across different cancers.^1–4 Analyzing multiplexed immunofluorescence images can be fraught with both technical challenges, including tissue quality, stain quality, and artifact removal; and computational challenges, such as the processing, curation, and analysis of extremely large raw image files. Excessive time spent manually annotating images, checking image and stain quality, aligning multiple images for multimodal analyses, and creating regions of interest to include or exclude in downstream analysis severely limits the throughput and utility of this technology. To address these challenges, we have developed and tested a suite of deep learning models and image processing utilities in a flexible, performant, and easy-to-use Python toolkit. These tools allow analysts to prepare multiplex immunofluorescence images for accurate analysis at scale in an automated fashion.

Methods Here we present a Python toolkit which includes six functions for processing multiplex immunofluorescence images: a function to calculate image quality based on the NIQE algorithm,⁵ a deep learning model⁶ to check the stain quality of six immune markers (CD3e, CD4, CD8, CD45, CD45RO, CD68), automated alignment of different assays using a SIFT method,⁷ deep learning⁸ based tissue detection, deep learning based artifact detection, and a thresholding function for creating masks based on biomarker expression.

Results The use of this toolkit reduces turnaround time and analysis time by decreasing manual intervention (visual quality control checks, hand-drawn annotations, manual alignment) and increases the accuracy of results by accurately detecting artifacts to exclude (detects non biological artifacts with 97% accuracy, 95% Jaccard similarity).

Conclusions The analysis of multiplexed immunofluorescence images is key to the advancement of the spatial biology field. Automating the preprocessing steps required to accurately analyze these images is necessary to be able to keep up with the amount of data analysis needed to make these advancements. These data-driven tools should allow biologists and clinicians to focus on research questions rather than technical hurdles. We further expect these models to improve as new data are incorporated.

References

1. Lin J-R, Chen Y-A, Campton D, Cooper J, Coy S, Yapp C, et al. High-plex immunofluorescence imaging and traditional histology of the same tissue section for discovering image-based biomarkers. Nat Cancer. 2023 Jun 22;

2. Wu Z, Trevino AE, Wu E, Swanson K, Kim HJ, D’Angio HB, et al. SPACE-GM: geometric deep learning of disease-associated microenvironments from multiplex spatial protein profiles. BioRxiv. 2022 May 13;

3. Schürch CM, Bhate SS, Barlow GL, Phillips DJ, Noti L, Zlobec I, et al. Coordinated cellular neighborhoods orchestrate antitumoral immunity at the colorectal cancer invasive front. Cell. 2020 Sep 3;182(5):1341–1359.e19.

4. Phillips D, Matusiak M, Gutierrez BR, Bhate SS, Barlow GL, Jiang S, et al. Immune cell topography predicts response to PD-1 blockade in cutaneous T cell lymphoma. Nat Commun. 2021 Nov 18;12(1):6726.

5. Mittal A, Moorthy AK, Bovik AC. No-reference image quality assessment in the spatial domain. IEEE Trans Image Process. 2012 Dec;21(12):4695–708.

6. He K, Zhang X, Ren S, Sun J. Deep Residual Learning for Image Recognition. arXiv. 2015;

7. Lowe DG. Distinctive Image Features from Scale-Invariant Keypoints. Int J Comput Vis. 2004 Nov;60(2):91–110.

8. Chen L-C, Papandreou G, Schroff F, Adam H. Rethinking Atrous Convolution for Semantic Image Segmentation. arXiv. 2017;