Background Immunotherapy has revolutionized the treatment paradigm of advanced non-small cell lung cancer (NSCLC), opening possibilities for long-term survival outcomes with superior tolerability.¹ Eligibility for immune checkpoint inhibitors is set by estimating the tumor proportion score (TPS) in programmed cell death ligand 1 (PD-L1) immunohistochemically stained slides.² However, high interobserver variability in reporting PD-L1 expression may result in suboptimal treatment decisions.³ We developed HALO PD-L1 Lung AI to support pathologists in quantifying PD-L1 expression in NSCLC samples, with the aim of reducing interobserver variability.

Methods HALO PD-L1 Lung AI was trained using 146,984 expert annotations to identify PD-L1 tumor-positive and tumor-negative cells, within segmented tumor regions. The agreement and reliability of the algorithm’s TPS score with scores obtained from three independent pathologists were analyzed by calculatingthe interobserver percent agreement and intraclass correlation coefficient (ICC) in a cohort of 203 whole slide images stained with the SP263 clone.

Results Pairwise pathologist agreement ranged from 74.9% to 77.3%. Agreement of HALO PD-L1 Lung AI TPS scores with the pathologists’ mode (where at least 2/3 pathologists agreed) was 75.4%. Agreement of HALO PD-L1 Lung AI with the pathologists’ mode at the clinically relevant cut-offs <1%, 1–49% and >50% was 0.81 (95% CI 0.75 – 0.88), 0.72 (95% CI 0.64 – 0.79), and 0.70 (95% CI 0.57 – 0.81), respectively. In 15 of the 18 disagreement cases at the 1% cut-off, the algorithm score was within a 1–4% range, showing that although in a categorical scale the cases were in disagreement, the results were close on a continuous scale. In fact, the intraclass correlation coefficient (ICC) between the algorithm and pathologists’ TPS scores was 0.95 (95% CI 0.93 – 0.97) and between the three pathologists was 0.96 (95% CI 0.93 – 0.97).

Conclusions The percent agreement of HALO PD-L1 Lung AI with the pathologists’ mode is in line with the pairwise agreement between pathologists. On the continuous scale, ICC results show good-to-excellent reliability between the algorithm and pathologists’ TPS scores. Computer-aided diagnostic tools such as HALO PD-L1 Lung AI have the potential to increase consistency in the reported TPS results and ultimately improve treatment decision making.

References

1. Jiang M, Liu C, Ding D, Tian H, Yu C. Comparative Efficacy and Safety of Anti-PD-1/PD-L1 for the Treatment of Non-Small Cell Lung Cancer: A Network Meta-Analysis of 13 Randomized Controlled Studies. Front Oncol. 2022;12:827050.

2. Tsao MS, Kerr KM, Kockx M, Beasley MB, Borczuk AC, Botling J, et al. PD-L1 Immunohistochemistry Comparability Study in Real-Life Clinical Samples: Results of Blueprint Phase 2 Project. J Thorac Oncol. 2018 Sep 1;13(9):1302–11.

3. Rimm DL, Han G, Taube JM, Yi ES, Bridge JA, Flieder DB, et al. A Prospective, Multi-institutional, Pathologist-Based Assessment of 4 Immunohistochemistry Assays for PD-L1 Expression in Non-Small Cell Lung Cancer. JAMA Oncol. 2017 Aug 1;3(8):1051–8.