Article Text

Download PDFPDF

1305 Agreement and reliability of an automated PD-L1 tumor proportion scoring algorithm in non-small cell lung cancer (NSCLC)
  1. Daniela F Rodrigues1,
  2. Christina Neppl2,
  3. David Dorward3,
  4. Tereza Losmanová4,
  5. Rebecca Wyatt1,
  6. Donna Mulkern1,
  7. Samuel Pattle3,
  8. Raphaël Oberson4,
  9. Stefan Reinhard4,
  10. Therese Waldburger4,
  11. Inti Zlobec4 and
  12. Peter Caie1
  1. 1Indica Labs, Albuquerque, NM, USA
  2. 2Institute of Pathology, University Hospital Düsseldorf, Düsseldorf, Germany
  3. 3NHS Lothian, Lothian, UK
  4. 4Institute of Tissue Medicine and Pathology, University of Bern, Bern, Switzerland
  • Journal for ImmunoTherapy of Cancer (JITC) preprint. The copyright holder for this preprint are the authors/funders, who have granted JITC permission to display the preprint. All rights reserved. No reuse allowed without permission.


Background Immunotherapy has revolutionized the treatment paradigm of advanced non-small cell lung cancer (NSCLC), opening possibilities for long-term survival outcomes with superior tolerability.1 Eligibility for immune checkpoint inhibitors is set by estimating the tumor proportion score (TPS) in programmed cell death ligand 1 (PD-L1) immunohistochemically stained slides.2 However, high interobserver variability in reporting PD-L1 expression may result in suboptimal treatment decisions.3 We developed HALO PD-L1 Lung AI to support pathologists in quantifying PD-L1 expression in NSCLC samples, with the aim of reducing interobserver variability.

Methods HALO PD-L1 Lung AI was trained using 146,984 expert annotations to identify PD-L1 tumor-positive and tumor-negative cells, within segmented tumor regions. The agreement and reliability of the algorithm’s TPS score with scores obtained from three independent pathologists were analyzed by calculatingthe interobserver percent agreement and intraclass correlation coefficient (ICC) in a cohort of 203 whole slide images stained with the SP263 clone.

Results Pairwise pathologist agreement ranged from 74.9% to 77.3%. Agreement of HALO PD-L1 Lung AI TPS scores with the pathologists’ mode (where at least 2/3 pathologists agreed) was 75.4%. Agreement of HALO PD-L1 Lung AI with the pathologists’ mode at the clinically relevant cut-offs <1%, 1–49% and >50% was 0.81 (95% CI 0.75 – 0.88), 0.72 (95% CI 0.64 – 0.79), and 0.70 (95% CI 0.57 – 0.81), respectively. In 15 of the 18 disagreement cases at the 1% cut-off, the algorithm score was within a 1–4% range, showing that although in a categorical scale the cases were in disagreement, the results were close on a continuous scale. In fact, the intraclass correlation coefficient (ICC) between the algorithm and pathologists’ TPS scores was 0.95 (95% CI 0.93 – 0.97) and between the three pathologists was 0.96 (95% CI 0.93 – 0.97).

Conclusions The percent agreement of HALO PD-L1 Lung AI with the pathologists’ mode is in line with the pairwise agreement between pathologists. On the continuous scale, ICC results show good-to-excellent reliability between the algorithm and pathologists’ TPS scores. Computer-aided diagnostic tools such as HALO PD-L1 Lung AI have the potential to increase consistency in the reported TPS results and ultimately improve treatment decision making.


  1. Jiang M, Liu C, Ding D, Tian H, Yu C. Comparative Efficacy and Safety of Anti-PD-1/PD-L1 for the Treatment of Non-Small Cell Lung Cancer: A Network Meta-Analysis of 13 Randomized Controlled Studies. Front Oncol. 2022;12:827050.

  2. Tsao MS, Kerr KM, Kockx M, Beasley MB, Borczuk AC, Botling J, et al. PD-L1 Immunohistochemistry Comparability Study in Real-Life Clinical Samples: Results of Blueprint Phase 2 Project. J Thorac Oncol. 2018 Sep 1;13(9):1302–11.

  3. Rimm DL, Han G, Taube JM, Yi ES, Bridge JA, Flieder DB, et al. A Prospective, Multi-institutional, Pathologist-Based Assessment of 4 Immunohistochemistry Assays for PD-L1 Expression in Non-Small Cell Lung Cancer. JAMA Oncol. 2017 Aug 1;3(8):1051–8.

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See

Statistics from

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.