Background Immune checkpoint inhibitors (ICIs) have significantly improved progression-free and overall survival in non-small-cell lung cancer (NSCLC), but less than 20% of patients respond well, with significant variability in adverse events. Traditional biomarkers like PD-L1 have performance and availability limitations. In contrast, histopathological images offer a comprehensive view of cancer biology. Our study uses deep learning on pathological H&E images to develop a pathomics signature for predicting ICI response and assesses its clinical significance.

Methods This study included 797 NSCLC patients treated with ICIs at MD Anderson. We used a three-step framework for analysis: First, pathologists annotated images from 139 patients into eight subregions: Tumor, Stroma, Immune, Vessel, Bronchi, Necrosis, Lung, and Background. Each whole slide image (WSI) was divided into 256x256 patches, and a ResNet-34 model classified these patches. Second, we developed an ensemble deep multi-instance learning model to predict overall survival (OS) and progression-free survival (PFS) using tissue habitat maps. We combined different pooling operators on slide-level predicted risk scores to derive patient-level risk scores. Third, we integrated multiple modalities for 460 matched patients with both CT and H&E data to evaluate the added predictive value of the deep pathomics model alongside existing clinicopathological factors and a radiomics model.

Results The proposed tissue classification model achieved an 87% accuracy for labeling whole slide images during validation. The deep pathomics model demonstrated robust patient stratification on the MD Anderson test set (hazard ratio (HR) 1.01, 95% confidence interval (CI) 1.00 to 1.02, p-value =0.0018), maintaining significant performance across subgroup analyses stratified by PD-L1 expression, pathology, WSI type (biopsy and resected), age and gender. In univariate analysis, it outperformed conventional risk factors (histology, smoking status, PD-L1), remaining an independent predictor post multivariate adjustment. When integrated with conventional risk factors, the Deep-HE model significantly improved prediction performance, with the OS C-index increasing from 0.60 to 0.65 when combining radiological risks with HE risks. This performance further improved to 0.73 when clinical factors were included during testing. Additionally, a surrogate model guided by the deep pathomics risk scores was developed using hand-crafted pathological features as input. This surrogate model classifies patients into high-risk and low-risk groups, further demonstrating the efficacy of the deep pathomics model.

Conclusions Our study shows that the deep pathomics model improves tissue classification and patient stratification in NSCLC, enhancing prognostic accuracy when combined with clinicopathological and radiomics metrics, potentially informing treatment strategies for patients undergoing ICI therapy.