Background Immunotherapy shows prominent clinical activity across multiple advanced cancers, but less than half of patients respond even with molecule-based selection. Therefore, improved biomarkers are required to improve patient selection and treatment outcomes.

Methods In this multicohort study, we developed a machine learning model to predict Immunotherapy response by integrating tumor nucleus morphological features, tumor microenvironment features, and deep learning features of digitized hematoxylin and eosin (H&E) image from a three cohorts of gastric cancer patients treated with PD-(L)1 blockade. Using domain expert annotations, we developed a computational workflow to extract patient-level pathomics features and used a machine-learning approach to integrate multimodal features into a risk prediction model.

Results The model achieved high sensitivity and specificity in predicting clinical response to immunotherapy and predicted both overall survival and progression-free survival in the test data across different cohorts. Our pathomics model had favorable accuracy for the prediction of immunotherapy response in both internal and external validation cohorts (area under the curve: 0.75–0.78), outperformed unimodal measures, and programmed death ligand-1 immunohistochemistry score.

Conclusions Our study provides a quantitative rationale for using multimodal pathomics features to improve prediction of immunotherapy response in patients with advanced gastric cancer using expert-guided machine learning.

Ethics Approval Ethical approval was obtained from the institutional review boards of the participating centers, and patient consent was waived for this retrospective analysis.

Consent Ethical approval was obtained from the institutional review boards of the participating centers, and patient consent was waived for this retrospective analysis.