Development and validation of an interpretable machine learning model for predicting the risk of distant metastasis in papillary thyroid cancer: a multicenter study
Background The survival rate of patients with distant metastasis (DM) of papillary thyroid carcinoma (PTC) is significantly reduced. It is of great significance to fi nd an effective method for early prediction of the risk of DM for formulating individualized diagnosis and treatment plans and improving prognosis. Previous studies have significant limitations, and it is still necessary to develop new models for predicting the risk of DM of PTC. We aimed to develop and validate interpretable machine learning (ML) models for early prediction of DM in patients with PTC using a multicenter cohort. Methods We collected data on patients with PTC who were admitted between June 2013 and May 2023. Data from 1430 patients at Yunnan Cancer Hospital (YCH) served as the training and internal validation set, while data from 434 patients at the First Affiliated Hospital of Kunming Medical University (KMU 1st AH) was used as the external test set. Nine ML methods such as random forest (RF) were used to construct the model. Model prediction performance was compared using evaluation indicators such as the area under the receiver operating characteristic curve (AUC). The SHapley Additive exPlanation (SHAP) method was used to rank the feature importance and explain the final model. Findings Among the nine ML models, the RF model performed the best. The RF model accurately predicted the risk of DM in patients with PTC in both the internal validation of the training set [AUC: 0.913, 95% confidence interval (CI) (0.9075-0.9185)] and the external test set [AUC: 0.8996, 95% CI (0.8483-0.9509)]. The calibration curve showed high agreement between the predicted and observed risks. In the sensitivity analysis focusing on DM sites of PTC, the RF model exhibited outstanding performance in predicting "lung-only metastasis" showing high AUC, specificity, sensitivity, F1 score, and a low Brier score. SHAP analysis identified variables that contributed to the model predictions. An online calculator based on the RF model was developed and made available for clinicians at https://predictingdistantmetastasis.shinyapps.io/shiny1/. 11 variables were included in the fi nal RF model: age of the patient with PTC, whether the tumor size is > 2 cm, whether the tumor size is <= 1 cm, lymphocyte (LYM) count, monocyte (MONO) count, monocyte/lymphocyte ratio (MLR), thyroglobulin (TG) level, thyroid peroxidase antibody (TPOAb) level, whether the T stage is T1/2, whether the T stage is T3/4, and whether the N stage is N0. Interpretation On the basis of large-sample and multicenter data, we developed and validated an explainable ML model for predicting the risk of DM in patients with PTC. The model helps clinicians to identify high-risk patients early and provides a basis for individualized patient treatment plans.
基金:
National Natural Science Foundation of China [81960426, 82360345, 82001986]; Outstanding Youth Science Foundation of Yunnan Basic Research Project [202401AY070001-316]; Yunnan Province Applied and Basic Research Foundation [202401AT070008]; Ten Thousand Talent Plans for Young Top-notch Talents of Yunnan Province
第一作者机构:[1]Kunming Med Univ, Peking Univ Canc Hosp Yunnan, Yunnan Canc Hosp, Dept Nucl Med,Affiliated Hosp 3, 519 Kunzhou Rd, Kunming 650118, Yunnan, Peoples R China
共同第一作者:
通讯作者:
推荐引用方式(GB/T 7714):
Hou Fei,Zhu Yun,Zhao Hongbo,et al.Development and validation of an interpretable machine learning model for predicting the risk of distant metastasis in papillary thyroid cancer: a multicenter study[J].ECLINICALMEDICINE.2024,77:doi:10.1016/j.eclinm.2024.102913.
APA:
Hou, Fei,Zhu, Yun,Zhao, Hongbo,Cai, Haolin,Wang, Yinghui...&Chen, Ting.(2024).Development and validation of an interpretable machine learning model for predicting the risk of distant metastasis in papillary thyroid cancer: a multicenter study.ECLINICALMEDICINE,77,
MLA:
Hou, Fei,et al."Development and validation of an interpretable machine learning model for predicting the risk of distant metastasis in papillary thyroid cancer: a multicenter study".ECLINICALMEDICINE 77.(2024)