Predicting Post-Thoracic Surgery Life Expectancy for Lung Cancer Using Machine Learning with SHapley Additive exPlanations

Published in ICCIT 2024, IEEE Conference, DOI: 10.1109/ICCIT64611.2024.11022499, Link

Abstract: Lung cancer, accounting for 11.6% of cancer diagnoses, is still one of the most significant health concerns all over the world. Thoracic surgery is an effective and common treatment for lung cancer. Even though it is considered a medium risk procedure, predicting life expectancy after thoracic surgery is important. In this study, we want to use machine learning for accurately predicting the life expectancy of patients following thoracic surgery for lung cancer. The dataset used in this research has 470 datapoints with 16 attributes. We have performed thorough comparison between multiple models which are: K-Neighbors, Random Forest, XGboost, Adaboost classifiers, and an Ensemble Model (Stacking Technique). Since, the dataset was heavily imbalanced we solved it utilizing oversampling technique such as Random oversampling and SMOTE-ENN (Synthetic Minority Oversampling-Edited Nearest Neighbor). For validation k-fold method has been used.  The optimal model turns out to be Stacking Classifier, which achieved an impressive accuracy of 99.38% (std: 0.0040) using K-fold cross-validation (k=5) when using Random Oversampling.