Multimodal Deep Learning for Lung Cancer Analysis Using Data Visualisation

Abstract

Lung cancer is still one of the leading causes of cancer deaths worldwide. Early and accurate risk prediction can help doctors make better decisions and improve patient outcomes. In this work, we develop a deep-learning framework that combines clinical records, imaging features, and survey data to predict lung cancer and its prognosis. For imaging, we use pretrained convolutional neural networks to extract features from CT and X-ray images. For clinical history, we use recurrent models, and for structured data, we apply gradient-ensemble models. We combine these features into a fully connected layer and fine-tune the model end-to-end. We test our model on several open datasets, including Kaggle lung CT sets, IQ-OTHNCCD, and a diagnostic survey dataset. We report accuracy, precision, recall, F1, and ROC-AUC. To ensure a fair evaluation, we use stratified cross-validation, tune hyperparameters, and run ablation studies to see how each data type contributes. Our combined model consistently outperforms both image-only and tabular-only models. It improves ROC-AUC and F1 scores and reduces false negatives, which is especially important for diagnosis. We also provide interactive visualisations in Looker Studio to show which features matter most, how risk is distributed, and confusion matrices, helping clinicians understand the model. We use statistical tests to confirm our improvements and discuss ways to make predictions more understandable for clinical use. Our results demonstrate that combining different data types enhances accuracy and provides valuable insights that support informed clinical decisions. We also discuss limitations, such as differences between datasets and label noise, and suggest further experiments and external validation to move closer to clinical use.