In this study, the long-term mortality in the National Lung Screening Trial (NLST) was investigated using a deep learning-based method. Binary classification of the non-lung-cancer mortality (i.e. cardiovascular and respiratory mortality) was performed using neural network models centered around a 3D-ResNet. The models were trained on a participant age, gender, and smoking history matched cohort. Utilising both the 3D CT scan and clinical information, the models can achieve an AUC of 0.73 which outperforms humans at cardiovascular mortality prediction. The corresponding F1 and Matthews Correlation Coefficient are 0.60 and 0.38 respectively. By interpreting the trained models with 3D saliency maps, we examined the features on the CT scans that correspond to the mortality signal. By extracting information from 3D CT volumes, we can highlight regions in the thorax region that contribute to mortality that might be overlooked by the clinicians. Therefore, this can help focus preventative interventions appropriately, particularly for under-recognised pathologies and thereby reducing patient morbidity