Brain tumour detection in an initialpoint is a critical step to saving human life. Computed Tomography (CT) and Magnetic Resonance Image (MRI) provide very detailed information about brain tumour tissues. So the segmentation of tumour region is possible from pancreatic CT and brain MRI. CT and MRI is a non-invasive technique and it does not produce any harmful radiation to the patient. The patient suspected of tumour undergoes radiological evaluation such that the area, location and grade of the tumour can be predicted from the CT and MRI analysis. This critical information helps the doctors to decide about further treatment like chemotherapy, surgery, or radiation. The diagnosis requires an accurate and very fast segmentation and classification of CT and MRI images. But nowadays radiologists are doing this task manually and it is a tedious and time-consuming procedure. Also, there is a chance of variation in the result from one expert to another. Here comes the significance of automatic segmentation and classification of tumour types with the help of computers.The proposed work aims to develop an efficient system that can detect pancreatic and brain tumour and can classify the pancreatic CT and brain MRI into normal, benign or malignant. This work can be categorized into two approaches. Thus the dataset prepared for this research work contains CT and MRI images.The first approach proposes traditional machine learning technologies to achieve the goal. Image pre-processing, feature extraction, segmentation and classification are the various steps of the traditional machine learning method. A detailed investigation is performed through various feature extraction techniques and classification techniques for pancreatic (CT) and brain MRI. Discrete Wavelet Transform (DWT) feature, Grey Level Co-occurrence Matrix (GLCM) feature, Gabor feature, Tamura features and Edge Orientation Histogram (EOH) features and their combinations are used for the extraction of CT and MRI features. Benign tumours are non-cancerous, but malignant tumours are cancerous. In the first approach, the Support Vector Machine (SVM) is the main classifier used for pancreatic CT and brain MRI classification as normal, benign or malignant.In this technology, a huge amount of data and machines with high computational capabilities like Graphic Processing Unit (GPU) are available. Thus the second approach of this paper is to exploit all these available resources to produce accurate results. In this part, deep learning, the latest fast growing technology introduced in 2015 is used for the classification of brain MRI. A Deep Convolutional Neural Networks (DCNN) model is proposed to perform the classification task efficiently. The CNN results are compared with the results of a simple neural network classifier. This method provides accurate and it shows that deep learning based classification outperforms traditional machine learning techniques which produce an accurate result only. This research work again concentrates on the Transfer Learning (TL) methods to classify pancreatic CT and brain MRI