Data fusion by using machine learning and computational intelligence techniques for medical image analysis and classification

Data fusion is the process of integrating information from multiple sources to produce specific, comprehensive, unified data about an entity. Data fusion is categorized as low level, feature level and decision level. This research is focused on both investigating and developing feature- and decision-level data fusion for automated image analysis and classification. The common procedure for solving these problems can be described as: 1) process image for region of interest\u27 detection, 2) extract features from the region of interest and 3) create learning model based on the feature data. Image processing techniques were performed using edge detection, a histogram threshold and a color drop algorithm to determine the region of interest. The extracted features were low-level features, including textual, color and symmetrical features. For image analysis and classification, feature- and decision-level data fusion techniques are investigated for model learning using and integrating computational intelligence and machine learning techniques. These techniques include artificial neural networks, evolutionary algorithms, particle swarm optimization, decision tree, clustering algorithms, fuzzy logic inference, and voting algorithms. This work presents both the investigation and development of data fusion techniques for the application areas of dermoscopy skin lesion discrimination, content-based image retrieval, and graphic image type classification --Abstract, page v

Risk factors and biomarkers for metastatic cutaneous squamous cell carcinoma

The incidence of cutaneous squamous cell carcinoma (cSCC), the most common skin cancer with metastatic potential, continues to increase. Although proportion of cSCCs metastasize and cause mortality, sufficient means to identify the metastasis-prone tumors are not available. In this thesis the metastatic cSCCs from the area served by Turku University Hospital were identified and characterized revealing that the rate of metastasis in the study region was 2.3%. Further, it was discovered that metastasis occurs rapidly and that there was no history of cSCC in 85% of patients with metastatic cSCC. Invasion depth, tumor diameter, age and location on lower lip or forehead were associated with increased risk of metastasis. On the other hand, usage of isosorbide mono-/dinitrate and aspirin as well as comorbidity with premalignant lesions or basal cell carcinoma were associated with lower risk of metastasis. With multiplexed immunohistochemistry, it was demonstrated that the activity and phenotype of cancer-associated fibroblasts (CAFs) evolve during the progression of cSCC. Elevation of α-smooth muscle actin (αSMA), secreted protein acidic and rich in cysteine (SPARC) and fibroblast activating protein (FAP) expression was associated with invasion and expression of FAP and platelet-derived growth factor receptor-β (PDGFRβ) with metastasis. High expression of stromal PDGFRβ and periostin were associated with worse prognosis. CAF107 (PDGFRα-/PDGFRβ+/FAP+) subset was associated with invasion and metastasis, and predicted poor prognosis of cSCC. A deep learning algorithm was harnessed to distinguish primary tumors that metastasize rapidly from non-metastatic cSCCs with slide level area under the receiver operating characteristic curve (AUROC) of 0.747 on whole slide images representing primary cSCCs. Furthermore, a risk factor model, that utilized prediction by AI, was created and provided staging systems and comparative risk factor models surpassing classification and prognostivity. These results characterize features associated with the metastasis risk of cSCC and indicate that CAF-markers and AI could provide clinical tools for the metastasis risk assessment and thus improve the prognosis of patient with metastatic cSCC.Etäpesäkkeitä lähettävän okasolusyövän riskitekijät ja biomarkkerit Yleisimmän etäpesäkkeitä lähettävän ihosyövän, okasolusyövän, ilmaantuvuus jatkaa kasvuaan. Vaikka osa okasolusyövistä lähettää etäpesäkkeitä ja aiheuttaa kuolleisuutta, ei etäpesäkkeitä lähettämään tulevien okasolusyöpien tunnistamiseksi ole toistaiseksi riittäviä keinoja. Tässä väitöskirjassa karakterisoitiin Turun yliopistollisen keskussairaalan vastuualueen metastasoituneet okasolusyövät ja osoitettiin että tutkimusalueen okasolusyövistä 2.3% etenee etäpesäkkeitä lähettäväksi. Metastasoituminen tapahtui nopeasti ja valtaosassa tapauksista (85%) etäpesäkkeen lähetti ensimmäinen potilaalla todettu okasolusyöpä. Ikä, kasvaimen invaasiosyvyys, halkaisija ja sijainti alahuulessa tai otsalla yhdistyivät kohonneeseen metastaasiriskiin. Isosorbidinitraatin ja aspiriinin käyttö sekä esiasteiden ja tyvisolusyövän esiintyminen taas liittyivät alentuneeseen metastaasiriskiin. Multiplex-immunohistokemiaa hyödyntäen osoitettin, että syöpään liittyvien fibroblastien (CAF) aktiviteetti ja ilmiasu muuttuu okasolusyövän edetessä. Kohonnut sileälihasaktiini alfan (αSMA), osteonektiinin ja fibroblastia aktivoivan proteiinin (FAP) ilmentyminen liittyi invaasioon ja FAP:n sekä verihiutaleista johdetun kasvutekijäreseptori β:n (PDGFRβ) etäpesäkkeiden lähettämiseen. PDGFRβ:n ja periostiinin ilmentyminen taas yhdistyi huonoon ennusteeseen. CAF107 (PDGFRα-/PDGFRβ+/FAP+) alatyyppi liittyi invaasioon, metastasointiin ja huonoon ennusteeeseen. Etäpesäkkeitä lähettämään tulevien okasolusyöpien tunnistamiseen valjastettu syväoppimisalgoritmi erotti okasolusyöpiä edustavista digitalisoiduista mikroskopiakuvista nopeasti etäpesäkkeitä lähettävät okasolusyövät okasolusyövistä, jotka eivät lähetä etäpesäkkeitä, leiketason AUROC-arvolla 0.747. Tekoälyarviota hyödyntävä riskitekijämalli voitti luokittelujärjestelmät ja kilpailevat riskitekijämallit okasolusyöpien luokittelussa ja ennusteen arvioinnissa. Tulokset antavat lisätietoa metastasoituvan okasolusyövän luonteesta ja osoittavat CAF-markkereiden sekä tekoälyn voivan tarjota kliinisiä työkaluja okasolusyövän metastaasiriskin arviointiin ja täten voivan parantaa etäpesäkkeitä lähettävän okasolusyöpäpotilaan ennustetta tulevaisuudessa

Exfoliative cytology for diagnosing basal cell carcinoma and other skin cancers in adults

Background: Early accurate detection of all skin cancer types is essential to guide appropriate management and to reduce morbidity and improve survival. Basal cell carcinoma (BCC) is usually localised to the skin with potential to infiltrate and damage surrounding tissue, while cutaneous squamous cell carcinoma (cSCC) and melanoma have a much higher potential to metastasise and ultimately lead to death. Exfoliative cytology is a non–invasive test that uses the Tzanck smear technique to identify disease by examining the structure of cells obtained from scraped samples. This simple procedure is a less invasive diagnostic test than a skin biopsy, and for BCC has the potential to provide an immediate diagnosis that avoids an additional visit to clinic to receive skin biopsy results. This may benefit patients scheduled for either Mohs micrographic surgery or non–surgical treatments such as radiotherapy. A cytology scrape can never give the same information as a skin biopsy, however, so it is important to know more about which skin cancer situations it may be helpful.Objectives: The primary objective was to determine the diagnostic accuracy of exfoliative cytology for the detection of basal cell carcinoma (BCC) in adults. Secondary objectives were to determine diagnostic accuracy for the detection of i) cutaneous squamous cell carcinoma, ii) invasive melanoma and atypical intraepidermal melanocytic variants, and iii) any skin cancer, including keratinocyte skin cancer, invasive melanoma and atypical intraepidermal melanocytic variants, or any other skin cancer.Search methods: We undertook a comprehensive search of the following databases from inception up to August 2016: Cochrane Central Register of Controlled Trials; MEDLINE; EMBASE; CINAHL; CPCI; Zetoc; Science Citation Index; US National Institutes of Health Ongoing Trials Register; NIHR Clinical Research Network Portfolio Database; and the World Health Organization International Clinical Trials Registry Platform. We also studied the reference lists of published systematic review articles.Selection criteria: Studies evaluating exfoliative cytology in adults with lesions suspicious for BCC, cSCC or melanoma, compared with a reference standard of histological confirmation.Data collection and analysis: Two review authors independently extracted all data using a standardised data extraction and quality assessment form (based on QUADAS-2). Where possible we estimated summary sensitivities and specificities using the bivariate hierarchical model.Main results: This review reports on nine studies with a total of 1655 lesions including 1120 BCCs (14 datasets), 401 lesions with 44 cSCCs (two datasets), and 200 lesions with 10 melanomas (one dataset). Three of these datasets (one each for BCC, melanoma, and any malignant condition) were derived from one study which also performed a direct comparison with dermoscopy. Studies were of moderate to poor quality providing inadequate descriptions of participant selection, thresholds used to make cytological and histological diagnoses, and blinding. Reporting of patients’ prior referral pathways was particularly poor, as were descriptions of the cytodiagnostic criteria used to make diagnoses. No studies evaluated the use of exfoliative cytology as a primary diagnostic test for detecting BCC or other skin cancers in lesions suspicious for skin cancer. Pooled data from seven studies using standard cytomorphological criteria (but various stain methods) to detect BCC in patients with a high clinical suspicion of BCC estimated the sensitivity and specificity of exfoliative cytology as 97.5% (95% CI: 94.5 to 98.9%) and 90.1% (95% CI: 81.1 to 95.1%) respectively. When applied to a hypothetical population of 1000 clinically suspected BCC lesions with a median observed BCC prevalence of 86%, exfoliative cytology would miss 21 BCCs and would lead to 14 false positive diagnoses of BCC. No false positive cases were histologically confirmed to be melanoma. Insufficient data are available to make summary statements regarding the accuracy of exfoliative cytology to detect melanoma or cSCC, or its accuracy compared to dermoscopy.Authors' conclusions: The utility of exfoliative cytology for the primary diagnosis of skin cancer is unknown, as all included studies focused on the use of this technique for confirming strongly suspected clinical diagnoses. For the confirmation of BCC in lesions with a high clinical suspicion, there is evidence of high sensitivity and specificity for exfoliative cytology. Since decisions to treat low riskBCCs are unlikely in practice to require diagnostic confirmation given that clinical suspicion is already high, exfoliative cytology might be most useful for cases of BCC where the treatments being contemplated require a tissue diagnosis (e.g. radiotherapy). The small number of included studies, poor reporting and varying methodological quality means that no strong conclusions can currently be drawn to guide clinical practice. Despite insufficient data on the use of cytology for cSCC or melanoma, it is unlikely that cytology would be useful in these scenarios since preservation of the architecture of the whole lesion that would be available from a biopsy provides crucial diagnostic information. Given the paucity of good quality data, appropriately designed prospective comparative studies may be required to evaluate both the diagnostic value of exfoliative cytology by comparison to dermoscopy, and its confirmatory value in adequately reported populations with a high probability of BCC scheduled for further treatment requiring a tissue diagnosis

Visual examination and dermoscopy, alone or in combination, for the diagnosis of keratinocyte skin cancers in adults

Background Early accurate detection of all skin cancer types is important to guide appropriate management, to reduce morbidity and to improve survival. Basal cell carcinoma (BCC) is almost always a localised skin cancer with potential to infiltrate and damage surrounding tissue, whereas a minority of squamous cell carcinoma (cSCC) and invasive melanoma are higher risk skin cancers with the potential to metastasise and cause death. Dermoscopy has become an important tool to assist specialist clinicians in the diagnosis of melanoma, and is increasingly used in primary care settings. Dermoscopy is a precision-built handheld illuminated magnifier that allows more detailed examination of the skin down to the level of the superficial dermis. Establishing the value of dermoscopy over and above visual inspection for the diagnosis of BCC or cSCC in primary and secondary care settings is critical to understanding its potential contribution to appropriate skin cancer triage, including referral of higher risk cancers to secondary care, the identification of low risk skin cancers that might be treated in primary care and to provide reassurance to those with benign skin lesions who can be safely discharged. Objectives To determine the diagnostic accuracy of visual inspection and dermoscopy, alone or in combination, for the detection of a) BCC and b) cSCC, in adults. Studies were separated according to whether the diagnosis was recorded face-to-face (in-person) or based on remote (image-based) assessment. Search methods We undertook a comprehensive search of the following databases from inception up to August 2016: Cochrane Central Register of Controlled Trials; MEDLINE; Embase; CINAHL; CPCI; Zetoc; Science Citation Index; US National Institutes of Health Ongoing Trials Register; NIHR Clinical Research Network Portfolio Database; and the World Health Organization International Clinical Trials Registry Platform. We studied reference lists and published systematic review articles. Selection criteria Studies of any design that evaluated visual inspection and/or dermoscopy in adults with lesions suspicious for skin cancer, compared with a reference standard of either histological confirmation or clinical follow-up. Data collection and analysis Two review authors independently extracted all data using a standardised data extraction and quality assessment form (based on QUADAS-2). We contacted authors of included studies where information related to the target condition or diagnostic threshold were missing. We estimated accuracy using hierarchical summary ROC methods. Analysis of studies allowing direct comparison between tests was undertaken. To facilitate interpretation of results, we computed values of sensitivity at the point on the SROC curve with 80% fixed specificity and values of specificity with 80% fixed sensitivity. We investigated the impact of in-person test interpretation; use of a purposely developed algorithm to assist diagnosis; and observer expertise. Main results A total of 24 publications reporting on 24 study cohorts were included, providing 27 visual inspection datasets (8805 lesions; 2579 malignancies) and 33 dermoscopy datasets (6855 lesions; 1444 malignancies). The risk of bias was mainly low for the index test (for dermoscopy evaluations) and reference standard domains, particularly for in-person evaluations, and high or unclear for participant selection, application of the index test for visual inspection and for participant flow and timing. Concerns regarding the applicability of study findings were scored as ‘high’ or 'unclear' concern for almost all studies across all domains assessed. Selective participant recruitment, lack of reproducibility of diagnostic thresholds and lack of detail on observer expertise were particularly problematic. The detection of BCC was reported in 28 datasets; 15 on an in-person basis and 13 image-based. Analysis of studies by prior testing of participants and according to observer expertise was not possible due to lack of data. Studies were primarily conducted in participants referred for specialist assessment of lesions with available histological classification. No clear differences in accuracy were noted between dermoscopy studies undertaken in-person and those which evaluated images. The lack of effect observed is likely due to other sources of heterogeneity, including variations in the types of skin lesion studied, in dermatoscopes used, in the use of algorithms and varying thresholds for deciding on a positive test result. Meta-analysis found in-person evaluations of dermoscopy (7 evaluations; 4683 lesions and 363 BCCs) to be more accurate than visual inspection alone for the detection of BCC (8 evaluations; 7017 lesions and 1586 BCCs), with an RDOR of 8.2 (95% CI: 3.5 to 19.3; P < 0.001). This corresponds to predicted differences in sensitivity of 14% (93% vs 79%) at a fixed specificity of 80% and predicted differences in specificity of 22% (99% vs 77%) at a fixed sensitivity of 80%. Very similar results were observed for the image-based evaluations. When applied to a hypothetical population of 1000 lesions, of which 170 are BCC (based on median BCC prevalence across studies), an increased sensitivity of 14% from dermoscopy would lead to 24 fewer BCCs missed, assuming 166 false positive results from both tests. A 22% increase in specificity from dermoscopy with sensitivity fixed at 80% would result in 183 fewer unnecessary excisions assuming 34 BCCs missed for both tests. There was not enough evidence to assess the use of algorithms or structured checklists for either visual inspection or dermoscopy. Insufficient data were available to draw conclusions on the accuracy of either test for the detection of cSCC. Authors’ conclusions Dermoscopy may be a valuable tool for the diagnosis of BCC as an adjunct to visual inspection of a suspicious skin lesion following a thorough history-taking including assessment of risk factors for keratinocyte cancer. The evidence primarily comes from secondary care (referred) populations and populations with pigmented lesions or mixed lesion types. There is no clear evidence supporting the use of currently available formal algorithms to assist dermoscopy diagnosis