Adversarial Networks for Spatial Context-Aware Spectral Image Reconstruction from RGB

Hyperspectral signal reconstruction aims at recovering the original spectral input that produced a certain trichromatic (RGB) response from a capturing device or observer. Given the heavily underconstrained, non-linear nature of the problem, traditional techniques leverage different statistical properties of the spectral signal in order to build informative priors from real world object reflectances for constructing such RGB to spectral signal mapping. However, most of them treat each sample independently, and thus do not benefit from the contextual information that the spatial dimensions can provide. We pose hyperspectral natural image reconstruction as an image to image mapping learning problem, and apply a conditional generative adversarial framework to help capture spatial semantics. This is the first time Convolutional Neural Networks -and, particularly, Generative Adversarial Networks- are used to solve this task. Quantitative evaluation shows a Root Mean Squared Error (RMSE) drop of 44.7% and a Relative RMSE drop of 47.0% on the ICVL natural hyperspectral image dataset

HMDB: A Large Video Database for Human Motion Recognition

With nearly one billion online videos viewed everyday, an emerging new frontier in computer vision research is recognition and search in video. While much effort has been devoted to the collection and annotation of large scalable static image datasets containing thousands of image categories, human action datasets lag far behind. Current action recognition databases contain on the order of ten different action categories collected under fairly controlled conditions. State-of-the-art performance on these datasets is now near ceiling and thus there is a need for the design and creation of new benchmarks. To address this issue we collected the largest action video database to-date with 51 action categories, which in total contain around 7,000 manually annotated clips extracted from a variety of sources ranging from digitized movies to YouTube. We use this database to evaluate the performance of two representative computer vision systems for action recognition and explore the robustness of these methods under various conditions such as camera motion, viewpoint, video quality and occlusion.United States. Defense Advanced Research Projects Agency. Information Processing Techniques OfficeUnited States. Defense Advanced Research Projects Agency. System Science Division. Defense Sciences OfficeNational Science Foundation (U.S.) (NSF-0640097)National Science Foundation (U.S.) (NSF-0827427)United States. Air Force Office of Scientific Research (FA8650-05- C-7262)Adobe SystemsKing Abdullah University of Science and TechnologyNEC ElectronicsSony CorporationEugene McDermott FoundationBrown University. Center for Computing and VisualizationRobert J. and Nancy D. Carney Fund for Scientific InnovationUnited States. Defense Advanced Research Projects Agency (DARPA-BAA-09-31)United States. Office of Naval Research (ONR-BAA-11-001)Ministry of Science, Research and the Arts of Baden Württemberg, German

Trainable, vision-based automated home cage behavioral phenotyping

We describe a fully trainable computer vision system enabling the automated analysis of complex mouse behaviors. Our system computes a sequence of feature descriptors for each video sequence and a classifier is used to learn a mapping from these features to behaviors of interest. We collected a very large manually annotated video database of mouse behaviors for training and testing the system. Our system performs on par with human scoring, as measured from the ground-truth manual annotations of thousands of clips of freely behaving mice. As a validation of the system, we characterized the home cage behaviors of two standard inbred and two nonstandard mouse strains. From this data, we were able to predict the strain identity of individual mice with high accuracy.California Institute of Technology. Broad Fellows Program in Brain CircuitryNational Science Council of Taiwan (TMS-094-1-A032

Automated home-cage behavioral phenotyping of mice

We describe a trainable computer vision system enabling the automated analysis of complex mouse behaviors. We provide software and a very large manually annotated video database used for training and testing the system. Our system outperforms leading commercial software and performs on par with human scoring, as measured from the ground-truth manual annotations of thousands of clips of freely behaving animals. We show that the home-cage behavior profiles provided by the system is sufficient to accurately predict the strain identity of individual animals in the case of two standard inbred and two non-standard mouse strains. Our software should complement existing sensor-based automated approaches and help develop an adaptable, comprehensive, high-throughput, fine-grained, automated analysis of rodent behavior

Automatic pigmented lesion segmentation through a dermoscopy-guided OCT approach for early diagnosis

Early diagnosis of pigmented lesions, specially melanoma, is an unmet clinical need that would help to improve patient prognosis. Apart from histopathological biopsy, the only gold standard non-invasive imaging technique during diagnosis is dermatoscopy (DD). Over the last years, new medical imaging techniques are being developed and Optical Coherence Tomography (OCT) has demonstrated to be very helpful on dermatology. OCT is non-invasive and provides in-depth structural microscopic information of the skin in real-time. In comparison with other novel techniques, as Reflectance Confocal Microscopy (RCM), the acquisition time is lower and the field-of-view higher. Hence, consolidated diagnosis techniques and novel imaging modalities can be combined to improve decision making during diagnosis and treatment. With actual methods, the delineation of lesion margins directly on OCT images during early stages of the disease is still really challenging and, at the same time, relevant from a prognosis perspective. This work proposes combining DD and OCT images to take advantage of their complementary information. The goal is to guide lesions delineation on OCT images considering the clinical features on DD images. The developed method applies image processing techniques to DD image to automatically segment the lesion; later, and after a calibration procedure, DD and OCT images become coregistered. In a final step the DD segmentation is transferred into the OCT image. Applying advanced image processing techniques and the proposed strategy of lesion delimitation, histopathological characteristics of the segmented lesion can be studied on OCT images afterwards. This proposal can lead to early, real-time and non-invasive diagnosis of pigmented lesions.This work has been developed thanks to the funding of the ECSEL European project ASTONISH (ID.692470) and Basque Country (Spain) ELKARTEK projects MELAMICS (KK-2016-00036) and MELAMICS II (KK-2017/00041). Special thanks to the dermatologists and personnel of the Cruces University Hospital (Cruces, Spain) and the Basurto University Hospital (Bilbao, Spain) for their collaboration on the generation of the annotated database from real patients

Detección de fibrilación ventricular mediante técnicas de aprendizaje profundo

Detección de fibrilación ventricular mediante técnicas de aprendizaje profundo La detección de arritmias ventriculares, en particular la fibrilación ventricular (FV), es parte fundamental de los algoritmos de clasificación de arritmias de los desfibriladores. Dichos algoritmos deciden si administrar la descarga de desfibrilación, para lo que clasifican los ritmos en desfibrilables (Sh) o no desfibrilables (NSh). Este trabajo propone un nuevo abordaje para la clasificación Sh/NSh de ritmos basado en un sistema de aprendizaje profundo. Para el trabajo se emplearon tres bases de datos públicas de la plataforma Physionet (CUDB, VFDB y AHADB), y se extrajeron segmentos de 4 y 8 segundos. Se anotaron los segmentos como Sh y NSh en base a las anotaciones de las bases de datos, que fueron auditadas por expertos. Los datos se dividieron por paciente en 80% para desarrollar los algoritmos y 20% para evaluación. El sistema de aprendizaje profundo emplea dos etapas convolucionales seguidas de, una red longshort- term-memory y una etapa final de clasificación basada en red neuronal. A modo de referencia se optimizó un clasificador SVM basado en las características de detección de arritmias ventriculares más eficientes publicadas en la literatura. Se calculó la sensibilidad (Se), ritmos desfibrilables, especificidad (Sp), ritmos no desfibrilables, y la precisión (Acc). El método de aprendizaje profundo proporcionó Se, Sp y Acc de 98.5%, 99.4% y 99.2% para segmentos de 4 segundos y 99.7%, 98.9%, 99.1% para segmentos de 8 segundos. El algoritmo permite detectar FV de forma fiable con segmentos de 4 segundos, corrigiendo un 30% de los errores del método basado en SVM.Este trabajo ha sido financiado por el Ministerio de Economía y Competitividad mediante el proyecto TEC2015-64678R junto con el Fondo Europeo de Desarrollo Regional (FEDER), así como por la UPVEHU mediante el proyecto EHU16/18

MRI Deep Learning-Based Solution for Alzheimer’s Disease Prediction

Background: Alzheimer’s is a degenerative dementing disorder that starts with a mild memory impairment and progresses to a total loss of mental and physical faculties. The sooner the diagnosis is made, the better for the patient, as preventive actions and treatment can be started. Al though tests such as the Mini-Mental State Tests Examination are usually used for early identification, diagnosis relies on magnetic resonance imaging (MRI) brain analysis. Methods: Public initiatives such as the OASIS (Open Access Series of Imaging Studies) collection provide neuroimaging datasets openly available for research purposes. In this work, a new method based on deep learning and image processing techniques for MRI-based Alzheimer’s diagnosis is proposed and compared with previous literature works. Results: Our method achieves a balance accuracy (BAC) up to 0.93 for image-based automated diagnosis of the disease, and a BAC of 0.88 for the establishment of the disease stage (healthy tissue, very mild and severe stage). Conclusions: Results obtained surpassed the state-of-the-art proposals using the OASIS collection. This demonstrates that deep learning-based strategies are an effective tool for building a robust solution for Alzheimer’s-assisted diagnosis based on MRI data.This work was partially supported by the SUPREME project. This project has received funding from the Basque Government’s Industry Department HAZITEK program under agreement ZE-2019/00022. This research has also received funding from the Basque Government’s Industry Department under the ELKARTEK program’s project ONKOTOOLS under agreement KK-2020/0006

Characterization of Optical Coherence Tomography Images for Colon Lesion Differentiation under Deep Learning

(1) Background: Clinicians demand new tools for early diagnosis and improved detection of colon lesions that are vital for patient prognosis. Optical coherence tomography (OCT) allows microscopical inspection of tissue and might serve as an optical biopsy method that could lead to in-situ diagnosis and treatment decisions; (2) Methods: A database of murine (rat) healthy, hyperplastic and neoplastic colonic samples with more than 94,000 images was acquired. A methodology that includes a data augmentation processing strategy and a deep learning model for automatic classification (benign vs. malignant) of OCT images is presented and validated over this dataset. Comparative evaluation is performed both over individual B-scan images and C-scan volumes; (3) Results: A model was trained and evaluated with the proposed methodology using six different data splits to present statistically significant results. Considering this, 0.9695 (_0.0141) sensitivity and 0.8094 (_0.1524) specificity were obtained when diagnosis was performed over B-scan images. On the other hand, 0.9821 (_0.0197) sensitivity and 0.7865 (_0.205) specificity were achieved when diagnosis was made considering all the images in the whole C-scan volume; (4) Conclusions: The proposed methodology based on deep learning showed great potential for the automatic characterization of colon polyps and future development of the optical biopsy paradigm.This work was partially supported by PICCOLO project. This project has received funding from the European Union’s Horizon2020 Research and Innovation Programme under grant agreement No. 732111. This research has also received funding from the Basque Government’s Industry Department under the ELKARTEK program’s project ONKOTOOLS under agreement KK-2020/00069 and the industrial doctorate program UC- DI14 of the University of Cantabria

Mixed convolutional and long short-term memory network for the detection of lethal ventricular arrhythmia

Early defibrillation by an automated external defibrillator (AED) is key for the survival of out-of-hospital cardiac arrest (OHCA) patients. ECG feature extraction and machine learning have been successfully used to detect ventricular fibrillation (VF) in AED shock decision algorithms. Recently, deep learning architectures based on 1D Convolutional Neural Networks (CNN) have been proposed for this task. This study introduces a deep learning architecture based on 1D-CNN layers and a Long Short-Term Memory (LSTM) network for the detection of VF. Two datasets were used, one from public repositories of Holter recordings captured at the onset of the arrhythmia, and a second from OHCA patients obtained minutes after the onset of the arrest. Data was partitioned patient-wise into training (80%) to design the classifiers, and test (20%) to report the results. The proposed architecture was compared to 1D-CNN only deep learners, and to a classical approach based on VF-detection features and a support vector machine (SVM) classifier. The algorithms were evaluated in terms of balanced accuracy (BAC), the unweighted mean of the sensitivity (Se) and specificity (Sp). The BAC, Se, and Sp of the architecture for 4-s ECG segments was 99.3%, 99.7%, and 98.9% for the public data, and 98.0%, 99.2%, and 96.7% for OHCA data. The proposed architecture outperformed all other classifiers by at least 0.3-points in BAC in the public data, and by 2.2-points in the OHCA data. The architecture met the 95% Sp and 90% Se requirements of the American Heart Association in both datasets for segment lengths as short as 3-s. This is, to the best of our knowledge, the most accurate VF detection algorithm to date, especially on OHCA data, and it would enable an accurate shock no shock diagnosis in a very short time.This study was supported by the Ministerio de Economía, Industria y Competitividad, Gobierno de España (ES) (TEC-2015-64678-R) to UI and EA and by Euskal Herriko Unibertsitatea (ES) (GIU17/031) to UI and EA. The funders, Tecnalia Research and Innovation and Banco Bilbao Vizcaya Argentaria (BBVA), provided support in the form of salaries for authors AP, AA, FAA, CF, EG, but did not have any additional role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript. The specific roles of these authors are articulated in the author contributions section