Classification of mild cognitive impairment and alzheimer's disease patients: a multiscale and multivariate approach

[ANGLÈS] Alzheimer's Disease (AD) is the most common form of dementia and a growing health and socioeconomic problem. Moreover, the impact of the disease is expected to increase even more as the life expectancy is going to grow over the years. Consequently, a lot of research is focused on computer-aided diagnosis techniques that aim at quantitatively study Magnetic Resonance brain images of early stage patients. Early diagnosis could help in better future cure or disease- modifying treatments. An example of AD early stage is Mild Cognitive Impariment (MCI), as the 50% of the individuals who suffer from this pathology develop AD in three of four years. In this work, we use Support Vector Machines to classify subjects from AD, MCI and healthy control (CTL) groups. Our main objective is to study whether combining different anatomical scale brain regions and different image modalities could improve the classification accuracy. Thus, regional and global Grey Matter (GM) volumes (multiscale approach), Withe Matter (WM) regional volumes, Regional Asymmetry coefficients and T1- quantitative MRI data (multivariate) are combined. Our accuracies when comparing CTL vs AD and CTL vs MCI with large public databases (ADNI) are comparable to the results in the literature: 88.3% and 81.8% respectively. In this master thesis we study also smaller databases of MCI patients from Lausanne University Hospital. We pay special attention to the study of pre-processing steps: Intra Craneal Volume normalization and age correction. Our results show that for our small group of patients, better accuracies can be obtained when combining different types of features (multiscale and multivariate) than when only using classical GM region volumes. Moreover, the new region-based age-correction method proposed here presents encouraging results when applied prior to both CTL vsMCI and CTL vs AD classification.[CASTELLÀ] La Enfermedad de Alzheimer (EA) es la forma más común de demencia y se ha convertido en un problema socioeconómico creciente. Además, se prevé que el impacto de la enfermedad será aún mayor dentro de unos años debido al progresivo envejecimiento de la población mundial y al crecimiento de la esperanza de vida. Es por estas razones que en los últimos años se ha centrado la atención en técnicas computarizadas para la diagnosis que están dirigidas al estudio cuantitativo de imágenes de resonancia magnética (MRI) de cerebro de pacientes que se encuentran en una etapa temprana de la enfermedad. Un diagnóstico precoz podría mejorar la efectividad de los futuros tratamientos de curación o modificación del curso natural de la enfermedad. Un ejemplo de etapa temprana de EA es el Deterioro Cognitivo Ligero (Mild Cognitive Impariment o MCI), puesto que el 50% de los pacientes que padecen esta patología desarrollan EA en tres o cuatro años. En este estudio, usamos Support Vector Machines para clasificar sujetos de tres grupos diferentes: EA, MCI y sujetos sanos de control (CTL). Nuestro objetivo es estudiar si combinando información a diversas escalas anatómicas del cerebro y diferentes modalidades de imágenes se puede mejorar la precisión de la clasificación. De este modo, se han utilizado volúmenes regionales y globales (multiscale) de Materia Gris (GM), volúmenes regionales de Materia Blanca (WM), Coeficientes de asimetría e información de MRI T1 cuantitativa (multivariate). Nuestras precisiones cuando comparamos CTL vs EA y CTL vs MCI usando bases de datos públicas (ADNI) son comparables a los resultados de la literatura: 88.3% y 81.8% respectivamente. En este proyecto también estudiamos una base de datos más pequeña de pacientes con MCI del Lausanne University Hospital. Prestamos especial atención al estudio de los pasos de pre-procesado: normalización por Volumen InterCraneal y corrección de edad. Los resultados obtenidos muestran que, para nuestro grupo reducido de pacientes, se obtienen precisiones mejores cuando se combinan diferentes tipos de datos (multiscale y multivariate) que cuando solamente se usan los clásicos volúmenes regionales de GM. Además, el nuevo método propuesto de corrección de edad basado en regiones presenta resultados esperanzadores cuando se aplica previo a ambas clasificaciones CTL vsMCI y CTL vs EA.[CATALÀ] La Malaltia d'Alzheimer (MA) és la forma més comuna de demència i ha esdevingut un problema socioeconòmic creixent. A més, es preveu que l'impacte de la malaltia serà encara més gran d'aquí a uns anys a causa del progressiu envelliment de la població mundial i al creixement de l'esperança de vida. És per aquestes raons que en els últims anys s'ha centrat l'atenció en tècniques computaritzades per la diagnosi que estan dirigides a l'estudi quantitatiu d'imatges de ressonància magnètica (MRI) del cervell de pacients que es troben en una etapa primerenca de la malaltia. Un diagnòstic precoç podria millorar l'efectivitat dels futurs tractaments de curació o modificació del curs natural de la malaltia. Un exemple d'etapa primerenca de MA és el Deteriorament Cognitiu Lleuger (Mild Cognitive Impariment o MCI), ja que el 50 % dels pacients que pateixen aquesta patologia desenvolupen MA en tres o quatre anys. En aquest estudi, fem servir Support Vector Machines per classificar subjectes de tres grups diferents: MA, MCI i subjectes sans de control (CTL). El nostre objectiu és estudiar si combinant informació a diverses escales anatòmiques del cervell i diferents modalitats d'imatge es pot millorar la precisió de la classificació. D'aquesta manera, s'han utilitzat volums regionals i globals (multiscale) de Matèria Gris (GM), volums regionals de Matèria Blanca (WM), Coeficients d'asimetria i informació de MRI T1 quantitativa (Multivariate). Les nostres precisions quan comparem CTL vs MA i CTL vs MCI amb bases de dades públiques (ADNI) són comparables als resultats de la literatura: 88.3 % i 81.8 % respectivament. En aquest projecte també estudiem una base de dades més petita de pacients amb MCI del Lausanne University Hospital. Prestem especial atenció a l'estudi dels passos de preprocessament: normalització per Volum intercranial i correcció d'edat. Els resultats obtinguts mostren que, pel nostre grup reduït de pacients, s'obtenen millors precisions quan es combinen diferents tipus de dades (multiscale iMultivariate) que quan només s'usen els clàssics volums regionals de GM. A més, el nou mètode proposat de correcció d'edat basat en regions presenta resultats esperançadors quan s'aplica previ a la classificació CTL vsMCI i CTL vs MA

Fuji-SfM dataset: A collection of annotated images and point clouds for Fuji apple detection and location using structure-from-motion photogrammetry

The present dataset contains colour images acquired in a commercial Fuji apple orchard (Malus domestica Borkh. cv. Fuji) to reconstruct the 3D model of 11 trees by using structure-from-motion (SfM) photogrammetry. The data provided in this article is related to the research article entitled “Fruit detection and 3D location using instance segmentation neural networks and structure-from-motion photogrammetry” [1]. The Fuji-SfM dataset includes: (1) a set of 288 colour images and the corresponding annotations (apples segmentation masks) for training instance segmentation neural networks such as Mask-RCNN; (2) a set of 582 images defining a motion sequence of the scene which was used to generate the 3D model of 11 Fuji apple trees containing 1455 apples by using SfM; (3) the 3D point cloud of the scanned scene with the corresponding apple positions ground truth in global coordinates. With that, this is the first dataset for fruit detection containing images acquired in a motion sequence to build the 3D model of the scanned trees with SfM and including the corresponding 2D and 3D apple location annotations. This data allows the development, training, and test of fruit detection algorithms either based on RGB images, on coloured point clouds or on the combination of both types of data. Dades primàries associades a l'article http://hdl.handle.net/10459.1/68505This work was partly funded by the Secretaria d'Universitats i Recerca del Departament d'Empresa i Coneixement de la Generalitat de Catalunya (grant 2017 SGR 646), the Spanish Ministry of Economy and Competitiveness (project AGL2013-48297-C2-2-R) and the Spanish Ministry of Science, Innovation and Universities (project RTI2018-094222-B-I00). Part of the work was also developed within the framework of the project TEC2016-75976-R, financed by the Spanish Ministry of Economy, Industry and Competitiveness and the European Regional Development Fund (ERDF). The Spanish Ministry of Education is thanked for Mr. J. Gené’s pre-doctoral fellowships (FPU15/03355)

Simultaneous fruit detection and size estimation using multitask deep neural networks

The measurement of fruit size is of great interest to estimate the yield and predict the harvest resources in advance. This work proposes a novel technique for in-field apple detection and measurement based on Deep Neural Networks. The proposed framework was trained with RGB-D data and consists of an end-to-end multitask Deep Neural Network architecture specifically designed to perform the following tasks: 1) detection and segmentation of each fruit from its surroundings; 2) estimation of the diameter of each detected fruit. The methodology was tested with a total of 15,335 annotated apples at different growth stages, with diameters varying from 27 mm to 95 mm. Fruit detection results reported an F1-score for apple detection of 0.88 and a mean absolute error of diameter estimation of 5.64 mm. These are state-of-the-art results with the additional advantages of: a) using an end-to-end multitask trainable network; b) an efficient and fast inference speed; and c) being based on RGB-D data which can be acquired with affordable depth cameras. On the contrary, the main disadvantage is the need of annotating a large amount of data with fruit masks and diameter ground truth to train the model. Finally, a fruit visibility analysis showed an improvement in the prediction when limiting the measurement to apples above 65% of visibility (mean absolute error of 5.09 mm). This suggests that future works should develop a method for automatically identifying the most visible apples and discard the prediction of highly occluded fruits.This work was partly funded by the Departament de Recerca i Universitats de la Generalitat de Catalunya (grant 2021 LLAV 00088), the Spanish Ministry of Science, Innovation and Universities (grants RTI2018-094222-B-I00[PAgFRUIT project], PID2021-126648OB-I00 [PAgPROTECT project] and PID2020-117142 GB-I00 [DeeLight project] by MCIN/AEI/10.13039/501100011033 and by “ERDF, a way of making Europe”, by the European Union). The work of Jordi Gené Mola was supported by the Spanish Ministry of Universities through a Margarita Salas postdoctoral grant funded by the European Union - NextGenerationEU.info:eu-repo/semantics/publishedVersio

KFuji RGB-DS database: Fuji apple multi-modal images for fruit detection with color, depth and range-corrected IR data

This article contains data related to the research article entitle 'Multi-modal Deep Learning for Fruit Detection Using RGB-D Cameras and their Radiometric Capabilities' [1]. The development of reliable fruit detection and localization systems is essential for future sustainable agronomic management of high-value crops. RGB-D sensors have shown potential for fruit detection and localization since they provide 3D information with color data. However, the lack of substantial datasets is a barrier for exploiting the use of these sensors. This article presents the KFuji RGBDS database which is composed by 967 multi-modal images of Fuji apples on trees captured using Microsoft Kinect v2 (Microsoft, Redmond, WA, USA). Each image contains information from 3 different modalities: color (RGB), depth (D) and range corrected IR intensity (S). Ground truth fruit locations were manually annotated, labeling a total of 12,839 apples in all the dataset. The current dataset is publicly available at http://www.grap.udl.cat/publicacions/datasets.html.This work was partly funded by the Secretaria d’Universitats i Recerca del Departament d’Empresa i Coneixement de la Generalitat de Catalunya, the Spanish Ministry of Economy and Competitiveness and the European Regional Development Fund (ERDF) under Grants 2017 SGR 646, AGL2013-48297-C2-2-R and MALEGRA, TEC2016-75976-R. The Spanish Ministry of Education is thanked for Mr. J. Gené’s pre-doctoral fellowships (FPU15/03355). We would also like to thank Nufri and Vicens Maquinària Agrícola S.A. for their support during data acquisition

Multi-modal deep learning for Fuji apple detection using RGB-D cameras and their radiometric capabilities

Fruit detection and localization will be essential for future agronomic management of fruit crops, with applications in yield prediction, yield mapping and automated harvesting. RGB-D cameras are promising sensors for fruit detection given that they provide geometrical information with color data. Some of these sensors work on the principle of time-of-flight (ToF) and, besides color and depth, provide the backscatter signal intensity. However, this radiometric capability has not been exploited for fruit detection applications. This work presents the KFuji RGB-DS database, composed of 967 multi-modal images containing a total of 12,839 Fuji apples. Compilation of the database allowed a study of the usefulness of fusing RGB-D and radiometric information obtained with Kinect v2 for fruit detection. To do so, the signal intensity was range corrected to overcome signal attenuation, obtaining an image that was proportional to the reflectance of the scene. A registration between RGB, depth and intensity images was then carried out. The Faster R-CNN model was adapted for use with five-channel input images: color (RGB), depth (D) and range-corrected intensity signal (S). Results show an improvement of 4.46% in F1-score when adding depth and range-corrected intensity channels, obtaining an F1-score of 0.898 and an AP of 94.8% when all channels are used. From our experimental results, it can be concluded that the radiometric capabilities of ToF sensors give valuable information for fruit detection.This work was partly funded by the Secretaria d’Universitats i Recerca del Departament d’Empresa i Coneixement de la Generalitat de Catalunya, the Spanish Ministry of Economy and Competitiveness and the European Regional Development Fund (ERDF) under Grants 2017SGR 646, AGL2013-48297-C2-2-R and MALEGRA, TEC2016-75976-R. The Spanish Ministry of Education is thanked for Mr. J. Gené’s predoctoral fellowships (FPU15/03355). We would also like to thank Nufri and Vicens Maquinària Agrícola S.A. for their support during data acquisition, and Adria Carbó for his assistance in Faster R-CNN implementation

Looking behind occlusions: A study on amodal segmentation for robust on-tree apple fruit size estimation

The detection and sizing of fruits with computer vision methods is of interest because it provides relevant information to improve the management of orchard farming. However, the presence of partially occluded fruits limits the performance of existing methods, making reliable fruit sizing a challenging task. While previous fruit segmentation works limit segmentation to the visible region of fruits (known as modal segmentation), in this work we propose an amodal segmentation algorithm to predict the complete shape, which includes its visible and occluded regions. To do so, an end-to-end convolutional neural network (CNN) for simultaneous modal and amodal instance segmentation was implemented. The predicted amodal masks were used to estimate the fruit diameters in pixels. Modal masks were used to identify the visible region and measure the distance between the apples and the camera using the depth image. Finally, the fruit diameters in millimetres (mm) were computed by applying the pinhole camera model. The method was developed with a Fuji apple dataset consisting of 3925 RGB-D images acquired at different growth stages with a total of 15,335 annotated apples, and was subsequently tested in a case study to measure the diameter of Elstar apples at different growth stages. Fruit detection results showed an F1-score of 0.86 and the fruit diameter results reported a mean absolute error (MAE) of 4.5 mm and R2 = 0.80 irrespective of fruit visibility. Besides the diameter estimation, modal and amodal masks were used to automatically determine the percentage of visibility of measured apples. This feature was used as a confidence value, improving the diameter estimation to MAE = 2.93 mm and R2 = 0.91 when limiting the size estimation to fruits detected with a visibility higher than 60%. The main advantages of the present methodology are its robustness for measuring partially occluded fruits and the capability to determine the visibility percentage. The main limitation is that depth images were generated by means of photogrammetry methods, which limits the efficiency of data acquisition. To overcome this limitation, future works should consider the use of commercial RGB-D sensors. The code and the dataset used to evaluate the method have been made publicly available at https://github.com/GRAP-UdL-AT/Amodal_Fruit_SizingThis work was partly funded by the Departament de Recerca i Universitats de la Generalitat de Catalunya (grant 2021 LLAV 00088), the Spanish Ministry of Science, Innovation and Universities (grants RTI2018-094222-B-I00 [PAgFRUIT project], PID2021-126648OB-I00 [PAgPROTECT project] and PID2020-117142GB-I00 [DeeLight project] by MCIN/AEI/10.13039/501100011033 and by “ERDF, a way of making Europe”, by the European Union). The work of Jordi Gené Mola was supported by the Spanish Ministry of Universities through a Margarita Salas postdoctoral grant funded by the European Union - NextGenerationEU. We would also like to thank Nufri (especially Santiago Salamero and Oriol Morreres) for their support during data acquisition, and Pieter van Dalfsen and Dirk de Hoog from Wageningen University & Research for additional data collection used in the case study.info:eu-repo/semantics/publishedVersio

Uso de redes neuronales convolucionales para la detección remota de frutos con cámaras RGB-D

La detección remota de frutos será una herramienta indispensable para la gestión agronómica optimizada y sostenible de las plantaciones frutícolas del futuro, con aplicaciones en previsión de cosecha, robotización de la recolección y elaboración de mapas de producción. Este trabajo propone el uso de cámaras de profundidad RGB-D para la detección y la posterior localización 3D de los frutos. El material utilizado para la adquisición de datos consiste en una plataforma terrestre autopropulsada equipada con dos sensores Kinect v2 de Microsoft y un sistema de posicionamiento RTK-GNSS, ambos conectados a un ordenador de campo que se comunica con los sensores mediante un software desarrollado ad-hoc. Con este equipo se escanearon 3 filas de manzanos Fuji de una explotación comercial. El conjunto de datos adquiridos está compuesto por 110 capturas que contienen un total de 12,838 manzanas Fuji. La detección de frutos se realizó mediante los datos RGB (imágenes de color proporcionadas por el sensor). Para ello, se implementó y se entrenó una red neuronal convolucional de detección de objetos Faster R-CNN. Los datos de profundidad (imagen de profundidad proporcionada por el sensor) se utilizaron para generar las nubes de puntos 3D, mientras que los datos de posición permitieron georreferenciar cada captura. Los resultados de test muestran un porcentaje de detección del 91.4% de los frutos con un 15.9% de falsos positivos (F1-score = 0.876). La evaluación cualitativa de las detecciones muestra que los falsos positivos corresponden a zonas de la imagen que presentan un patrón muy similar a una manzana, donde, incluso a percepción del ojo humano, es difícil de determinar si existe o no manzana. Por otro lado, las manzanas no detectadas corresponden a aquellas que estaban ocultas casi en su totalidad por otros órganos vegetativos (hojas o ramas), a manzanas cortadas por los márgenes de la imagen, o bien a errores humanos en el proceso de etiquetaje del dataset. El tiempo de computación medio fue de 17.3 imágenes por segundo, lo que permite su aplicación en tiempo real. De los resultados experimentales se concluye que el sensor Kinect v2 tiene un gran potencial para la detección y localización 3D de frutos. La principal limitación del sistema es que el rendimiento del sensor de profundidad se ve afectado en condiciones de alta iluminación. Palabras clave: Cámaras de profundidad, RGB-D, Detección de frutos, Redes neuronales convolucionales, Robótica agrícol

Challenges associated with biomarker-based classification systems for Alzheimer's disease

Altres ajuts: This work was also supported by research grants from the Carlos III Institute of Health, Spain and the CIBERNED program (Program 1, Alzheimer Disease to Alberto Lleó and SIGNAL study, www.signalstudy.es), partly funded by Fondo Europeo de Desarrollo Regional (FEDER), Unión Europea, "Una manera de hacer Europa". This work has also been supported by a "Marató TV3" grant (20141210 to Juan Fortea and 044412 to Rafael Blesa) and by Generalitat de Catalunya and a grant from the Fundació Bancaria La Caixa to Rafael Blesa. I. Illán-Gala is supported by the i-PFIS grant from the FIS, Instituto de Salud Carlos III and the Rio Hortega grant (CM17/00074) from "Acción estratégica en Salud 2013-2016" and the European Social Fund. USPHS NIH grants awarded to M.J.d.L. include: AG13616, AG022374, AG12101, and AG057570.We aimed to evaluate the consistency of the A/T/N classification system. We included healthy controls, mild cognitive impairment, and dementia patients from Alzheimer's disease Neuroimaging Initiative. We assessed subject classification consistency with different biomarker combinations and the agreement and correlation between biomarkers. Subject classification discordance ranged from 12.2% to 44.5% in the whole sample; 17.3%-46.4% in healthy controls; 11.9%-46.5% in mild cognitive impairment, and 1%-35.7% in dementia patients. Amyloid, but not neurodegeneration biomarkers, showed good agreement both in the whole sample and in the clinical subgroups. Amyloid biomarkers were correlated in the whole sample, but not along the Alzheimer's disease continuum (as defined by a positive amyloid positron emission tomography). Neurodegeneration biomarkers were poorly correlated both in the whole sample and along the Alzheimer's disease continuum. The relationship between biomarkers was stage-dependent. Our findings suggest that the current A/T/N classification system does not achieve the required consistency to be used in the clinical setting