34 research outputs found
Analyzing First-Person Stories Based on Socializing, Eating and Sedentary Patterns
First-person stories can be analyzed by means of egocentric pictures acquired
throughout the whole active day with wearable cameras. This manuscript presents
an egocentric dataset with more than 45,000 pictures from four people in
different environments such as working or studying. All the images were
manually labeled to identify three patterns of interest regarding people's
lifestyle: socializing, eating and sedentary. Additionally, two different
approaches are proposed to classify egocentric images into one of the 12 target
categories defined to characterize these three patterns. The approaches are
based on machine learning and deep learning techniques, including traditional
classifiers and state-of-art convolutional neural networks. The experimental
results obtained when applying these methods to the egocentric dataset
demonstrated their adequacy for the problem at hand.Comment: Accepted at First International Workshop on Social Signal Processing
and Beyond, 19th International Conference on Image Analysis and Processing
(ICIAP), September 201
Feature Selection for Big Visual Data: Overview and Challenges
International Conference Image Analysis and Recognition (ICIAR 2018, Póvoa de Varzim, Portugal
Playing to distraction: towards a robust training of CNN classifiers through visual explanation techniques
The field of deep learning is evolving in different directions, with still
the need for more efficient training strategies. In this work, we present a
novel and robust training scheme that integrates visual explanation techniques
in the learning process. Unlike the attention mechanisms that focus on the
relevant parts of images, we aim to improve the robustness of the model by
making it pay attention to other regions as well. Broadly speaking, the idea is
to distract the classifier in the learning process to force it to focus not
only on relevant regions but also on those that, a priori, are not so
informative for the discrimination of the class. We tested the proposed
approach by embedding it into the learning process of a convolutional neural
network for the analysis and classification of two well-known datasets, namely
Stanford cars and FGVC-Aircraft. Furthermore, we evaluated our model on a
real-case scenario for the classification of egocentric images, allowing us to
obtain relevant information about peoples' lifestyles. In particular, we work
on the challenging EgoFoodPlaces dataset, achieving state-of-the-art results
with a lower level of complexity. The obtained results indicate the suitability
of our proposed training scheme for image classification, improving the
robustness of the final model.Comment: 20 pages,3 figures, 4 table
Advancing the diagnosis of dry eye syndrome : development of automated assessments of tear film lipid layer patterns
[Resumen] El síndrome de ojo seco es una enfermedad sintomática que afecta a un amplio rango de la población, y tiene un impacto negativo en sus actividades diarias. Su diagnóstico es una tarea difícil debido a su etiología multifactorial, y por eso existen
varias pruebas clínicas. Una de esas pruebas es la evaluación de los patrones interferenciales
de la capa lipídica de la película lagrimal. Guillon dise˜nó un instrumento
denominado Tearscope Plus para evaluar el grosor de la película lagrimal de forma
rápida, y también definió una escala de clasificación compuesta de cinco categorías.
La clasificación en uno de esos cinco patrones es una tarea clínica dificil, especialmente con las capas lipídicas más finas que carecen de características de color y/o
morfológicas. Además, la interpretación subjetiva de los expertos mediante una
revisión visual puede afectar a la clasificación, pudiendo producirse un alto grado
de inter- e intra- variabilidad entre observadores. El desarrollo de un método sistemático y objetivo para análisis y clasificación es altamente deseable, permitiendo
un diagnóstico homogéneo y liberando a los expertos de esta tediosa tarea.
La propuesta de esta investigación es el diseño de un sistema automático para
evaluar los patrones de la capa lipídica de la película lagrimal mediante la interpretación de las imágenes obtenidas con el Tearscope Plus. Por una parte, se presenta
una metodología global para evaluar la capa lipídica de la película lagrimal
mediante la clasificación automática de estas imágenes en una de las categorías de
Guillon. El proceso se lleva a cabo mediante el uso de modelos de textura y color, y
algoritmos de aprendizaje máquina. A continuación, esta metodología global se optimiza
mediante la reducción de su complejidad computacional. Se utilizan técnicas
de reducción de la dimensión para disminuir los requisitos de memoria/tiempo sin
una degradación en su rendimiento. Por otra parte, se presenta una metodología
local para crear mapas de la película lagrimal, que representan la distribución local
de los patrones de la capa lipídica sobre la película lagrimal. Las diferentes evaluaciones
automáticas que se proponen ahorran tiempo a los expertos, y proporcionan
resultados imparciales que no están afectados por factores subjetivos.[Resumo] O síndrome de ollo seco é unha enfermidade sintomática que afecta a un amplo
rango da poboación, e ten un impacto negativo nas súas actividades diarias. O
seu diagnóstico é unha tarefa difícil debido á súa etioloxía multifactorial, e por
iso existen varias probas clínicas. Unha desas probas é a avaliación dos patróns
interferenciais da capa lipídica da película lagrimal. Guillon dese˜nou un instrumento
denominado Tearscope Plus para avaliar o grosor da película lagrimal de forma
rápida, e tamén definiu unha escala de clasificación composta de cinco categorías. A
clasificación nun deses cinco patróns é unha tarefa clínica difícil, especialmente coas
capas lipídicas máis finas que carecen de características de cor e/ou morfolóxicas.
Ademais, a interpretación subxectiva dos expertos mediante una revisión visual pode
afectar á clasificación, podendo producirse un alto grao de inter- e intra- variabilidade
entre observadores. O desenvolvemento dun método sistemático e obxectivo para
análise e clasificación é altamente desexable, permitindo un diagnóstico homoxéneo
e liberando aos expertos desta tediosa tarefa.
A proposta desta investigación é o deseño dun sistema automático para avaliar os
patróns da capa lipídica da película lagrimal mediante a interpretación das imaxes
obtidas co Tearscope Plus. Por unha parte, preséntase unha metodoloxía global
para avaliar a capa lipídica da película lagrimal mediante a clasificación automática
destas imaxes nunha das categorías de Guillon. O proceso é levado a cabo mediante
o uso de modelos de textura e cor, e algoritmos de aprendizaxe máquina.
A continuación, esta metodoloxía global é optimizada mediante a redución da súa
complexidade computacional. Utilízanse técnicas de redución da dimensión para
diminuír os requisitos de memoria/tempo sen unha degradación no seu rendemento.
Por outra parte, preséntase unha metodoloxía local para crear mapas da película
lagrimal, que representan a distribución local dos patróns da capa lipídica sobre a
película lagrimal. As diferentes avaliacións automáticas que se propoñen aforran
tempo aos expertos, e proporcionan resultados imparciais que non están afectados
por factores subxectivos.[Abstract] Dry eye syndrome is a symptomatic disease which affects a wide range of population,
and has a negative impact on their daily activities. Its diagnosis is a difficult task
due to its multifactorial etiology, and so there exist several clinical tests. One of
these tests is the evaluation of the interference patterns of the tear film lipid layer.
Guillon designed an instrument known as Tearscope Plus which allows clinicians to
rapidly assess the lipid layer thickness, and also defined a grading scale composed
of five categories. The classification into these five patterns is a difficult clinical
task, especially with thinner lipid layers which lack color and/or morphological features.
Furthermore, the subjective interpretation of the experts via visual inspection
may affect the classification, and so a high degree of inter- and also intra- observer
variability can be produced. The development of a systematic, objective computerized
method for analysis and classification is thus highly desirable, allowing for
homogeneous diagnosis and relieving the experts from this tedious task.
The proposal of this research is the design of an automatic system to assess
the tear film lipid layer patterns through the interpretation of the images acquired
with the Tearscope Plus. On the one hand, a global methodology is presented to
assess the tear film lipid layer by automatically classifying these images into the
Guillon categories. The process is carried out using texture and color models, and
machine learning algorithms. Then, this global methodology is optimized through
the reduction of its computational complexity. Dimensionality reduction techniques
are used in order to diminish the memory/time requirements with no degradation
in performance. On the other hand, a local methodology is also presented to create
tear film maps, which represent the local distribution of the lipid layer patterns over
the tear film. The different automated assessments proposed save time for experts,
and provide unbiased results which are not affected by subjective factors
Feature Selection in Big Image Datasets
[Abstract]
In computer vision, current feature extraction techniques generate high dimensional data. Both convolutional neural networks and traditional approaches like keypoint detectors are used as extractors of high-level features. However, the resulting datasets have grown in the number of features, leading into long training times due to the curse of dimensionality. In this research, some feature selection methods were applied to these image features through big data technologies. Additionally, we analyzed how image resolutions may affect to extracted features and the impact of applying a selection of the most relevant features. Experimental results show that making an important reduction of the extracted features provides classification results similar to those obtained with the full set of features and, in some cases, outperforms the results achieved using broad feature vectors.This research has been financially supported in part by European Union FEDER funds, by the Spanish Ministerio de Economía y Competitividad (research project PID2019-109238GB), by the Consellería de Industria of the Xunta de Galicia (research project GRC2014/035), and by the Principado de Asturias Regional Government (research project IDI-2018-000176). CITIC as a Research Centre of the Galician University System is financed by the Consellería de Educación, Universidades e Formación Profesional (Xunta de Galicia) through the ERDF (80%), Operational Programme ERDF Galicia 2014–2020, and the remaining 20% by the Secretaria Xeral de Universidades (ref. ED431G 2019/01).Xunta de Galicia; GRC2014/035Gobierno del Principado de Asturias; IDI-2018-000176Xunta de Galicia; ED431G 2019/0
Parallel definition of tear film maps on distributed-memory clusters for the support of dry eye diagnosis
[Abstract] Background and objectives
The analysis of the interference patterns on the tear film lipid layer is a useful clinical test to diagnose dry eye syndrome. This task can be automated with a high degree of accuracy by means of the use of tear film maps. However, the time required by the existing applications to generate them prevents a wider acceptance of this method by medical experts. Multithreading has been previously successfully employed by the authors to accelerate the tear film map definition on multicore single-node machines. In this work, we propose a hybrid message-passing and multithreading parallel approach that further accelerates the generation of tear film maps by exploiting the computational capabilities of distributed-memory systems such as multicore clusters and supercomputers.
Methods
The algorithm for drawing tear film maps is parallelized using Message Passing Interface (MPI) for inter-node communications and the multithreading support available in the C++11 standard for intra-node parallelization. The original algorithm is modified to reduce the communications and increase the scalability.
Results
The hybrid method has been tested on 32 nodes of an Intel cluster (with two 12-core Haswell 2680v3 processors per node) using 50 representative images. Results show that maximum runtime is reduced from almost two minutes using the previous only-multithreaded approach to less than ten seconds using the hybrid method.
Conclusions
The hybrid MPI/multithreaded implementation can be used by medical experts to obtain tear film maps in only a few seconds, which will significantly accelerate and facilitate the diagnosis of the dry eye syndrome.Ministerio de Economía y Competitividad; TIN2013-42148-PPortugal. Fundação para a Ciência e a Tecnologia; POCI-01-0145-FEDER-006961Portugal. Fundação para a Ciência e a Tecnologia; UID/EEA/50014/2013Portugal. Fundação para a Ciência e a Tecnologia; SFRH/BPD/111177/2015
On the Effectiveness of Convolutional Autoencoders on Image-Based Personalized Recommender Systems
[Abstract]
Over the years, the success of recommender systems has become remarkable. Due to the massive arrival of options that a consumer can have at his/her reach, a collaborative environment was generated, where users from all over the world seek and share their opinions based on all types of products. Specifically, millions of images tagged with users’ tastes are available on the web. Therefore, the application of deep learning techniques to solve these types of tasks has become a key issue, and there is a growing interest in the use of images to solve them, particularly through feature extraction. This work explores the potential of using only images as sources of information for modeling users’ tastes and proposes a method to provide gastronomic recommendations based on them. To achieve this, we focus on the pre-processing and encoding of the images, proposing the use of a pre-trained convolutional autoencoder as feature extractor. We compare our method with the standard approach of using convolutional neural networks and study the effect of applying transfer learning, reflecting how it is better to use only the specific knowledge of the target domain in this case, even if fewer examples are available.This research has been financially supported in part by European Union FEDER funds, by the Spanish Ministerio de Economía y Competitividad (research project PID2019-109238GB), by the Consellería de Industria of the Xunta de Galicia (research project GRC2014/035), and by the Principado de Asturias Regional Government (research project IDI-2018-000176). CITIC as a Research Center of the Galician University System is financed by the Consellería de Educación, Universidades e Formación Profesional (Xunta de Galicia) through the ERDF (80%), Operational Programme ERDF Galicia 2014–2020 and the remaining 20% by the Secretaria Xeral de Universidades (ref. ED431G 2019/01).Xunta de Galicia; GRC2014/035Gobierno del Principado de Asturias; IDI-2018-000176Xunta de Galicia; ED431G 2019/0
An end-to-end framework for intima media measurement and atherosclerotic plaque detection in the carotid artery
Background and objectives: The detection and delineation of atherosclerotic plaque are usually manually performed by medical experts on the carotid artery. Evidence suggests that this manual process is subject to errors and has a large variability between experts, equipment, and datasets. This paper proposes a robust end-to-end framework for automatic atherosclerotic plaque detection. Methods: The proposed framework is composed of: (1) a semantic segmentation model based on U-Net, with EfficientNet as the backbone, that obtains a segmentation mask with the carotid intima-media region; and (2) a convolutional neural network designed using Bayesian optimization that simultaneously performs a regression to get the average and maximum carotid intima media thickness, and a classification to determine the presence of plaque. Results: Our approach improves the state-of-the-art in both co and bulb territories in the REGICOR database, with more than 8000 images, while providing predictions in real-time. The correlation coefficient was 0.89 in the common carotid artery and 0.74 for bulb region, and the F1 score for atherosclerotic plaque detecting was 0.60 and 0.59, respectively. The experimentation carried out includes a comparison with other fully automatic methods for carotid intima media thickness estimation found in the literature. Additionally, we present an extensive experimental study to evaluate the robustness of our proposal, as well as its suitability and efficiency compared to different versions of the framework. Conclusions: The proposed end-to-end framework significantly improves the automatic characterization of atherosclerotic plaque. The generation of the segmented mask can be helpful for practitioners since it allows them to evaluate and interpret the model's results by visual inspection. Furthermore, the proposed framework overcomes the limitations of previous research based on ad-hoc post-processing, which could lead to overestimations in the case of oblique forms of the carotid artery
Correlation between tear osmolarity and tear meniscus
Purpose. To examine the relationship between tear meniscus height (TMH) and
subjective meniscus grading (subjective TM) with tear osmolarity.
Methods. Tear osmolarity measurements (using TearLab) and digital images of the
tear meniscus were obtained in 177 consecutive patients undergoing an eye
examination at our optometry clinic (Universidad de Santiago de Compostela, Spain)
who fulfilled the study's inclusion criteria. Participants were also administered the
McMonnies and Ocular Surface Disease Index (OSDI) questionnaires for the detection
of dry eye disease.
The lower tear meniscus was videotaped by a digital camera attached to a slit lamp in
its central portion without fluorescein instillation. After the study, a masked observer
extracted an image from each video, and measured the TMH using open source
software (NIH ImageJ). Subsequently, the masked observer subjectively graded the
appearance of each meniscus. For statistical analysis, subjects were stratified by age
and by dry eye symptoms as indicated by their scores in the two questionnaires.
Results. In the whole study population, a significant relationship was observed
between osmolarity and TMH (-0.41, p<0.001) and osmolarity and subjective TM (r =
0.35, p<0.001). A cluster analysis revealed similar correlations when subjects were
stratified by age or dry eye symptoms, these correlations being more pronounced in
older and more symptomatic individuals. Objective TMH measurements and subjective
meniscus quality were also correlated (r=-0.75, p<0.001).
Conclusions. Osmolarity and both objective TMH measurements and subjective
interpretation of the meniscus showed high correlation, especially in older symptomatic
subjects.Estudio patrocinado por el Ministerio de Educación y Ciencia y el Instituto de Salud Carlos III a través del proyecto PI10/01098.S
Polyvascular Subclinical Atherosclerosis: Correlation Between Ankle Brachial Index and Carotid Atherosclerosis in a Population-Based Sample
We assessed the correlation between the biomarkers of lower limb atherosclerosis (eg, ankle-brachial index [ABI]) and of carotid atherosclerosis (eg, common carotid intima-media thickness (IMT) and presence of atherosclerotic plaque) in a population-based cohort from Girona (Northwest Spain) recruited in 2010. Ankle-brachial index and carotid ultrasound were performed in all participants. Generalized additive multivariable models were used to adjust a regression model of common carotid IMT on ABI. Logistic regression multivariable models were adjusted to assess the probability of carotid plaque in individuals with peripheral artery disease. We included 3307 individuals (54.2% women), mean age 60 years (standard deviation 11). Two patterns of association were observed between subclinical biomarkers of atherosclerosis at the lower limb and carotid artery. Ankle-brachial index and common carotid IMT showed a linear trend in men [beta coefficient (95% confidence interval) =-.068 (-.123; -.012); P = .016]. Women with peripheral artery disease presented with high risk of atherosclerotic plaque at the carotid artery [Odds ratio (95% confidence interval) = 2.61, (1.46; 4.69); P = .001]. Men showed a significant linear association between ABI levels and common carotid IMT values. Women with peripheral artery disease presented with high risk of atherosclerotic plaque at the carotid artery