76 research outputs found
Efficient Semidefinite Spectral Clustering via Lagrange Duality
We propose an efficient approach to semidefinite spectral clustering (SSC),
which addresses the Frobenius normalization with the positive semidefinite
(p.s.d.) constraint for spectral clustering. Compared with the original
Frobenius norm approximation based algorithm, the proposed algorithm can more
accurately find the closest doubly stochastic approximation to the affinity
matrix by considering the p.s.d. constraint. In this paper, SSC is formulated
as a semidefinite programming (SDP) problem. In order to solve the high
computational complexity of SDP, we present a dual algorithm based on the
Lagrange dual formalization. Two versions of the proposed algorithm are
proffered: one with less memory usage and the other with faster convergence
rate. The proposed algorithm has much lower time complexity than that of the
standard interior-point based SDP solvers. Experimental results on both UCI
data sets and real-world image data sets demonstrate that 1) compared with the
state-of-the-art spectral clustering methods, the proposed algorithm achieves
better clustering performance; and 2) our algorithm is much more efficient and
can solve larger-scale SSC problems than those standard interior-point SDP
solvers.Comment: 13 page
Biconditional Generative Adversarial Networks for Multiview Learning with Missing Views
In this paper, we present a conditional GAN with two generators and a common
discriminator for multiview learning problems where observations have two
views, but one of them may be missing for some of the training samples. This is
for example the case for multilingual collections where documents are not
available in all languages. Some studies tackled this problem by assuming the
existence of view generation functions to approximately complete the missing
views; for example Machine Translation to translate documents into the missing
languages. These functions generally require an external resource to be set and
their quality has a direct impact on the performance of the learned multiview
classifier over the completed training set. Our proposed approach addresses
this problem by jointly learning the missing views and the multiview classifier
using a tripartite game with two generators and a discriminator. Each of the
generators is associated to one of the views and tries to fool the
discriminator by generating the other missing view conditionally on the
corresponding observed view. The discriminator then tries to identify if for an
observation, one of its views is completed by one of the generators or if both
views are completed along with its class. Our results on a subset of Reuters
RCV1/RCV2 collections show that the discriminator achieves significant
classification performance; and that the generators learn the missing views
with high quality without the need of any consequent external resource.Comment: 15 pages, 3 figur
Rank-Similarity Measures for Comparing Gene Prioritizations: A Case Study in Autism
We discuss the challenge of comparing three gene prioritization methods: network propagation, integer linear programming rank aggregation (RA), and statistical RA. These methods are based on different biological categories and estimate disease?gene association. Previously proposed comparison schemes are based on three measures of performance: receiver operating curve, area under the curve, and median rank ratio. Although they may capture important aspects of gene prioritization performance, they may fail to capture important differences in the rankings of individual genes. We suggest that comparison schemes could be improved by also considering recently proposed measures of similarity between gene rankings. We tested this suggestion on comparison schemes for prioritizations of genes associated with autism that were obtained using brain- and tissue-specific data. Our results show the effectiveness of our measures of similarity in clustering brain regions based on their relevance to autism
Tiny Machine Learning Environment: Enabling Intelligence on Constrained Devices
Running machine learning algorithms (ML) on constrained devices at the extreme edge of the network is problematic due to the computational overhead of ML algorithms, available resources on the embedded platform, and application budget (i.e., real-time requirements, power constraints, etc.). This required the development of specific solutions and development tools for what is now referred to as TinyML. In this dissertation, we focus on improving the deployment and performance of TinyML applications, taking into consideration the aforementioned challenges, especially memory requirements.
This dissertation contributed to the construction of the Edge Learning Machine environment (ELM), a platform-independent open-source framework that provides three main TinyML services, namely shallow ML, self-supervised ML, and binary deep learning on constrained devices. In this context, this work includes the following steps, which are reflected in the thesis structure. First, we present the performance analysis of state-of-the-art shallow ML algorithms including dense neural networks, implemented on mainstream microcontrollers. The comprehensive analysis in terms of algorithms, hardware platforms, datasets, preprocessing techniques, and configurations shows similar performance results compared to a desktop machine and highlights the impact of these factors on overall performance. Second, despite the assumption that TinyML only permits models inference provided by the scarcity of resources, we have gone a step further and enabled self-supervised on-device training on microcontrollers and tiny IoT devices by developing the Autonomous Edge Pipeline (AEP) system. AEP achieves comparable accuracy compared to the typical TinyML paradigm, i.e., models trained on resource-abundant devices and then deployed on microcontrollers. Next, we present the development of a memory allocation strategy for convolutional neural networks (CNNs) layers, that optimizes memory requirements. This approach reduces the memory footprint without affecting accuracy nor latency. Moreover, e-skin systems share the main requirements of the TinyML fields: enabling intelligence with low memory, low power consumption, and low latency. Therefore, we designed an efficient Tiny CNN architecture for e-skin applications. The architecture leverages the memory allocation strategy presented earlier and provides better performance than existing solutions. A major contribution of the thesis is given by CBin-NN, a library of functions for implementing extremely efficient binary neural networks on constrained devices. The library outperforms state of the art NN deployment solutions by drastically reducing memory footprint and inference latency. All the solutions proposed in this thesis have been implemented on representative devices and tested in relevant applications, of which results are reported and discussed. The ELM framework is open source, and this work is clearly becoming a useful, versatile toolkit for the IoT and TinyML research and development community
Simple but Not Simplistic: Reducing the Complexity of Machine Learning Methods
Programa Oficial de Doutoramento en Computación . 5009V01[Resumo]
A chegada do Big Data e a explosión do Internet das cousas supuxeron un gran
reto para os investigadores en Aprendizaxe Automática, facendo que o proceso de
aprendizaxe sexa mesmo roáis complexo. No mundo real, os problemas da aprendizaxe
automática xeralmente teñen complexidades inherentes, como poden ser as
características intrínsecas dos datos, o gran número de mostras, a alta dimensión dos
datos de entrada, os cambios na distribución entre o conxunto de adestramento e
test, etc. Todos estes aspectos son importantes, e requiren novoS modelos que poi dan
facer fronte a estas situacións. Nesta tese, abordáronse todos estes problemas, tratando
de simplificar o proceso de aprendizaxe automática no escenario actual. En
primeiro lugar, realízase unha análise de complexidade para observar como inflúe
esta na tarefa de clasificación, e se é posible que a aplicación dun proceso previo
de selección de características reduza esta complexidade. Logo, abórdase o proceso
de simplificación da fase de aprendizaxe automática mediante a filosofía divide e
vencerás, usando un enfoque distribuído. Seguidamente, aplicamos esa mesma filosofía
sobre o proceso de selección de características. Finalmente, optamos por un
enfoque diferente seguindo a filosofía do Edge Computing, a cal permite que os datos
producidos polos dispositivos do Internet das cousas se procesen máis preto de
onde se crearon. Os enfoques propostos demostraron a súa capacidade para reducir
a complexidade dos métodos de aprendizaxe automática tradicionais e, polo tanto,
espérase que a contribución desta tese abra as portas ao desenvolvemento de novos
métodos de aprendizaxe máquina máis simples, máis robustos, e máis eficientes
computacionalmente.[Resumen]
La llegada del Big Data y la explosión del Internet de las cosas han supuesto
un gran reto para los investigadores en Aprendizaje Automático, haciendo que el
proceso de aprendizaje sea incluso más complejo. En el mundo real, los problemas de
aprendizaje automático generalmente tienen complejidades inherentes) como pueden
ser las características intrínsecas de los datos, el gran número de muestras, la alta
dimensión de los datos de entrada, los cambios en la distribución entre el conjunto de
entrenamiento y test, etc. Todos estos aspectos son importantes, y requieren nuevos
modelos que puedan hacer frente a estas situaciones. En esta tesis, se han abordado
todos estos problemas, tratando de simplificar el proceso de aprendizaje automático
en el escenario actual. En primer lugar, se realiza un análisis de complejidad para
observar cómo influye ésta en la tarea de clasificación1 y si es posible que la aplicación
de un proceso previo de selección de características reduzca esta complejidad.
Luego, se aborda el proceso de simplificación de la fase de aprendizaje automático
mediante la filosofía divide y vencerás, usando un enfoque distribuido. A continuación,
aplicamos esa misma filosofía sobre el proceso de selección de características.
Finalmente, optamos por un enfoque diferente siguiendo la filosofía del Edge Computing,
la cual permite que los datos producidos por los dispositivos del Internet de
las cosas se procesen más cerca de donde se crearon. Los enfoques propuestos han
demostrado su capacidad para reducir la complejidad de los métodos de aprendizaje
automático tnidicionales y, por lo tanto, se espera que la contribución de esta
tesis abra las puertas al desarrollo de nuevos métodos de aprendizaje máquina más
simples, más robustos, y más eficientes computacionalmente.[Abstract]
The advent of Big Data and the explosion of the Internet of Things, has brought
unprecedented challenges to Machine Learning researchers, making the learning task
more complexo Real-world machine learning problems usually have inherent complexities,
such as the intrinsic characteristics of the data, large number of instauces,
high input dimensionality, dataset shift, etc. AH these aspects matter, and can
fOI new models that can confront these situations. Thus, in this thesis, we have
addressed aH these issues) simplifying the machine learning process in the current
scenario. First, we carry out a complexity analysis to see how it inftuences the
classification models, and if it is possible that feature selection might result in a
deerease of that eomplexity. Then, we address the proeess of simplifying learning
with the divide-and-conquer philosophy of the distributed approaeh. Later, we aim
to reduce the complexity of the feature seleetion preprocessing through the same
philosophy. FinallYl we opt for a different approaeh following the eurrent philosophy
Edge eomputing, whieh allows the data produeed by Internet of Things deviees
to be proeessed closer to where they were ereated. The proposed approaehes have
demonstrated their eapability to reduce the complexity of traditional maehine learning
algorithms, and thus it is expeeted that the eontribution of this thesis will open
the doors to the development of new maehine learning methods that are simpler,
more robust, and more eomputationally efficient
Pattern Recognition
Pattern recognition is a very wide research field. It involves factors as diverse as sensors, feature extraction, pattern classification, decision fusion, applications and others. The signals processed are commonly one, two or three dimensional, the processing is done in real- time or takes hours and days, some systems look for one narrow object class, others search huge databases for entries with at least a small amount of similarity. No single person can claim expertise across the whole field, which develops rapidly, updates its paradigms and comprehends several philosophical approaches. This book reflects this diversity by presenting a selection of recent developments within the area of pattern recognition and related fields. It covers theoretical advances in classification and feature extraction as well as application-oriented works. Authors of these 25 works present and advocate recent achievements of their research related to the field of pattern recognition
- …