Using multiple visual tandem streams in audio-visual speech recognition

The method which is called the "tandem approach" in speech recognition has been shown to increase performance by using classifier posterior probabilities as observations in a hidden Markov model. We study the effect of using visual tandem features in audio-visual speech recognition using a novel setup which uses multiple classifiers to obtain multiple visual tandem features. We adopt the approach of multi-stream hidden Markov models where visual tandem features from two different classifiers are considered as additional streams in the model. It is shown in our experiments that using multiple visual tandem features improve the recognition accuracy in various noise conditions. In addition, in order to handle asynchrony between audio and visual observations, we employ coupled hidden Markov models and obtain improved performance as compared to the synchronous model

Hydroelectric power plant management relying on neural networks and expert system integration

The use of Neural Networks (NN) is a novel approach that can help in taking decisions when integrated in a more general system, in particular with expert systems. In this paper, an architecture for the management of hydroelectric power plants is introduced. This relies on monitoring a large number of signals, representing the technical parameters of the real plant. The general architecture is composed of an Expert System and two NN modules: Acoustic Prediction (NNAP) and Predictive Maintenance (NNPM). The NNAP is based on Kohonen Learning Vector Quantization (LVQ) Networks in order to distinguish the sounds emitted by electricity-generating machine groups. The NNPM uses an ART-MAP to identify different situations from the plant state variables, in order to prevent future malfunctions. In addition, a special process to generate a complete training set has been designed for the ART-MAP module. This process has been developed to deal with the absence of data about abnormal plant situations, and is based on neural nets trained with the backpropagation algorithm.Publicad

Machine Learning in Wireless Sensor Networks: Algorithms, Strategies, and Applications

Wireless sensor networks monitor dynamic environments that change rapidly over time. This dynamic behavior is either caused by external factors or initiated by the system designers themselves. To adapt to such conditions, sensor networks often adopt machine learning techniques to eliminate the need for unnecessary redesign. Machine learning also inspires many practical solutions that maximize resource utilization and prolong the lifespan of the network. In this paper, we present an extensive literature review over the period 2002-2013 of machine learning methods that were used to address common issues in wireless sensor networks (WSNs). The advantages and disadvantages of each proposed algorithm are evaluated against the corresponding problem. We also provide a comparative guide to aid WSN designers in developing suitable machine learning solutions for their specific application challenges.Comment: Accepted for publication in IEEE Communications Surveys and Tutorial

A probabilistic, discriminative and distributed system for the recognition of human actions from multiple views

This paper presents a distributed system for the recognition of human actions using views of the scene grabbed by different cameras. 2D frame descriptors are extracted for each available view to capture the variability in human motion. These descriptors are projected into a lower dimensional space and fed into a probabilistic classifier to output a posterior distribution of the action performed according to the descriptor computed at each camera. Classifier fusion algorithms are then used to merge the posterior distributions into a single distribution. The generated single posterior distribution is fed into a sequence classifier to make the final decision on the performed activity. The system can instantiate different algorithms for the different tasks, as the interfaces between modules are clearly defined. Results on the classification of the actions in the IXMAS dataset are reported. The accuracy of the proposed system is similar to state-of-the-art 3D methods, even though it uses only well-known 2D pattern recognition techniques and does not need to project the data into a 3D space or require camera calibration parameters.This work was supported in part by Projects CICYT TIN 2008-06742-C02-02/TSI, CICYT TEC2008-06732-C02-02/TEC, CAM CONTEXTS (S2009/TIC1485) and DPS2008-07029-C02-02.publicad