Adaptive neural network control of fes-induced cyclical lower leg movements

As a first step to the control of paraplegic gait by functional electrical stimulation (FES), the control of the swinging lower leg is being studied. This paper deals with a neural control system, that has been developed for this case. The control system has been tested for a model of the swinging lower leg using computer simulations. The neural controller was trained by supervised learning (SL) and by backpropagation through time (BTT). The performance of the controller with random initial weights was poor after training with BTT and fair after SL. BTT training of the neural controller with weights, which had been initialized by SL, resulted in good control. Training with BTT thus improved the performance of the controller that initially had been trained by SL. An adaptive neural control system based on BTT has been proposed and partially tested. The controller adapted relatively fast to the change of an important model parameter

Fixed-Point Performance Analysis of Recurrent Neural Networks

Recurrent neural networks have shown excellent performance in many applications, however they require increased complexity in hardware or software based implementations. The hardware complexity can be much lowered by minimizing the word-length of weights and signals. This work analyzes the fixed-point performance of recurrent neural networks using a retrain based quantization method. The quantization sensitivity of each layer in RNNs is studied, and the overall fixed-point optimization results minimizing the capacity of weights while not sacrificing the performance are presented. A language model and a phoneme recognition examples are used

Adaptive learning rates and parallelization for stochastic, sparse, non-smooth gradients

Recent work has established an empirically successful framework for adapting learning rates for stochastic gradient descent (SGD). This effectively removes all needs for tuning, while automatically reducing learning rates over time on stationary problems, and permitting learning rates to grow appropriately in non-stationary tasks. Here, we extend the idea in three directions, addressing proper minibatch parallelization, including reweighted updates for sparse or orthogonal gradients, improving robustness on non-smooth loss functions, in the process replacing the diagonal Hessian estimation procedure that may not always be available by a robust finite-difference approximation. The final algorithm integrates all these components, has linear complexity and is hyper-parameter free.Comment: Published at the First International Conference on Learning Representations (ICLR-2013). Public reviews are available at http://openreview.net/document/c14f2204-fd66-4d91-bed4-153523694041#c14f2204-fd66-4d91-bed4-15352369404

Distributed computing methodology for training neural networks in an image-guided diagnostic application

Distributed computing is a process through which a set of computers connected by a network is used collectively to solve a single problem. In this paper, we propose a distributed computing methodology for training neural networks for the detection of lesions in colonoscopy. Our approach is based on partitioning the training set across multiple processors using a parallel virtual machine. In this way, interconnected computers of varied architectures can be used for the distributed evaluation of the error function and gradient values, and, thus, training neural networks utilizing various learning methods. The proposed methodology has large granularity and low synchronization, and has been implemented and tested. Our results indicate that the parallel virtual machine implementation of the training algorithms developed leads to considerable speedup, especially when large network architectures and training sets are used

A neural network based fall detector

In this project we present an intelligent fall detector system based on a 3-axis accelerometer and a neural network model that allows recognizing several possible motion situations and performing an emergency call only when a fall situation occurs, with low false negatives rate and low false positives rate. The system is based on a two module platform. The first one is a Mobile Station (MS) and should be carried always by the person. An accelerometer is implemented in this module and its information is transferred via a radio-frequency channel (RF) to the Base Station (BS). The BS is fixed and is connected to a GSM (Global System for Mobile communication) module. A neural network model was built into the BS and is able to identify falls from other possible motion situations, based on the received information. According to the neural network response the system sends a SMS (Short Message Service) to a destination number requesting for assistance

Neural network-based colonoscopic diagnosis using on-line learning and differential evolution

In this paper, on-line training of neural networks is investigated in the context of computer-assisted colonoscopic diagnosis. A memory-based adaptation of the learning rate for the on-line back-propagation (BP) is proposed and used to seed an on-line evolution process that applies a differential evolution (DE) strategy to (re-) adapt the neural network to modified environmental conditions. Our approach looks at on-line training from the perspective of tracking the changing location of an approximate solution of a pattern-based, and thus, dynamically changing, error function. The proposed hybrid strategy is compared with other standard training methods that have traditionally been used for training neural networks off-line. Results in interpreting colonoscopy images and frames of video sequences are promising and suggest that networks trained with this strategy detect malignant regions of interest with accuracy