Neural network-based colonoscopic diagnosis using on-line learning and differential evolution

In this paper, on-line training of neural networks is investigated in the context of computer-assisted colonoscopic diagnosis. A memory-based adaptation of the learning rate for the on-line back-propagation (BP) is proposed and used to seed an on-line evolution process that applies a differential evolution (DE) strategy to (re-) adapt the neural network to modified environmental conditions. Our approach looks at on-line training from the perspective of tracking the changing location of an approximate solution of a pattern-based, and thus, dynamically changing, error function. The proposed hybrid strategy is compared with other standard training methods that have traditionally been used for training neural networks off-line. Results in interpreting colonoscopy images and frames of video sequences are promising and suggest that networks trained with this strategy detect malignant regions of interest with accuracy

Metaheuristic design of feedforward neural networks: a review of two decades of research

Over the past two decades, the feedforward neural network (FNN) optimization has been a key interest among the researchers and practitioners of multiple disciplines. The FNN optimization is often viewed from the various perspectives: the optimization of weights, network architecture, activation nodes, learning parameters, learning environment, etc. Researchers adopted such different viewpoints mainly to improve the FNN's generalization ability. The gradient-descent algorithm such as backpropagation has been widely applied to optimize the FNNs. Its success is evident from the FNN's application to numerous real-world problems. However, due to the limitations of the gradient-based optimization methods, the metaheuristic algorithms including the evolutionary algorithms, swarm intelligence, etc., are still being widely explored by the researchers aiming to obtain generalized FNN for a given problem. This article attempts to summarize a broad spectrum of FNN optimization methodologies including conventional and metaheuristic approaches. This article also tries to connect various research directions emerged out of the FNN optimization practices, such as evolving neural network (NN), cooperative coevolution NN, complex-valued NN, deep learning, extreme learning machine, quantum NN, etc. Additionally, it provides interesting research challenges for future research to cope-up with the present information processing era

Online Learning of a Memory for Learning Rates

The promise of learning to learn for robotics rests on the hope that by extracting some information about the learning process itself we can speed up subsequent similar learning tasks. Here, we introduce a computationally efficient online meta-learning algorithm that builds and optimizes a memory model of the optimal learning rate landscape from previously observed gradient behaviors. While performing task specific optimization, this memory of learning rates predicts how to scale currently observed gradients. After applying the gradient scaling our meta-learner updates its internal memory based on the observed effect its prediction had. Our meta-learner can be combined with any gradient-based optimizer, learns on the fly and can be transferred to new optimization tasks. In our evaluations we show that our meta-learning algorithm speeds up learning of MNIST classification and a variety of learning control tasks, either in batch or online learning settings.Comment: accepted to ICRA 2018, code available: https://github.com/fmeier/online-meta-learning ; video pitch available: https://youtu.be/9PzQ25FPPO

A neural-network controlled dynamic evolutionary scheme for global molecular geometry optimization

A novel, neural network controlled, dynamic evolutionary algorithm is proposed for the purposes of molecular geometry optimization. The approach is tested for selected model molecules and some molecular systems of importance in biochemistry. The new algorithm is shown to compare favorably with the standard, statically parametrized memetic algorithm

Estimation of Travel Time using Temporal and Spatial Relationships in Sparse Data

Travel time is a basic measure upon which e.g. traveller information systems, traffic management systems, public transportation planning and other intelligent transport systems are developed. Collecting travel time information in a large and dynamic road network is essential to managing the transportation systems strategically and efficiently. This is a challenging and expensive task that requires costly travel time measurements. Estimation techniques are employed to utilise data collected for the major roads and traffic network structure to approximate travel times for minor links. Although many methodologies have been proposed, they have not yet adequately solved many challenges associated with travel time, in particular, travel time estimation for all links in a large and dynamic urban traffic network. Typically focus is placed on major roads such as motorways and main city arteries but there is an increasing need to know accurate travel times for minor urban roads. Such information is crucial for tackling air quality problems, accommodate a growing number of cars and provide accurate information for routing, e.g. self-driving vehicles. This study aims to address the aforementioned challenges by introducing a methodology able to estimate travel times in near-real-time by using historical sparse travel time data. To this end, an investigation of temporal and spatial dependencies between travel time of traffic links in the datasets is carefully conducted. Two novel methodologies are proposed, Neighbouring Link Inference method (NLIM) and Similar Model Searching method (SMS). The NLIM learns the temporal and spatial relationship between the travel time of adjacent links and uses the relation to estimate travel time of the targeted link. For this purpose, several machine learning techniques including support vector machine regression, neural network and multi-linear regression are employed. Meanwhile, SMS looks for similar NLIM models from which to utilise data in order to improve the performance of a selected NLIM model. NLIM and SMS incorporates an additional novel application for travel time outlier detection and removal. By adapting a multivariate Gaussian mixture model, an improvement in travel time estimation is achieved. Both introduced methods are evaluated on four distinct datasets and compared against benchmark techniques adopted from literature. They efficiently perform the task of travel time estimation in near-real-time of a target link using models learnt from adjacent traffic links. The training data from similar NLIM models provide more information for NLIM to learn the temporal and spatial relationship between the travel time of links to support the high variability of urban travel time and high data sparsity.Ministry of Education and Training of Vietna