Improving time efficiency of feedforward neural network learning

Feedforward neural networks have been widely studied and used in many applications in science and engineering. The training of this type of networks is mainly undertaken using the well-known backpropagation based learning algorithms. One major problem with this type of algorithms is the slow training convergence speed, which hinders their applications. In order to improve the training convergence speed of this type of algorithms, many researchers have developed different improvements and enhancements. However, the slow convergence problem has not been fully addressed. This thesis makes several contributions by proposing new backpropagation learning algorithms based on the terminal attractor concept to improve the existing backpropagation learning algorithms such as the gradient descent and Levenberg-Marquardt algorithms. These new algorithms enable fast convergence both at a distance from and in a close range of the ideal weights. In particular, a new fast convergence mechanism is proposed which is based on the fast terminal attractor concept. Comprehensive simulation studies are undertaken to demonstrate the effectiveness of the proposed backpropagataion algorithms with terminal attractors. Finally, three practical application cases of time series forecasting, character recognition and image interpolation are chosen to show the practicality and usefulness of the proposed learning algorithms with comprehensive comparative studies with existing algorithms

A Face Detection and Recognition System for Color Images using Neural Networks with Boosting and Deep Learning

A face detection and recognition system is a biometric identification mechanism which compared to other methods such as finger print identification, speech, signature, hand written and iris recognition, is shown to be more important both theoretically and practically. In principle, the biometric identification methods use a wide range of techniques such as machine learning, computer vision, image processing, pattern recognition and neural networks. The methods have various applications such as in photo and film processing, control access networks, etc. In recent years, the automatic recognition of a human face has become an important problem in pattern recognition. The main reasons are structural similarity of human faces and great impact of illumination conditions, facial expression and face orientation. Face recognition is considered one of the most challenging problems in pattern recognition. A face recognition system consists of two main components, face detection and recognition. In this dissertation a face detection and recognition system using color images with multiple faces is designed, implemented, and evaluated. In color images, the information of skin color is used in order to distinguish between the skin pixels and non-skin pixels, dividing the image into several components. Neural networks and deep learning methods has been used in order to detect skin pixels in the image. A skin database has been built that contains skin pixels from different human skin colors. Information from different color spaces has been used and applied to neural networks. In order to improve system performance, bootstrapping and parallel neural networks with voting have been used. Deep learning has been used as another method for skin detection and compared to other methods. Experiments have shown that in the case of skin detection, deep learning and neural networks methods produce better results in terms of precision and recall compared to the other methods in this field. The step after skin detection is to decide which of these components belong to human face. A template based method has been modified in order to detect the faces. The template is rotated and rescaled to match the component and then the correlation between the template and component is calculated, to determine if the component belongs to a human face. The designed algorithm also succeeds if there are more than one face in the component. A rule based method has been designed in order to detect the eyes and lips in the detected components. After detecting the location of eyes and lips in the component, the face can be detected. After face detection, the faces which were detected in the previous step are to be recognized. Appearance based methods used in this work are one of the most important methods in face recognition due to the robustness of the algorithms to head rotation in the images, noise, low quality images, and other challenges. Different appearance based methods have been designed, implemented and tested. Canonical correlation analysis has been used in order to increase the recognition rate