Development of self-adaptive back propagation and derivative free training algorithms in artificial neural networks

Three new iterative, dynamically self-adaptive, derivative-free and training parameter free artificial neural network (ANN) training algorithms are developed. They are defined as self-adaptive back propagation, multi-directional and restart ANN training algorithms. The descent direction in self-adaptive back propagation training is determined implicitly by a central difference approximation scheme, which chooses its step size according to the convergence behavior of the error function. This approach trains an ANN when the gradient information of the corresponding error function is not readily available. The self- adaptive variable learning rates per epoch are determined dynamically using a constrained interpolation search. As a result, appropriate descent to the error function is achieved. The multi-directional training algorithm is self-adaptive and derivative free. It orients an initial search vector in a descent location at the early stage of training. Individual learning rates and momentum term for all the ANN weights are determined optimally. The search directions are derived from rectilinear and Euclidean paths, which explore stiff ridges and valleys of the error surface to improve training. The restart training algorithm is derivative free. It redefines a de-generated simplex at a re-scale phase. This multi-parameter training algorithm updates ANN weights simultaneously instead of individually. The descent directions are derived from the centroid of a simplex along a reflection point opposite to the worst vertex. The algorithm is robust and has the ability to improve local search. These ANN training methods are appropriate when there is discontinuity in corresponding ANN error function or the Hessian matrix is ill conditioned or singular. The convergence properties of the algorithms are proved where possible. All the training algorithms successfully train exclusive OR (XOR), parity, character-recognition and forecasting problems. The simulation results with XOR, parity and character recognition problems suggest that all the training algorithms improve significantly over the standard back propagation algorithm in average number of epoch, function evaluations and terminal function values. The multivariate ANN calibration problem as a regression model with small data set is relatively difficult to train. In forecasting problems, an ANN is trained to extrapolate the data in validation period. The extrapolation results are compared with the actual data. The trained ANN performs better than the statistical regression method in mean absolute deviations; mean squared errors and relative percentage error. The restart training algorithm succeeds in training a problem, where other training algorithms face difficulty. It is shown that a seasonal time series problem possesses a Hessian matrix that has a high condition number. Convergence difficulties as well as slow training are therefore not atypical. The research exploits the geometry of the error surface to identify self-adaptive optimized learning rates and momentum terms. Consequently, the algorithms converge with high success rate. These attributes brand the training algorithms as self-adaptive, automatic, parameter free, efficient and easy to use

Sliding Mode Control

The main objective of this monograph is to present a broad range of well worked out, recent application studies as well as theoretical contributions in the field of sliding mode control system analysis and design. The contributions presented here include new theoretical developments as well as successful applications of variable structure controllers primarily in the field of power electronics, electric drives and motion steering systems. They enrich the current state of the art, and motivate and encourage new ideas and solutions in the sliding mode control area

An approach to the use of neural network in the analysis of a class of flight vehicles

Thesis (M.S.)--Massachusetts Institute of Technology, Dept. of Aeronautics and Astronautics, 1998.Includes bibliographical references (p. 163-165).by Agus Budiyono.M.S

Προηγμένες υπολογιστικές μέθοδοι αντισεισμικού σχεδιασμού και αποτίμησης κατασκευών από οπλισμένο σκυρόδεμα

309 σ.Ο κύριος στόχος της διατριβής ήταν η ανάπτυξη ενός ολοκληρωμένου πλαισίου για την αξιολόγηση και τον οικονομικό-ασφαλή αντισεισμικό σχεδιασμό κατασκευών. Αυτός ο καθολικός στόχος της διατριβής επετεύχθη μέσω των ακόλουθων βημάτων: (i) Στο πρώτο μέρος της διατριβής πραγματοποιήθηκε αριθμητική βαθμονόμηση ορισμένων από τους πιο δημοφιλείς δείκτες βλάβης που έχουν προταθεί και υιοθετηθεί από πολλούς ερευνητές προκειμένου να προσδιορίσουν το επίπεδο ζημίας κατασκευών από οπλισμένο σκυρόδεμα. (ii) Στο δεύτερο στάδιο της διατριβής πραγματοποιήθηκε αξιολόγηση των περιγραφικών διαδικασιών αντισεισμικού σχεδιασμού σε σχέση με τον συντελεστή συμπεριφοράς που υιοθετείται από τους Ευρωκώδικες και διερευνήθηκε η βέλτιστη επιλογή που οδηγεί στον οικονομικότερο και ασφαλέστερο σχεδιασμό. Επιπροσθέτως έγινε σύγκριση των μεθόδων σχεδιασμού με βάση την επίδοση σε σχέση με τις περιγραφικές μεθόδους σχεδιασμού. Στη συνέχεια, με βάση τους βαθμονομημένους δείκτες βλάβης, διατυπώθηκαν προβλήματα βελτιστοποίησης με στόχο να προσδιοριστεί ο δείκτης βλάβης ή ο συνδυασμός δεικτών βλάβης που αποτελούν την σωστότερη επιλογή προκειμένου να ενσωματωθεί στο πλαίσιο σχεδιασμού που βασίζεται στην επιτελεστικότητα. (iii) Ως επόμενο βήμα της διατριβής ήταν η βελτίωση της διαδικασίας του κόστους κύκλου ζωής όσον αφορά την αξιοπιστία και την υπολογιστική αποδοτικότητά της.. (iv) Ο τελευταίος στόχος της διατριβής ήταν η βελτίωση της διαδικασίας ανάλυσης τρωτότητας σε σχέση με την αξιοπιστία και την υπολογιστική αποδοτικότητα. Η υπολογιστική αποδοτικότητα επετεύχθη μέσω της προτεινόμενης στο πλαίσιο της παρούσης διατριβής προσαυξητικής δυναμικής ανάλυσης με βάση πρόβλεψης νευρωνικών δικτύων η οποία μειώνει τις υπολογιστικές απαιτήσεις κατά μία τάξη μεγέθους.The major objective of this Dissertation is to develop an integrated framework for the economical and safe antiseismic design and assessment of new reinforced concrete structures by means of life-cycle cost and fragility analysis. This objective of the dissertation is achieved through the accomplishment of the following tasks: (i) At the first part of the Dissertation numerical calibration for some of the most popular damage indices (DIs) that have been proposed by many researchers was performed, in order to quantify the extent of damage in reinforced concrete structures. (ii) A critical assessment of prescriptive design procedures was performed with reference to their ability to lead to safe and economical designs. Furthermore, a comparison between prescriptive and performance-based seismic design procedures was carried out. For this purpose a number of structural seismic design optimisation problems have been formulated. On the other hand, based on the calibrated DIs, structural optimization problems were formulated aiming at identifying the DI, or the combination of Dis that will provide reliable information on damage so that they can be incorporated into a Performance-Based Design framework. The ultimate objective of this task is to compare lower-bound designs that satisfy the design code requirements in the most cost-effective way using a Life-Cycle Cost Analysis (LCCA) methodology. (iii) The next step is to improve the LCCA procedure with reference to both its robustness and efficiency. (iv) The last objective of the dissertation is to improve the fragility analysis procedure with reference to both robustness and efficiency. The efficiency is achieved by introducing a neural network-based incremental dynamic analysis (IDA) procedure that reduces the computational effort by one order of magnitude.Χαρίκλεια Χ. Μητροπούλο

An optimal nephelometric model design method for particle characterisation

Scattering nephelometry is a particle characterisation method applicable to fluid suspensions containing impurities. Solutions derived by the method feature particle classification by size (diameter), volume or texture as well as continuous on-line and in-situ monitoring, The replacement of turbidimeters with nephelometers in many existing turbidity applications could result in suppression of side effects caused by limitations and uncontrolled parameter drifts and satisfaction of problem-defined constraints at virtually no change in implementation cost. A major issue of nephelometric model design is the selection of a mathematical tool suitable for the modelling of the data analysis system. [Continues.

Object-oriented analysis and design of computational intelligence systems

Machine learning from data, neuro-fuzzy information processing, approximate reasoning and genetic and evolutionary computation are all aspects of computational intelligence (also called soft computing methods). Soft computing methods differ from conventional computing in that they are tolerant of imprecision, uncertainty and partial truths. These characteristics can be exploited to achieve tractability, robustness and low solution costs when the solution to a complex (in machine terms) problem is required. The principal constituents of soft computing include: Neural Networks, Fuzzy Logic and Probabilistic Reasoning Systems. Genetic Algorithms (GAs), Evolutionary Algorithms, Chaos Theory', Complexity Theory and parts of Learning Theory all come under Probabilistic Reasoning Systems. Hybrid systems can be designed incorporating 2 or more aspects of soft computing that are more powerful than any of the components used in a stand alone fashion. A unified framework is needed to implement and manipulate such systems. Such a framework will allow for easy visualisation of the underlying concepts and easy modification of the resulting computer models. In this thesis, an investigation of the major aspects of computational intelligence has been carried out. The main emphasis has been placed on developing an object-oriented framework for architecting computational intelligence systems. Object models for Neural Networks, Fuzzy Logic Systems and Evolutionary Computation systems have been developed. Software has been written in C++ to realise sample implementations of the various systems. Finally, practical applications and the results of using the Neural Networks, Fuzzy Logic systems and Genetic Algorithms developed in solving real world problems are presented. A consistent notation based on the Object Modelling Technique (OMT) is used throughout the thesis to describe the software architectures from which the computer implementation models have been derived

Globally convergent modification of the quickprop method

A mathematical framework for the convergence analysis of the well-known Quickprop method is described. Furthermore, we propose a modification of this method that exhibits improved convergence speed and stability, and, at the same time, alleviates the use of heuristic learning parameters. Simulations are conducted to compare and evaluate the performance of the new modified Quickprop algorithm with various popular training algorithms. The results of the experiments indicate that the increased convergence rates achieved by the proposed algorithm, affect by no means its generalization capability and stability

Klassifikation morphologischer und pathologischer Strukturen in koronaren Gefäßen auf Basis intravaskulärer Ultraschallaufnahmen zur klinischen Anwendung in einem IVB-System

Erkrankungen des Herz-Kreislaufsystems sind in Deutschland für fast 50% der Todesfälle verantwortlich. Insbesondere die Arteriosklerose (vulgo: „Arterienverkalkung“) ist dabei ein dominierendes Krankheitsbild. So ist es auch nicht verwunderlich, dass die Arteriosklerose seit den Anfängen der wissenschaftlichen Medizin ein Feld für umfangreiche Untersuchungen gewesen ist. Speziell durch den technischen Fortschritt bildgebender Verfahren war es möglich neuartige Diagnose- und Therapiemethoden zu entwickeln. Dabei hat sich gerade der intravaskuläre Ultraschall zu einem Goldstandard in der Diagnose arteriosklerotischer Erkrankungen und, in Kombination mit der intravaskulären Brachytherapie, zu einer Erfolg versprechenden Basistechnik für therapeutische Maßnahmen entwickelt. Grundvoraussetzung fast jeder bildbasierten Intervention ist aber die Separierung der Bilddaten in anatomisch und pathologisch differenzierte, saliente Regionen. In Anbetracht zunehmender, umfangreicherer Datenmengen kann eine derartige Aufarbeitung nur rechnergestützt durch Problem adaptierte Klassifikationsalgorithmen gewährleistet werden. Daher war es das Ziel dieser Arbeit, neue Methoden zur Merkmalsextraktion und Algorithmen zur Klassifikation morphologischer und pathologischer Strukturen in koronaren Gefäßen bereitzustellen. Aus der initialen Fragestellung wurde zudem zeitnah deutlich, dass das Forschungsvorhaben Anknüpfungspunkte zu weiteren hochgradig relevanten inter- und intradisziplinären Forschungsthemen, beispielsweise der Histologie, Systembiologie oder Chemietechnik, aufweist. Aber auch vonseiten der Anwendungsszenarien wurden teilweise völlig neue, innovative Wege beschritten. Exemplarisch sei ein E-Learning-Ansatz zur „Übersetzung“ digitaler Bilddaten in haptisch erfahrbare Reliefs für blinde und sehbehinderte Schülerinnen und Schüler genannt. In Anbetracht dieser partiell divergierenden Sichtweisen war auch die generalisierte, von der expliziten Fragestellung abstrahierte Umsetzung eine Ausrichtung der Arbeit. Dieser Intention folgend wurden drei wesentliche methodische und konzeptionelle Entwicklungen innerhalb der Arbeit realisiert: ein Expertensystem zur Approximation arterieller Kompartimente mittels unscharfer elliptischer Templates, ein neuartiger, effizienter Ansatz zur signaltheoretischen Extraktion textureller Merkmale und die Etablierung maschinelle Lernverfahren unter Integration von a priori Wissen. Über eine konsequente Integration statistischer Gütemaße konnte zudem eine ausgeprägte Rückkopplung zwischen Klassifikations- und Bewertungsansätzen gewährleistet werden. Gemeinsam ist allen Ansätzen das Ansinnen, trotz hoch anwendungsbezogener Umsetzungen, die fortwährende Portabilität zu beachten. In einer übergeordneten Abstraktion kann die Intention der Arbeit somit auch in der „generalisierten Nutzung signaltheoretischer Merkmale zur Klassifikation heterogener, durch texturelle Ausprägungen zu differenzierende Kompartimente mittels maschineller Lernverfahren“ verstanden werden