A hybrid training method for ANNs and its application in multi faults diagnosis of rolling bearing

A hybrid training method with probabilistic adaptive strategy for feedforward artificial neural network was proposed and applied to the problem of multi faults diagnosis of rolling bearing. The traditional training method such as LM shows fast convergence speed, but it’s easy to fall into local minimum. The heuristic method such as DE shows good global continuous optimization ability, but its convergence speed is slow. A hybrid training method of LM and DE is presented, and it overcomes the defects by using the advantages of each other. Probabilistic adaptive strategy which could save the time in some situation is adopted. Finally, this method is applied to the problem of rolling bearing faults diagnosis, and compares to other methods. The results show that, high correct classification rate were achieved by LM, and hybrid training methods still continued to converge while traditional method such as LM stopped the convergence. The probabilistic adaptive strategy strengthened the convergence ability of hybrid method in the latter progress, and achieved higher correct rate

Biologically inspired evolutionary temporal neural circuits

Biological neural networks have always motivated creation of new artificial neural networks, and in this case a new autonomous temporal neural network system. Among the more challenging problems of temporal neural networks are the design and incorporation of short and long-term memories as well as the choice of network topology and training mechanism. In general, delayed copies of network signals can form short-term memory (STM), providing a limited temporal history of events similar to FIR filters, whereas the synaptic connection strengths as well as delayed feedback loops (ER circuits) can constitute longer-term memories (LTM). This dissertation introduces a new general evolutionary temporal neural network framework (GETnet) through automatic design of arbitrary neural networks with STM and LTM. GETnet is a step towards realization of general intelligent systems that need minimum or no human intervention and can be applied to a broad range of problems. GETnet utilizes nonlinear moving average/autoregressive nodes and sub-circuits that are trained by enhanced gradient descent and evolutionary search in terms of architecture, synaptic delay, and synaptic weight spaces. The mixture of Lamarckian and Darwinian evolutionary mechanisms facilitates the Baldwin effect and speeds up the hybrid training. The ability to evolve arbitrary adaptive time-delay connections enables GETnet to find novel answers to many classification and system identification tasks expressed in the general form of desired multidimensional input and output signals. Simulations using Mackey-Glass chaotic time series and fingerprint perspiration-induced temporal variations are given to demonstrate the above stated capabilities of GETnet

Evolutionary Neuro-Computing Approaches to System Identification

System models are essentially required for analysis, controller design and future prediction. System identification is concerned with developing models of physical system. Although linear system identification got enriched with several useful classical methods, nonlinear system identification always remained active area of research due to the reason that most of the real world systems are nonlinear in nature and moreover, having non-unique models. Among the several conventional system identification techniques, the Volterra series, Hammerstein-Wiener and polynomial model identification involve considerable computational complexities. The other techniques based on regression models such as nonlinear autoregressive exogenous (NARX) and nonlinear autoregressive moving average exogenous (NARMAX), also suffer from dfficulty in choosing regressors

Optimization in bioinformatics

In this work, we present novel optimization approaches for important bioinformatical problems. The rst part deals mainly with the local optimization of molecular structures and its applications to molecular docking, while the second part discusses discrete global optimization. In the rst part, we present a novel algorithm to an old task: nd the next local optimum into a given direction on a molecular potential energy function (line search). We show that replacing a standard line search method with the new algorithm reduced the number of function/gradient evaluations in our test runs down to 47.7% (down to 85% on average) . Then, we include this method into our novel approach for locally optimizing exible ligands in the presence of their receptors, which we describe in detail, avoiding the singularity problem of orientational parameters. We extend this approach to a full ligand-receptor docking program using a Lamarckian genetic algorithm. Our validation runs show that we gained an up to tenfold speedup in comparison to other tested methods. Then, we further incorporate side chain exibility of the receptor into our approach and introduce limited backbone exibility by interpolating between known extremal conformations using spherical linear extrapolation. Our results show that this approach is very promising for exible ligand-receptor docking. However, the drawback is that we need known extremal backbone conformations for the interpolation. In the last section of the rst part, we allow a loop region to be fully exible. We present a new method to nd all possible conformations using the Go-Scheraga ring closure equations and interval arithmetic. Our results show that this algorithm reliably nds alternative conformations and is able to identify promising loop/ligand complexes of the studied example. In the second part of this work, we describe the bond order assignment problem for molecular structures. We present our novel linear 0-1-programming formulation for the very efficient computation of all optimal and suboptimal bond order assignments and show that our approach does not only outperform the original heuristic approach of Wang et al. but also commonly used software for determining bond orders on our test set considering all optimal results. This test set consists of 761 thoroughly prepared drug like molecules that were originally used for the validation of the Merck Molecular Force Field. Then, we present our lter method for feature subset selection that is based on mutual information and uses second order information. We show our mathematically well motivated criterion and, in contrast to other methods, solve the resulting optimization problem exactly by quadratic 0-1-programming. In the validation runs, our method could achieve in 18 out of 21 test scenarios the best classification accuracies. In the last section, we give our integer linear programming formulation for the detection of deregulated subgraphs in regulatory networks using expression proles. Our approach identies the subnetwork of a certain size of the regulatory network with the highest sum of node scores. To demonstrate the capabilities of our algorithm, we analyzed expression proles from nonmalignant primary mammary epithelial cells derived from BRCA1 mutation carriers and epithelial cells without BRCA1 mutation. Our results suggest that oxidative stress plays an important role in epithelial cells with BRCA1 mutations that may contribute to the later development of breast cancer. The application of our algorithm to already published data can yield new insights. As expression data and network data are still growing, methods as our algorithm will be valuable to detect deregulated subgraphs in different conditions and help contribute to a better understanding of diseases.In der vorliegenden Arbeit präsentieren wir neue Optimierungsansätze für wichtige Probleme der Bioinformatik. Der erste Teil behandelt vorwiegend die lokale Optimierung von Molekülen und die Anwendung beim molekularen Docking. Der zweite Teil diskutiert diskrete globale Optimierung. Im ersten Teil präsentieren wir einen neuartigen Algorithmus für ein altes Problem: finde das nächste lokale Optimum in einer gegebenen Richtung auf einer Energiefunktion (Liniensuche, "line search"). Wir zeigen, dass die Ersetzung einer Standardliniensuche mit unserer neuen Methode die Anzahl der Funktions- und Gradientauswertungen in unseren Testläufen auf bis zu 47.7% reduzierte (85% im Mittel). Danach nehmen wir diese Methode in unseren neuen Ansatz zur lokalen Optimierung von flexiblen Liganden im Beisein ihres Rezeptors auf, den wir im Detail beschreiben. Unser Verfahren vermeidet das Singularitätsproblem von Orientierungsparametern. Wir erweitern diese Methode zu einem vollständigen Liganden-Rezeptor-Dockingprogramm, indem wir einen Lamarck'schen genetischen Algorithmus einsetzen. Unsere Validierungsläufe zeigen, dass wir im Vergleich zu anderen getesteten Methoden einen bis zu zehnfachen Geschwindigkeitszuwachs erreichen. Danach arbeiten wir in unseren Ansatz Seitenketten- und begrenzte Backbone exibilität ein, indem wir zwischen bekannten Extremkonformationen mittels sphärischer linearer Extrapolation interpolieren. Unsere Resultate zeigen, dass unsere Methode sehr viel versprechend für flexibles Liganden-Rezeptor-Docking ist. Dennoch hat dieser Ansatz den Nachteil, dass man bekannte Extremkonformationen des Backbones für die Interpolation benötigt. Im letzten Abschnitt des ersten Teils behandeln wir eine Loopregion voll flexibel. Wir zeigen eine neue Methode, die die Go-Scheraga Ringschlussgleichungen und Intervalarithmetik nutzt, um alle möglichen Konformationen zu nden. Unsere Resultate zeigen, dass dieser Algorithmus zuverlässig in der Lage ist, alternative Konformationen zu nden. Er identiziert sehr vielversprechende Loop-Ligandenkomplexe unseres Testbeispiels. Im zweiten Teil dieser Arbeit beschreiben wir das Bindungsordnungszuweisungsproblem von Molekülen. Wir präsentieren unsere neuartige Formulierung, die auf linearer 0-1-Programmierung basiert. Dieser Ansatz ist in der Lage sehr effizient alle optimalen und suboptimalen Bindngsordnungszuweisungen zu berechnen. Unsere Methode ist nicht nur besser als der ursprüngliche Ansatz von Wang et al., sondern auch weitverbreiteter Software zur Bindungszuordnung auf unserem Testdatensatz überlegen. Dieser Datensatz besteht aus 761 sorgfältig präparierten, arzneimittelähnlichen Molekülen, die ursprünglich zur Validierung des Merck-Kraftfeldes eingesetzt wurden. Danach präsentieren wir unsere Filtermethode zur "Feature Subset Selection", die auf "Mutual Information" basiert und Informationen zweiter Ordnung nutzt. Wir geben unser mathematisch motiviertes Kriterium an und lösen das resultierende Optimierungsproblem global optimal im Gegensatz zu anderen Ansätzen. In unseren Validierungsläufen konnte unsere Methode in 18 von 21 Testszenarien die beste Klassizierungsrate erreichen. Im letzten Abschnitt geben wir unsere, auf linearer 0-1-Programmierung basierende Formulierung zur Berechnung von deregulierten Untergraphen in regulatorischen Netzwerken an. Die Basisdaten für diese Methode sind Expressionsprole. Unser Ansatz identiziert die Unternetze einer gewissen Größe mit der höchsten Summe der Knotenscores. Wir analysierten Expressionsprole von nicht bösartigen Brustepithelzellen von BRCA1 Mutationsträgern und Epithelzellen ohne BRCA1 Mutation, um die Fähigkeiten unseres Algorithmuses zu demonstrieren. Unsere Resultate legen nahe, dass oxidativer Stress eine wichtige Rolle bei Epithelzellen mit BRCA1 Mutation spielt, der zur späteren Entwicklung von Brustkrebs beitragen könnte. Die Anwendung unseres Ansatzes auf bereits publizierte Daten kann zu neuen Erkenntnissen führen. Da sowohl Expressions- wie auch Netzwerkdaten ständig anwachsen, sind es Methoden wie unser Algorithmus die wertvoll sein werden, um deregulierte Subgraphen in verschiedenen Situationen zu entdecken. Damit trägt unser Ansatz zu einem besseren Verständnis von Krankheiten und deren Verlauf bei

Problem Decomposition and Adaptation in Cooperative Neuro-Evolution

One way to train neural networks is to use evolutionary algorithms such as cooperative coevolution - a method that decomposes the network's learnable parameters into subsets, called subcomponents. Cooperative coevolution gains advantage over other methods by evolving particular subcomponents independently from the rest of the network. Its success depends strongly on how the problem decomposition is carried out. This thesis suggests new forms of problem decomposition, based on a novel and intuitive choice of modularity, and examines in detail at what stage and to what extent the different decomposition methods should be used. The new methods are evaluated by training feedforward networks to solve pattern classification tasks, and by training recurrent networks to solve grammatical inference problems. Efficient problem decomposition methods group interacting variables into the same subcomponents. We examine the methods from the literature and provide an analysis of the nature of the neural network optimization problem in terms of interacting variables. We then present a novel problem decomposition method that groups interacting variables and that can be generalized to neural networks with more than a single hidden layer. We then incorporate local search into cooperative neuro-evolution. We present a memetic cooperative coevolution method that takes into account the cost of employing local search across several sub-populations. The optimisation process changes during evolution in terms of diversity and interacting variables. To address this, we examine the adaptation of the problem decomposition method during the evolutionary process. The results in this thesis show that the proposed methods improve performance in terms of optimization time, scalability and robustness. As a further test, we apply the problem decomposition and adaptive cooperative coevolution methods for training recurrent neural networks on chaotic time series problems. The proposed methods show better performance in terms of accuracy and robustness

Resistance is Futile: Physical Science, Systems Biology and Single-Cell Analysis to Understanding the Plastic and Heterogeneous Nature of Melanoma and Their Role in Non-Genetic Drug Resistance

Melanoma is the most deadly form of skin cancer due to its great metastatic potential. Targeted therapy that inhibits the BRAF-V600E driver mutation has shown impressive initial responses in melanoma patients. However, drug resistance, as the universal phenomenon for any cancer therapy, always limits treatment efficacy and compromises outcomes. As the early-step of resistance development, non-genetic mechanisms enable cancer cells to transition into a drug-resistant state in as early as a few days after drug treatment without alteration of the genome. This early mechanism is, to a large extent, due to the heterogeneous and highly plastic nature of tumor cells. Therefore, it imperative to understand the plastic and heterogeneous nature of the melanoma cells in order to identify combination therapies that can overcome resistance. In this thesis, we investigate these two fundamental natures of non-genetic drug resistance using BRAF inhibition of BRAF-mutant melanomas as the model system. These melanoma cells undergo multi-step, reversible drug-induced cell-state transitions from the original sensitive phenotype to a drug-resistant one. We first conducted bulk analysis to characterize the detailed kinetics of the entire transition from drug-sensitive state towards drug-resistant state, revealing expression changes of thousands of genes and extensive chromatin remodeling. A 3-step computational biology approach greatly simplified the complexity and revealed that the whole cell-state transition was controlled by a gene module activated within just the first three days of drug treatment, with the RelA transcription factor driving chromatin remodeling to establish an epigenetic program encoding long-term phenotype changes towards resistance. From there, a detailed mechanism connecting tumor epigenetic plasticity with non-genetic drug resistance was resolved through in-depth molecular biology experiments. The mechanism was validated in clinical patient samples. We further investigated heterogeneity by moving from bulk cellular studies to single-cell analysis. The single-cell view further revealed that two driving forces from both cell-state interconversions and phenotype-specific drug selection control the cell-state transition dynamics. The single-cell studies also pinpointed the signaling network hub, RelA, as the driver molecule of the initiation of the adaptive transition. These two competing driving forces were further quantitatively modeled via a thermodynamic-inspired surprisal analysis and a modified Fokker-Planck-type kinetic model. Finally, using integrated single-cell proteomic and metabolic technology I developed to characterize the early-stage signaling and metabolic changes upon initial drug responses, we further identified two distinct paths connecting drug-sensitive and drug-tolerant states. Melanoma cells exclusively traverse one of the two paths depending on the level of MITF in the drug-naïve cells. The two trajectories are associated with distinct signaling and metabolic susceptibilities and are independently druggable. In total, this thesis combines and synergizes various physical science and systems biology approaches together with several unique single-cell technologies and analysis to obtain a deep and comprehensive understanding of non-genetic drug resistance in cancer. The findings from this thesis provide several novel insights into the rational design of effective combination therapy for overcoming the development of resistance in response to cancer treatments.</p

Training issues and learning algorithms for feedforward and recurrent neural networks

Ph.DDOCTOR OF PHILOSOPH

Advances in Evolutionary Algorithms

With the recent trends towards massive data sets and significant computational power, combined with evolutionary algorithmic advances evolutionary computation is becoming much more relevant to practice. Aim of the book is to present recent improvements, innovative ideas and concepts in a part of a huge EA field