Numerical Comparison of Line Search Criteria in Nonlinear Conjugate Gradient Algorithms

One of the open problems known to researchers on the application of nonlinear conjugate gradient methods for addressing unconstrained optimization problems is the influence of accuracy of linear search procedure on the performance of the conjugate gradient algorithm. Key to any CG algorithm is the computation of an optimalstep size for which many procedures have been postulated. In this paper, we assess and compare the performance of a modified Armijo and Wolfe line search procedures on three variants of nonlinear CGM by carrying out a numerical test. Experiments reveal that our modified procedure and the strong Wolfe procedures guaranteed fast convergence

An extended Dai-Liao conjugate gradient method with global convergence for nonconvex functions

Using an extension of some previously proposed modified secant equations in the Dai-Liao approach, a modified nonlinear conjugate gradient method is proposed. As interesting features, the method employs the objective function values in addition to the gradient information and satisfies the sufficient descent property with proper choices for its parameter. Global convergence of the method is established without convexity assumption on the objective function. Results of numerical comparisons are reported. They demonstrate efficiency of the proposed method in the sense of the Dolan-Moré performance profile

A new conjugate gradient method based on the modified secant equations

Abstract. Based on the secant equations proposed by Zhang, Deng and Chen, we propose a new nonlinear conjugate gradient method for unconstrained optimization problems. Global convergence of this method is established under some proper conditions

A Modified Conjugacy Condition and Related Nonlinear Conjugate Gradient Method

The conjugate gradient (CG) method has played a special role in solving large-scale nonlinear optimization problems due to the simplicity of their very low memory requirements. In this paper, we propose a new conjugacy condition which is similar to Dai-Liao (2001). Based on this condition, the related nonlinear conjugate gradient method is given. With some mild conditions, the given method is globally convergent under the strong Wolfe-Powell line search for general functions. The numerical experiments show that the proposed method is very robust and efficient

Empirical analysis of neural networks training optimisation

A Dissertation submitted to the Faculty of Science, University of the Witwatersrand, Johannesburg, in fulfilment of the requirements for the degree of Master of Science in Mathematical Statistics,School of Statistics and Actuarial Science. October 2016.Neural networks (NNs) may be characterised by complex error functions with attributes such as saddle-points, local minima, even-spots and plateaus. This complicates the associated training process in terms of efficiency, convergence and accuracy given that it is done by minimising such complex error functions. This study empirically investigates the performance of two NNs training algorithms which are based on unconstrained and global optimisation theories, i.e. the Resilient propagation (Rprop) and the Conjugate Gradient with Polak-Ribière updates (CGP). It also shows how the network structure plays a role in the training optimisation of NNs. In this regard, various training scenarios are used to classify two protein data, i.e. the Escherichia coli and Yeast data. These training scenarios use varying numbers of hidden nodes and training iterations. The results show that Rprop outperforms CGP. Moreover, it appears that the performance of classifiers varies under various training scenarios.LG201

Some modified Yabe–Takano conjugate gradient methods with sufficient descent condition

Descent condition is a crucial factor to establish the global convergence of nonlinear conjugate gradient method. In this paper, we propose some modified Yabe–Takano conjugate gradient methods, in which the corresponding search directions always satisfy the sufficient descent property independently of the convexity of the objective function. Differently from the existent methods, a new update strategy in constructing the search direction is proposed to establish the global convergence of the presented methods for the general nonconvex objective function. Numerical results illustrate that our methods can efficiently solve the test problems and therefore is promising

Choice of parameters in gradient methods for the unconstrained optimization problems

Posmatra se problem optimizacije bez ograničenja. Za rešavanje problema  optimizacije bez ograničenja postoji mnoštvo raznovrsnih metoda. Istraživanje ovde motivisano je potrebom za metodama koje će brzo konvergirati. Cilj je sistematizacija poznatih rezultata, kao i teorijska i numerička analiza mogućnosti uvođenja parametra u gradijentne metode. Najpre se razmatra problem minimizacije konveksne funkcije više promenljivih. Problem minimizacije konveksne funkcije više promenljivih ovde se rešava bez izračunavanja matrice hesijana, što je naročito aktuelno za sisteme velikih dimenzija, kao i za probleme optimizacije kod kojih ne raspolažemo ni tačnom vrednošću funkcije cilja, ni tačnom vrednošću gradijenta. Deo motivacije za istraživanjem ovde leži i u postojanju problema kod kojih je funkcija cilja rezultat simulacija. Numerički rezultati, predstavljeni u Glavi 6, pokazuju da uvođenje izvesnog parametra može biti korisno, odnosno, dovodi do ubrzanja određenog metoda optimizacije. Takođe se predstavlja jedan novi hibridni metod konjugovanog gradijenta, kod koga je parametar konjugovanog gradijenta konveksna kombinacija dva poznata parametra konjugovanog gradijenta. U prvoj glavi opisuje se motivacija kao i osnovni pojmovi potrebni za praćenje preostalih glava. U drugoj glavi daje se pregled nekih gradijentnih metoda prvog i drugog reda. Četvrta glava sadrži pregled osnovnih pojmova i nekih rezultata vezanih za metode konjugovanih gradijenata. Pomenute glave su tu radi pregleda nekih poznatih rezultata, dok se originalni doprinos predstavlja u trećoj, petoj i šestoj glavi. U trećoj glavi se opisuje izvesna modifikacija određenog metoda u kome se koristi multiplikativni parametar, izabran na slučajan način. Dokazuje se linearna konvergencija tako formiranog novog metoda. Peta glava sadrži originalne rezultate koji se odnose na metode konjugovanih gradijenata. Naime, u ovoj glavi predstavlja se novi hibridni metod konjugovanih gradijenata, koji je konveksna kombinacija dva poznata metoda konjugovanih gradijenata. U šestoj glavi se daju rezultati numeričkih eksperimenata, izvršenih na  izvesnom skupu test funkcija, koji se odnose na metode iz treće i pete glave. Implementacija svih razmatranih algoritama rađena je u paketu MATHEMATICA. Kriterijum upoređivanja je vreme rada centralne procesorske jedinice.6The problem under consideration is an unconstrained optimization problem. There are many different methods made in aim to solve the optimization problems.  The investigation made here is motivated by the fact that the methods which converge fast are necessary. The main goal is the systematization of some known results and also theoretical and numerical analysis of the possibilities to int roduce some parameters within gradient methods. Firstly, the minimization problem is considered, where the objective function is a convex, multivar iable function. This problem is solved here without the calculation of Hessian, and such solution is very important, for example, when the  big dimension systems are solved, and also for solving optimization problems with unknown values of the objective function and its gradient. Partially, this investigation is motivated by the existence of problems where the objective function is the result of simulations. Numerical results, presented in  Chapter  6, show that the introduction of a parameter is useful, i.e., such introduction results by the acceleration of the known optimization method. Further, one new hybrid conjugate gradient method is presented, in which the conjugate gradient parameter is a convex combination of two known conjugate gradient parameters. In the first chapter, there is motivation and also the basic co ncepts which are necessary for the other chapters. The second chapter contains the survey of some first order and second order gradient methods. The fourth chapter contains the survey of some basic concepts and results corresponding to conjugate gradient methods. The first, the second and the fourth  chapters are here to help in considering of some known results, and the original results are presented in the chapters 3,5 and 6. In the third chapter, a modification of one unco nstrained optimization method is presented, in which the randomly chosen multiplicative parameter is used. Also, the linear convergence of such modification is proved. The fifth chapter contains the original results, corresponding to conjugate gradient methods. Namely, one new hybrid conjugate gradient method is presented, and this  method is the convex combination of two known conjugate gradient methods. The sixth chapter consists of the numerical results, performed on a set of test functions, corresponding to methods in the chapters 3 and 5. Implementation of all considered algorithms is made in Mathematica. The comparison criterion is CPU time.The problem under consideration is an unconstrained optimization problem. There are many different methods made in aim to solve the optimization problems.  The investigation made here is motivated by the fact that the methods which converge fast are necessary. The main goal is the systematization of some known results and also theoretical and numerical analysis of the possibilities to int roduce some parameters within gradient methods. Firstly, the minimization problem is considered, where the objective function is a convex, multivar iable function. This problem is solved here without the calculation of Hessian, and such solution is very important, for example, when the  big dimension systems are solved, and also for solving optimization problems with unknown values of the objective function and its gradient. Partially, this investigation is motivated by the existence of problems where the objective function is the result of simulations. Numerical results, presented in  Chapter  6, show that the introduction of a parameter is useful, i.e., such introduction results by the acceleration of the known optimization method. Further, one new hybrid conjugate gradient method is presented, in which the conjugate gradient parameter is a convex combination of two known conjugate gradient parameters. In the first chapter, there is motivation and also the basic co ncepts which are necessary for the other chapters. Key  Words Documentation  97 The second chapter contains the survey of some first order and second order gradient methods. The fourth chapter contains the survey of some basic concepts and results corresponding to conjugate gradient methods. The first, the second and the fourth  chapters are here to help in considering of some known results, and the original results are presented in the chapters 3,5 and 6. In the third chapter, a modification of one unco nstrained optimization method is presented, in which the randomly chosen multiplicative parameter is used. Also, the linear convergence of such modification is proved. The fifth chapter contains the original results, corresponding to conjugate gradient methods. Namely, one new hybrid conjugate gradient method is presented, and this  method is the convex combination of two known conjugate gradient methods. The sixth chapter consists of the numerical results, performed on a set of test functions, corresponding to methods in the chapters 3 and 5. Implementation of all considered algorithms is made in Mathematica. The comparison criterion is CPU tim

Development and validation of novel methods for the study of Staphylococcus aureus PVL strains

Since the initial association of the Panton Valentine leucocidin (PVL) toxin with highly virulent strains of Staphylococcus aureus found in the community, a firm epidemiological link has been established between the PVL encoding genes and community-acquired strains of both meticillin resistant (CA-MRSA) and susceptible (CA-MSSA) Staphylococcus aureus. While most research has predominantly concentrated on the genotyping of CA-MRSA strains, PVL-MSSA appear to pose an increasing public health risk. Currently though, there exists a dearth of epidemiological data on PVL-MSSA strains, particularly with regard to the lukS and lukF genes which encode the PVL toxin. This first aim of the present study therefore was to explore the genetic diversity of a group of PVL-MSSA clinical local isolates in order to contribute to the limited current data and provide insight into the evolution and emergence of PVL-MRSA isolates. In addition, as current typing systems are cumbersome, time consuming and expensive, this present study was also aimed at the development of a rapid high resolution melt (HRM) typing system for the characterisation of PVL-positive isolates. The PVL toxin is encoded for by two highly conserved adjacent genes (lukF and lukS) which are co-transcribed. Variations in these genes correlate with a strain’s genotype. Therefore, the present study set out to genotype isolates based on the four major SNPs at positions 527 and 663 of the lukS gene and 1396 and 1729 of the lukF gene. The final aim of the present study was the development of an enzyme linked immunosorbent assay (ELISA) system (for the detection and quantification of both PVL and alpha haemolysin) that has potential application in clinical diagnosis and as a research tool. Characterisation of a collection of UK PVL-MSSA isolates by MLST and spa typing which is presented in the present study, showed a genetic similarity to circulating PVL-MRSA strains, with 94.7% of the isolates belonging to CA-MRSA related genetic backgrounds (ST1, ST22, ST30, ST772 and ST88). Three novel spa types (t6642, t6643 and t6769) and a novel ST (ST1518) were however detected in this population. Furthermore, the presence of identical PVL phages and haplotypes in the PVL-MSSA isolates to those previously described in PVL-MRSA isolates point strongly at the role these strains may play as precursors of emerging lineages of clinical significance. The HRM assay developed in this study was able to accurately genotype PVL-positive isolates based on the double allelic variations in both the lukS and lukF genes. The high degree of sensitivity of this technique was clearly demonstrated by its ability to differentiate between the lukS A527/G663 and G527/T663 genotypes which theoretically should have the same melt temperature. Despite the issues in data interpretation, which arose following attempts to improve the discrimination of this technique by the addition of a third locus (spa), the technique still showed potential as a useful tool for the rapid genotyping of PVL-positive isolates. While HRM was useful in rapidly detecting and genotyping PVL-positive isolates, the actual level of protein production of both PVL and HLA toxins could only be determined following the development and validation in the present study of a simple competitive ELISA platform which exploited the high affinity biotin/streptavidin interaction to improve sensitivity. This technique would be especially useful in settings which lack the specialised equipment required for genetic studies like HRM. In summary, in addition to contributing to the limited epidemiological information about PVL-MSSA strains and demonstrating a clear role for these strains in the evolution of PVL-MRSA strains, the present study has developed two distinct methods to aid the study of S. aureus PVL producing strains which are becoming a significant healthcare problem worldwide. The present study will contribute to our understanding of these strains and to the development of intervention strategies to curb their spread and threat

Development and validation of novel methods for the study of Staphylococcus aureus PVL strains

Since the initial association of the Panton Valentine leucocidin (PVL) toxin with highly virulent strains of Staphylococcus aureus found in the community, a firm epidemiological link has been established between the PVL encoding genes and community-acquired strains of both meticillin resistant (CA-MRSA) and susceptible (CA-MSSA) Staphylococcus aureus. While most research has predominantly concentrated on the genotyping of CA-MRSA strains, PVL-MSSA appear to pose an increasing public health risk. Currently though, there exists a dearth of epidemiological data on PVL-MSSA strains, particularly with regard to the lukS and lukF genes which encode the PVL toxin. This first aim of the present study therefore was to explore the genetic diversity of a group of PVL-MSSA clinical local isolates in order to contribute to the limited current data and provide insight into the evolution and emergence of PVL-MRSA isolates. In addition, as current typing systems are cumbersome, time consuming and expensive, this present study was also aimed at the development of a rapid high resolution melt (HRM) typing system for the characterisation of PVL-positive isolates. The PVL toxin is encoded for by two highly conserved adjacent genes (lukF and lukS) which are co-transcribed. Variations in these genes correlate with a strain’s genotype. Therefore, the present study set out to genotype isolates based on the four major SNPs at positions 527 and 663 of the lukS gene and 1396 and 1729 of the lukF gene. The final aim of the present study was the development of an enzyme linked immunosorbent assay (ELISA) system (for the detection and quantification of both PVL and alpha haemolysin) that has potential application in clinical diagnosis and as a research tool. Characterisation of a collection of UK PVL-MSSA isolates by MLST and spa typing which is presented in the present study, showed a genetic similarity to circulating PVL-MRSA strains, with 94.7% of the isolates belonging to CA-MRSA related genetic backgrounds (ST1, ST22, ST30, ST772 and ST88). Three novel spa types (t6642, t6643 and t6769) and a novel ST (ST1518) were however detected in this population. Furthermore, the presence of identical PVL phages and haplotypes in the PVL-MSSA isolates to those previously described in PVL-MRSA isolates point strongly at the role these strains may play as precursors of emerging lineages of clinical significance. The HRM assay developed in this study was able to accurately genotype PVL-positive isolates based on the double allelic variations in both the lukS and lukF genes. The high degree of sensitivity of this technique was clearly demonstrated by its ability to differentiate between the lukS A527/G663 and G527/T663 genotypes which theoretically should have the same melt temperature. Despite the issues in data interpretation, which arose following attempts to improve the discrimination of this technique by the addition of a third locus (spa), the technique still showed potential as a useful tool for the rapid genotyping of PVL-positive isolates. While HRM was useful in rapidly detecting and genotyping PVL-positive isolates, the actual level of protein production of both PVL and HLA toxins could only be determined following the development and validation in the present study of a simple competitive ELISA platform which exploited the high affinity biotin/streptavidin interaction to improve sensitivity. This technique would be especially useful in settings which lack the specialised equipment required for genetic studies like HRM. In summary, in addition to contributing to the limited epidemiological information about PVL-MSSA strains and demonstrating a clear role for these strains in the evolution of PVL-MRSA strains, the present study has developed two distinct methods to aid the study of S. aureus PVL producing strains which are becoming a significant healthcare problem worldwide. The present study will contribute to our understanding of these strains and to the development of intervention strategies to curb their spread and threat

Metisilliini-resistentti Staphylococcus aureus (MRSA) ja Streptococcus pneumoniae : molekyylimenetelmien käyttö epidemiologisissa analyyseissa

Methicillin-resistant Staphylococcus aureus (MRSA) and Streptococcus pneumoniae are major health problems worldwide, both found in symptomless carriage but also causing even life-threatening infections. The aim of this thesis was to characterise MRSA and S. pneumoniae in detail by using several molecular typing methods for various epidemiological purposes: clonality analysis, epidemiological surveillance, outbreak investigation, and virulence factor analysis. The characteristics of MRSA isolates from the strain collection of the Finnish National Infectious Disease Register (NIDR) and pneumococcal isolates collected from military recruits and children with acute otitis media (AOM) were analysed using various typing techniques. Antimicrobial susceptibility testing, pulsed-field gel electrophoresis (PFGE), multilocus sequence typing (MLST), spa typing, staphylococcal cassette chromosome mec (SCCmec) typing, and the detection of Panton-Valentine leukocidin (PVL) genes were performed for MRSA isolates. Pneumococcal isolates were analysed using antimicrobial susceptibility testing, serotyping, MLST, and by detecting pilus islet 1 (PI-1) and 2 (PI-2) genes. Several international community- and hospital-associated MRSA clones were recognised in Finland. The genetic diversity among MRSA FIN-4 isolates and among FIN-16 isolates was low. Overall, MRSA blood isolates from 1997 to 2006 were genetically diverse. spa typing was found to be a highly discriminatory, rapid and accurate typing method and it also qualifies as the primary typing method in countries with a long history of PFGE-based MRSA strain nomenclature. However, additional typing by another method, e.g. PFGE, is needed in certain situations to be able to provide adequate discrimination for epidemiological surveillance and outbreak investigation. An outbreak of pneumonia was associated with one pneumococcal strain among military recruits, previously healthy young men living in a crowded setting. The pneumococcal carriage rate after the outbreak was found to be exceptionally high. PI-1 genes were detected at a rather low prevalence among pneumococcal isolates from children with AOM. However, the study demonstrated that PI-1 has existed among pneumococcal isolates prior to pneumococcal conjugate vaccine and the increased antimicrobial resistance era. Moreover, PI-1 was found to associate with the serotype rather than the genotype. This study adds to our understanding of the molecular epidemiology of MRSA strains in Finland and the importance of an appropriate genotyping method to be able to perform high-level laboratory-based surveillance of MRSA. Epidemiological and molecular analyses of S. pneumoniae add to our knowledge of the characteristics of pneumococcal strains in Finland.Metisilliini-resistentti Staphylococcus aureus (MRSA) ja Streptococcus pneumoniae aiheuttavat merkittäviä terveysongelmia maailmanlaajuisesti. Molemmat voivat löytyä oireettomilta kantajilta, mutta ne voivat aiheuttaa myös hengenvaarallisia infektioita. Staphylococcus aureus on tavallinen pehmytkudosinfektioiden aiheuttajamikrobi, joka voi aiheuttaa myös vakavia infektioita kuten verenmyrkytystä. Viime vuosien aikana MRSA-bakteerien aiheuttamat infektiot ovat yleistyneet Suomessa sekä sairaala- että avohoitopotilailla. Vuoden 2009 aikana MRSA-tilanne parani ja pysyi ennallaan myös vuonna 2010. Streptococcus pneumoniae eli pneumokokki on tavallisin hengitystieinfektioita kuten poskiontelontulehdusta, lasten välikorvantulehduksia tai keuhkokuumetta aiheuttava bakteeri, mutta myös yleisimpiä vakavien infektioiden, kuten verenmyrkytyksen tai aivokalvontulehduksen, aiheuttajia. Pneumokokkeja kantaa nenänielussaan viidennes terveistä aikuisista, pikkulapsista jopa puolet. Pneumokokin aiheuttamia epidemioita esiintyy etenkin lastentarhoissa ja armeijassa. MRSA- ja pneumokokkikantojen tyypitykseen ja nimeämiseen käytetään useita erilaisia DNA-pohjaisia menetelmiä kuten geenien monistamista ja sekvensointia sekä pulssikenttägeelielektroforeesia (PFGE). Tässä työssä karakterisoitiin MRSA- ja S. pneumoniae -bakteereita tarkemmin käyttämällä useita tyypitysmenetelmiä erilaisiin epidemiologisiin tutkimuksiin: klonaalisiin analyyseihin, epidemiologiseen seurantaan, epidemiaselvityksiin ja virulenssitekijöiden analysointiin. Useita kansainvälisiä sairaalaperäisiä ja avohoitoon liittyviä MRSA-klooneja tunnistettiin Suomessa. spa-geenin sekvensointiin perustuva spa-tyypitys oli hyvin erottelukykyinen, nopea ja tarkka tyypitysmenetelmä ja se täyttää vaatimukset ensisijaisena tyypitysmenetelmänä myös maissa joilla on pitkä historia MRSA-kantojen nimeämisestä PFGE-tyypityksellä. Ahtaissa oloissa elävien, aiemmin terveiden varusmiesten keuhkokuume-epidemiaan liittyi yksi pneumokokkikanta. Melko pieneltä osalta pneumokokkikannoista, jotka oli eristetty akuuttia välikorvantulehdusta sairastavilta lapsilta, löydettiin pilus-1 (PI-1) -geenejä. Tutkimus lisää tietoutta MRSA-kantojen molekyyliepidemiologiasta Suomessa ja käytössä olevien genotyypitysmenetelmien soveltuvuudesta korkeatasoiseen MRSA-seurantatyöhön. Pneumokokin epidemiologiset ja molekylaariset tutkimukset lisäävät tietoa Suomalaisten pneumokokkikantojen ominaisuuksista ja niiden taudinaiheuttamiskyvystä