Medical Image Diagnosis of Lung Cancer by Deep Feedback GMDH-Type Neural Network

The deep feedback Group Method of Data Handling (GMDH)-type neural network is applied to the medical image diagnosis of lung cancer. The deep feedback GMDH-type neural network can identified very complex nonlinear systems using heuristic self-organization method which is a type of evolutionary computation. The deep neural network architectures are organized so as to minimize the prediction error criterion defined as Akaike’s Information Criterion (AIC) or Prediction Sum of Squares (PSS). In this algorithm, the principal component-regression analysis is used for the learning calculation of the neural network. It is shown that the deep feedback GMDH-type neural network algorithm is useful for the medical image diagnosis of lung cancer because deep neural network architectures are automatically organized using only input and output data

Combining group method of data handling models using artificial bee colony algorithm for time series forecasting

Time series forecasting which uses models to predict future values based on some historical data is an important area of forecasting, and has gained the attention of researchers from various related fields of study. In line with its popularity, various models have been introduced for producing accurate time series forecasts. However, to produce an accurate forecast is not an easy feat especially when dealing with nonlinear data due to the abstract nature of the data. In this study, a model for accurate time series forecasting based on Artificial Bee Colony (ABC) algorithm and Group Method of Data Handling (GMDH) models with variant transfer functions, namely polynomial, sigmoid, radial basis function and tangent was developed. Initially, in this research, the GMDH models were used to forecast the time series data followed by each forecast that was combined using ABC. Then, the ABC produced the weight for each forecast before aggregating the forecasts. To evaluate the performance of the developed GMDH-ABC model, input data on tourism arrivals (Singapore and Indonesia) and airline passengers’ data were processed using the model to produce reliable forecast on the time series data. To validate the evaluation, the performance of the model was compared against benchmark models such as the individual GMDH models, Artificial Neural Network (ANN) model and combined GMDH using simple averaging (GMDH-SA) model. Experimental results showed that the GMDH-ABC model had the highest accuracy compared to the other models, where it managed to reduce the Root Mean Square Error (RMSE) of the conventional GMDH model by 15.78% for Singapore data, 28.2% for Indonesia data and 30.89% for airline data. As a conclusion, these results demonstrated the reliability of the GMDH-ABC model in time series forecasting, and its superiority when compared to the other existing models

Analysis and implimentation of radial basis function neural network for controlling non-linear dynamical systems

PhD ThesisModelling and control of non-linear systems are not easy, which are now being solved by the application of neural networks. Neural networks have been proved to solve these problems as they are described by adjustable parameters which are readily adaptable online. Many types of neural networks have been used and the most common one is the backpropagation algorithm. The algorithm has some disadvantages, such as slow convergence and construction complexity. An alternative neural networks to overcome the limitations associated with the backpropagation algorithm is the Radial Basis Function Network which has been widely used for solving many complex problems. The Radial Basis Function Network is considered in this theses, along with a new adaptive algorithm which has been developed to overcome the problem of the optimum parameter selection. Use of the new algorithm reduces the trial and error of selecting the minimum required number of centres and guarantees the optimum values of the centres, the widths between the centres and the network weights. Computer simulation usmg SimulinklMatlab packages, demonstrated the results of modelling and control of non-linear systems. Moreover, the algorithm is used for selecting the optimum parameters of a non-linear real system 'Brushless DC Motor'. In the laboratory implementation satisfactory results have been achieved, which show that the Radial Basis Function may be used for modelling and on-line control of such real non-linear systems.The Libyan Ministry of Higher Education

Approximation methodologies for explicit model predictive control of complex systems

This thesis concerns the development of complexity reduction methodologies for the application of multi-parametric/explicit model predictive (mp-MPC) control to complex high fidelity models. The main advantage of mp-MPC is the offline relocation of the optimization task and the associated computational expense through the use of multi-parametric programming. This allows for the application of MPC to fast sampling systems or systems for which it is not possible to perform online optimization due to cycle time requirements. The application of mp-MPC to complex nonlinear systems is of critical importance and is the subject of the thesis. The first part is concerned with the adaptation and development of model order reduction (MOR) techniques for application in combination to mp-MPC algorithms. This first part includes the mp-MPC oriented use of existing MOR techniques as well as the development of new ones. The use of MOR for multi-parametric moving horizon estimation is also investigated. The second part of the thesis introduces a framework for the ‘equation free’ surrogate-model based design of explicit controllers as a possible alternative to multi-parametric based methods. The methodology relies upon the use of advanced data-classification approaches and surrogate modelling techniques, and is illustrated with different numerical examples.Open Acces

Previsão não-linear da taxa de câmbio real/dólar utilizando redes neurais e sistemas nebulosos

Dissertação (mestrado) - Universidade Federal de Santa Catarina, Centro Sócio-Econômico. Programa de Pós-graduação em Economi

WEIGH-IN-MOTION DATA-DRIVEN PAVEMENT PERFORMANCE PREDICTION MODELS

The effective functioning of pavements as a critical component of the transportation system necessitates the implementation of ongoing maintenance programs to safeguard this significant and valuable infrastructure and guarantee its optimal performance. The maintenance, rehabilitation, and reconstruction (MRR) program of the pavement structure is dependent on a multidimensional decision-making process, which considers the existing pavement structural condition and the anticipated future performance. Pavement Performance Prediction Models (PPPMs) have become indispensable tools for the efficient implementation of the MRR program and the minimization of associated costs by providing precise predictions of distress and roughness based on inventory and monitoring data concerning the pavement structure\u27s state, traffic load, and climatic conditions. The integration of PPPMs has become a vital component of Pavement Management Systems (PMSs), facilitating the optimization, prioritization, scheduling, and selection of maintenance strategies. Researchers have developed several PPPMs with differing objectives, and each PPPM has demonstrated distinct strengths and weaknesses regarding its applicability, implementation process, and data requirements for development. Traditional statistical models, such as linear regression, are inadequate in handling complex nonlinear relationships between variables and often generate less precise results. Machine Learning (ML)-based models have become increasingly popular due to their ability to manage vast amounts of data and identify meaningful relationships between them to generate informative insights for better predictions. To create ML models for pavement performance prediction, it is necessary to gather a significant amount of historical data on pavement and traffic loading conditions. The Long-Term Pavement Performance Program (LTPP) initiated by the Federal Highway Administration (FHWA) offers a comprehensive repository of data on the environment, traffic, inventory, monitoring, maintenance, and rehabilitation works that can be utilized to develop PPPMs. The LTPP also includes Weigh-In-Motion (WIM) data that provides information on traffic, such as truck traffic, total traffic, directional distribution, and the number of different axle types of vehicles. High-quality traffic loading data can play an essential role in improving the performance of PPPMs, as the Mechanistic-Empirical Pavement Design Guide (MEPDG) considers vehicle types and axle load characteristics to be critical inputs for pavement design. The collection of high-quality traffic loading data has been a challenge in developing Pavement Performance Prediction Models (PPPMs). The Weigh-In-Motion (WIM) system, which comprises WIM scales, has emerged as an innovative solution to address this issue. By leveraging computer vision and machine learning techniques, WIM systems can collect accurate data on vehicle type and axle load characteristics, which are critical factors affecting the performance of flexible pavements. Excessive dynamic loading caused by heavy vehicles can result in the early disintegration of the pavement structure. The Long-Term Pavement Performance Program (LTPP) provides an extensive repository of WIM data that can be utilized to develop accurate PPPMs for predicting pavement future behavior and tolerance. The incorporation of comprehensive WIM data collected from LTPP has the potential to significantly improve the accuracy and effectiveness of PPPMs. To develop artificial neural network (ANN) based pavement performance prediction models (PPPMs) for seven distinct performance indicators, including IRI, longitudinal crack, transverse crack, fatigue crack, potholes, polished aggregate, and patch failure, a total of 300 pavement sections with WIM data were selected from the United States of America. Data collection spanned 20 years, from 2001 to 2020, and included information on pavement age, material properties, climatic properties, structural properties, and traffic-related characteristics. The primary dataset was then divided into two distinct subsets: one which included WIMgenerated traffic data and another which excluded WIM-generated traffic data. Data cleaning and normalization were meticulously performed using the Z-score normalization method. Each subset was further divided into two separate groups: the first containing 15 years of data for model training and the latter containing 5 years of data for testing purposes. Principal Component Analysis (PCA) was then employed to reduce the number of input variables for the model. Based on a cumulative Proportion of Variation (PoV) of 96%, 12 input variables were selected. Subsequently, a single hidden layer ANN model with 12 neurons was generated for each performance indicator. The study\u27s results indicate that incorporating Weigh-In-Motion (WIM)-generated traffic loading data can significantly enhance the accuracy and efficacy of pavement performance prediction models (PPPMs). This improvement further supports the suitability of optimized pavement maintenance scheduling with minimal costs, while also ensuring timely repairs to promote acceptable serviceability and structural stability of the pavement. The contributions of this research are twofold: first, it provides an enhanced understanding of the positive impacts that high-quality traffic loading data has on pavement conditions; and second, it explores potential applications of WIM data within the Pavement Management System (PMS)

Longitudinal study of first-time freshmen using data mining

In the modern world, higher education is transitioning from enrollment mode to recruitment mode. This shift paved the way for institutional research and policy making from historical data perspective. More and more universities in the U.S. are implementing and using enterprise resource planning (ERP) systems, which collect vast amounts of data. Although few researchers have used data mining for performance, graduation rates, and persistence prediction, research is sparse in this area, and it lacks the rigorous development and evaluation of data mining models. The primary objective of this research was to build and analyze data mining models using historical data to find out patterns and rules that classified students who were likely to drop-out and students who were likely to persist.;Student retention is a major problem for higher education institutions, and predictive models developed using traditional quantitative methods do not produce results with high accuracy, because of massive amounts of data, correlation between attributes, missing values, and non-linearity of variables; however, data mining techniques work well with these conditions. In this study, various data mining models were used along with discretization, feature subset selection, and cross-validation; the results were not only analyzed using the probability of detection and probability of false alarm, but were also analyzed using variances obtained in these performance measures. Attributes were grouped together based on the current hypotheses in the literature. Using the results of feature subset selectors and treatment learners, attributes that contributed the most toward a student\u27s decision of dropping out or staying were found, and specific rules were found that characterized a successful student. The performance measures obtained in this study were significantly better than previously reported in the literature

Design of neuro-fuzzy models by evolutionary and gradient-based algorithms

All systems found in nature exhibit, with different degrees, a nonlinear behavior. To emulate this behavior, classical systems identification techniques use, typically, linear models, for mathematical simplicity. Models inspired by biological principles (artificial neural networks) and linguistically motivated (fuzzy systems), due to their universal approximation property, are becoming alternatives to classical mathematical models. In systems identification, the design of this type of models is an iterative process, requiring, among other steps, the need to identify the model structure, as well as the estimation of the model parameters. This thesis addresses the applicability of gradient-basis algorithms for the parameter estimation phase, and the use of evolutionary algorithms for model structure selection, for the design of neuro-fuzzy systems, i.e., models that offer the transparency property found in fuzzy systems, but use, for their design, algorithms introduced in the context of neural networks. A new methodology, based on the minimization of the integral of the error, and exploiting the parameter separability property typically found in neuro-fuzzy systems, is proposed for parameter estimation. A recent evolutionary technique (bacterial algorithms), based on the natural phenomenon of microbial evolution, is combined with genetic programming, and the resulting algorithm, bacterial programming, advocated for structure determination. Different versions of this evolutionary technique are combined with gradient-based algorithms, solving problems found in fuzzy and neuro-fuzzy design, namely incorporation of a-priori knowledge, gradient algorithms initialization and model complexity reduction.Todos os sistemas encontrados na natureza exibem, com maior ou menor grau, um comportamento linear. De modo a emular esse comportamento, as técnicas de identificação clássicas usam, tipicamente e por simplicidade matemática, modelos lineares. Devido à sua propriedade de aproximação universal, modelos inspirados por princípios biológicos (redes neuronais artificiais) e motivados linguisticamente (sistemas difusos) tem sido cada vez mais usados como alternativos aos modelos matemáticos clássicos. Num contexto de identificação de sistemas, o projeto de modelos como os acima descritos é um processo iterativo, constituído por vários passos. Dentro destes, encontra-se a necessidade de identificar a estrutura do modelo a usar, e a estimação dos seus parâmetros. Esta Tese discutirá a aplicação de algoritmos baseados em derivadas para a fase de estimação de parâmetros, e o uso de algoritmos baseados na teoria da evolução de espécies, algoritmos evolutivos, para a seleção de estrutura do modelo. Isto será realizado no contexto do projeto de modelos neuro-difusos, isto é, modelos que simultaneamente exibem a propriedade de transparência normalmente associada a sistemas difusos mas que utilizam, para o seu projeto algoritmos introduzidos no contexto de redes neuronais. Os modelos utilizados neste trabalho são redes B-Spline, de Função de Base Radial, e sistemas difusos dos tipos Mamdani e Takagi-Sugeno. Neste trabalho começa-se por explorar, para desenho de redes B-Spline, a introdução de conhecimento à-priori existente sobre um processo. Neste sentido, aplica-se uma nova abordagem na qual a técnica para a estimação dos parâmetros é alterada a fim de assegurar restrições de igualdade da função e das suas derivadas. Mostra-se ainda que estratégias de determinação de estrutura do modelo, baseadas em computação evolutiva ou em heurísticas determinísticas podem ser facilmente adaptadas a este tipo de modelos restringidos. É proposta uma nova técnica evolutiva, resultante da combinação de algoritmos recentemente introduzidos (algoritmos bacterianos, baseados no fenómeno natural de evolução microbiana) e programação genética. Nesta nova abordagem, designada por programação bacteriana, os operadores genéticos são substituídos pelos operadores bacterianos. Deste modo, enquanto a mutação bacteriana trabalha num indivíduo, e tenta otimizar a bactéria que o codifica, a transferência de gene é aplicada a toda a população de bactérias, evitando-se soluções de mínimos locais. Esta heurística foi aplicada para o desenho de redes B-Spline. O desempenho desta abordagem é ilustrada e comparada com alternativas existentes. Para a determinação dos parâmetros de um modelo são normalmente usadas técnicas de otimização locais, baseadas em derivadas. Como o modelo em questão é não-linear, o desempenho deste género de técnicas é influenciado pelos pontos de partida. Para resolver este problema, é proposto um novo método no qual é usado o algoritmo evolutivo referido anteriormente para determinar pontos de partida mais apropriados para o algoritmo baseado em derivadas. Deste modo, é aumentada a possibilidade de se encontrar um mínimo global. A complexidade dos modelos neuro-difusos (e difusos) aumenta exponencialmente com a dimensão do problema. De modo a minorar este problema, é proposta uma nova abordagem de particionamento do espaço de entrada, que é uma extensão das estratégias de decomposição de entrada normalmente usadas para este tipo de modelos. Simulações mostram que, usando esta abordagem, se pode manter a capacidade de generalização com modelos de menor complexidade. Os modelos B-Spline são funcionalmente equivalentes a modelos difusos, desde que certas condições sejam satisfeitas. Para os casos em que tal não acontece (modelos difusos Mamdani genéricos), procedeu-se à adaptação das técnicas anteriormente empregues para as redes B-Spline. Por um lado, o algoritmo Levenberg-Marquardt é adaptado e a fim de poder ser aplicado ao particionamento do espaço de entrada de sistema difuso. Por outro lado, os algoritmos evolutivos de base bacteriana são adaptados para sistemas difusos, e combinados com o algoritmo de Levenberg-Marquardt, onde se explora a fusão das características de cada metodologia. Esta hibridização dos dois algoritmos, denominada de algoritmo bacteriano memético, demonstrou, em vários problemas de teste, apresentar melhores resultados que alternativas conhecidas. Os parâmetros dos modelos neuronais utilizados e dos difusos acima descritos (satisfazendo no entanto alguns critérios) podem ser separados, de acordo com a sua influência na saída, em parâmetros lineares e não-lineares. Utilizando as consequências desta propriedade nos algoritmos de estimação de parâmetros, esta Tese propõe também uma nova metodologia para estimação de parâmetros, baseada na minimização do integral do erro, em alternativa à normalmente utilizada minimização da soma do quadrado dos erros. Esta técnica, além de possibilitar (em certos casos) um projeto totalmente analítico, obtém melhores resultados de generalização, dado usar uma superfície de desempenho mais similar aquela que se obteria se se utilizasse a função geradora dos dados