Individual and ensemble functional link neural networks for data classification

This study investigated the Functional Link Neural Network (FLNN) for solving data classification problems. FLNN based models were developed using evolutionary methods as well as ensemble methods. The outcomes of the experiments covering benchmark classification problems, positively demonstrated the efficacy of the proposed models for undertaking data classification problems

Thirty Years of Machine Learning: The Road to Pareto-Optimal Wireless Networks

Future wireless networks have a substantial potential in terms of supporting a broad range of complex compelling applications both in military and civilian fields, where the users are able to enjoy high-rate, low-latency, low-cost and reliable information services. Achieving this ambitious goal requires new radio techniques for adaptive learning and intelligent decision making because of the complex heterogeneous nature of the network structures and wireless services. Machine learning (ML) algorithms have great success in supporting big data analytics, efficient parameter estimation and interactive decision making. Hence, in this article, we review the thirty-year history of ML by elaborating on supervised learning, unsupervised learning, reinforcement learning and deep learning. Furthermore, we investigate their employment in the compelling applications of wireless networks, including heterogeneous networks (HetNets), cognitive radios (CR), Internet of things (IoT), machine to machine networks (M2M), and so on. This article aims for assisting the readers in clarifying the motivation and methodology of the various ML algorithms, so as to invoke them for hitherto unexplored services as well as scenarios of future wireless networks.Comment: 46 pages, 22 fig

Interval and Fuzzy Computing in Neural Network for System Identification Problems

Increase of population and growing of societal and commercial activities with limited land available in a modern city leads to construction up of tall/high-rise buildings. As such, it is important to investigate about the health of the structure after the occurrence of manmade or natural disasters such as earthquakes etc. A direct mathematical expression for parametric study or system identification of these structures is not always possible. Actually System Identification (SI) problems are inverse vibration problems consisting of coupled linear or non-linear differential equations that depend upon the physics of the system. It is also not always possible to get the solutions for these problems by classical methods. Few researchers have used different methods to solve the above mentioned problems. But difficulties are faced very often while finding solution to these problems because inverse problem generally gives non-unique parameter estimates. To overcome these difficulties alternate soft computing techniques such as Artificial Neural Networks (ANNs) are being used by various researchers to handle the above SI problems. It is worth mentioning that traditional neural network methods have inherent advantage because it can model the experimental data (input and output) where good mathematical model is not available. Moreover, inverse problems have been solved by other researchers for deterministic cases only. But while performing experiments it is always not possible to get the data exactly in crisp form. There may be some errors that are due to involvement of human or experiment. Accordingly, those data may actually be in uncertain form and corresponding methodologies need to be developed. It is an important issue about dealing with variables, parameters or data with uncertain value. There are three classes of uncertain models, which are probabilistic, fuzzy and interval. Recently, fuzzy theory and interval analysis are becoming powerful tools for many applications in recent decades. It is known that interval and fuzzy computations are themselves very complex to handle. Having these in mind one has to develop efficient computational models and algorithms very carefully to handle these uncertain problems. As said above, in general we may not obtain the corresponding input and output values (experimental) exactly or in crisp form but we may have only uncertain information of the data. Hence, investigations are needed to handle the SI problems where data is available in uncertain form. Identification methods with crisp (exact) data are known and traditional neural network methods have already been used by various researchers. But when the data are in uncertain form then traditional ANN may not be applied. Accordingly, new ANN models need to be developed which may solve the targeted uncertain SI problems. Hence present investigation targets to develop powerful methods of neural network based on interval and fuzzy theory for the analysis and simulation with respect to the uncertain system identification problems. In this thesis, these uncertain data are assumed as interval and fuzzy numbers. Accordingly, identification methodologies are developed for multistorey shear buildings by proposing new models of Interval Neural Network (INN) and Fuzzy Neural Network (FNN) models which can handle interval and fuzzified data respectively. It may however be noted that the developed methodology not only be important for the mentioned problems but those may very well be used in other application problems too. Few SI problems have been solved in the present thesis using INN and FNN model which are briefly described below. From initial design parameters (namely stiffness and mass in terms of interval and fuzzy) corresponding design frequencies may be obtained for a given structural problem viz. for a multistorey shear structure. The uncertain (interval/fuzzy) frequencies may then be used to estimate the present structural parameter values by the proposed INN and FNN. Next, the identification has been done using vibration response of the structure subject to ambient vibration with interval/fuzzy initial conditions. Forced vibration with horizontal displacement in interval/fuzzified form has also been used to investigate the identification problem. Moreover this study involves SI problems of structures (viz. shear buildings) with respect to earthquake data in order to know the health of a structure. It is well known that earthquake data are both positive and negative. The Interval Neural Network and Fuzzy Neural Network model may not handle the data with negative sign due to the complexity in interval and fuzzy computation. As regards, a novel transformation method have been developed to compute response of a structural system by training the model for Indian earthquakes at Chamoli and Uttarkashi using uncertain (interval/fuzzified) ground motion data. The simulation may give an idea about the safety of the structural system in case of future earthquakes. Further a single layer interval and fuzzy neural network based strategy has been proposed for simultaneous identification of the mass, stiffness and damping of uncertain multi-storey shear buildings using series/cluster of neural networks. It is known that training in MNN and also in INN and FNN are time consuming because these models depend upon the number of nodes in the hidden layer and convergence of the weights during training. As such, single layer Functional Link Neural Network (FLNN) with multi-input and multi-output model has also been proposed to solve the system identification problems for the first time. It is worth mentioning that, single input single output FLNN had been proposed by previous authors. In FLNN, the hidden layer is replaced by a functional expansion block for enhancement of the input patterns using orthogonal polynomials such as Chebyshev, Legendre and Hermite, etc. The computations become more efficient than the traditional or classical multi-layer neural network due to the absence of hidden layer. FLNN has also been used for structural response prediction of multistorey shear buildings subject to earthquake ground motion. It is seen that FLNN can very well predict the structural response of different floors of multi-storey shear building subject to earthquake data. Comparison of results among Multi layer Neural Network (MNN), Chebyshev Neural Network (ChNN), Legendre Neural Network (LeNN), Hermite Neural Network (HNN) and desired are considered and it is found that Functional Link Neural Network models are more effective and takes less computation time than MNN. In order to show the reliability, efficacy and powerfulness of INN, FNN and FLNN models variety of problems have been solved here. Finally FLNN is also extended to interval based FLNN which is again proposed for the first time to the best of our knowledge. This model is implemented to estimate the uncertain stiffness parameters of a multi-storey shear building. The parameters are identified here using uncertain response of the structure subject to ambient and forced vibration with interval initial condition and horizontal displacement also in interval form

Ny forståelse av gasshydratfenomener og naturlige inhibitorer i råoljesystemer gjennom massespektrometri og maskinlæring

Gas hydrates represent one of the main flow assurance issues in the oil and gas industry as they can cause complete blockage of pipelines and process equipment, forcing shut downs. Previous studies have shown that some crude oils form hydrates that do not agglomerate or deposit, but remain as transportable dispersions. This is commonly believed to be due to naturally occurring components present in the crude oil, however, despite decades of research, their exact structures have not yet been determined. Some studies have suggested that these components are present in the acid fractions of the oils or are related to the asphaltene content of the oils. Crude oils are among the worlds most complex organic mixtures and can contain up to 100 000 different constituents, making them difficult to characterise using traditional mass spectrometers. The high mass accuracy of Fourier Transform Ion Cyclotron Resonance Mass Spectrometry (FT-ICR MS) yields a resolution greater than traditional techniques, making FT-ICR MS able to characterise crude oils to a greater extent, and possibly identify hydrate active components. FT-ICR MS spectra usually contain tens of thousands of peaks, and data treatment methods able to find underlying relationships in big data sets are required. Machine learning and multivariate statistics include many methods suitable for big data. A literature review identified a number of promising methods, and the current status for the use of machine learning for analysis of gas hydrates and FT-ICR MS data was analysed. The literature study revealed that although many studies have used machine learning to predict thermodynamic properties of gas hydrates, very little work have been done in analysing gas hydrate related samples measured by FT-ICR MS. In order to aid their identification, a successive accumulation procedure for increasing the concentrations of hydrate active components was developed by SINTEF. Comparison of the mass spectra from spiked and unspiked samples revealed some peaks that increased in intensity over the spiking levels. Several classification methods were used in combination with variable selection, and peaks related to hydrate formation were identified. The corresponding molecular formulas were determined, and the peaks were assumed to be related to asphaltenes, naphthenes and polyethylene glycol. To aid the characterisation of the oils, infrared spectroscopy (both Fourier Transform infrared and near infrared) was combined with FT-ICR MS in a multiblock analysis to predict the density of crude oils. Two different strategies for data fusion were attempted, and sequential fusion of the blocks achieved the highest prediction accuracy both before and after reducing the dimensions of the data sets by variable selection. As crude oils have such complex matrixes, samples are often very different, and many methods are not able to handle high degrees of variations or non-linearities between the samples. Hierarchical cluster-based partial least squares regression (HC-PLSR) clusters the data and builds local models within each cluster. HC-PLSR can thus handle non-linearities between clusters, but as PLSR is a linear model the data is still required to be locally linear. HC-PLSR was therefore expanded into deep learning (HC-CNN and HC-RNN) and SVR (HC-SVR). The deep learning-based models outperformed HC-PLSR for a data set predicting average molecular weights from hydrolysed raw materials. The analysis of the FT-ICR MS spectra revealed that the large amounts of information contained in the data (due to the high resolution) can disturb the predictive models, but the use of variable selection counteracts this effect. Several methods from machine learning and multivariate statistics were proven valuable for prediction of various parameters from FT-ICR MS using both classification and regression methods.Gasshydrater er et av hovedproblemene for Flow assurance i olje- og gassnæringen ettersom at de kan forårsake blokkeringer i oljerørledninger og prosessutstyr som krever at systemet må stenges ned. Tidligere studier har vist at noen råoljer danner hydrater som ikke agglomererer eller avsetter, men som forblir som transporterbare dispersjoner. Dette antas å være på grunn av naturlig forekommende komponenter til stede i råoljen, men til tross for årevis med forskning er deres nøyaktige strukturer enda ikke bestemt i detalj. Noen studier har indikert at disse komponentene kan stamme fra syrefraksjonene i oljen eller være relatert til asfalteninnholdet i oljene. Råoljer er blant verdens mest komplekse organiske blandinger og kan inneholde opptil 100 000 forskjellige bestanddeler, som gjør dem vanskelig å karakterisere ved bruk av tradisjonelle massespektrometre. Den høye masseoppløsningen Fourier-transform ion syklotron resonans massespektrometri (FT-ICR MS) gir en høyere oppløsning enn tradisjonelle teknikker, som gjør FT-ICR MS i stand til å karakterisere råoljer i større grad og muligens identifisere hydrataktive komponenter. FT-ICR MS spektre inneholder vanligvis titusenvis av topper, og det er nødvendig å bruke databehandlingsmetoder i stand til å håndtere store datasett, med muligheter til å finne underliggende forhold for å analysere spektrene. Maskinlæring og multivariat statistikk har mange metoder som er passende for store datasett. En litteratur studie identifiserte flere metoder og den nåværende statusen for bruken av maskinlæring for analyse av gasshydrater og FT-ICR MS data. Litteraturstudien viste at selv om mange studier har brukt maskinlæring til å predikere termodynamiske egenskaper for gasshydrater, har lite arbeid blitt gjort med å analysere gasshydrat relaterte prøver målt med FT-ICR MS. For å bistå identifikasjonen ble en suksessiv akkumuleringsprosedyre for å øke konsentrasjonene av hydrataktive komponenter utviklet av SINTEF. Sammenligninger av massespektrene fra spikede og uspikede prøver viste at noen topper økte sammen med spikingnivåene. Flere klassifikasjonsmetoder ble brukt i kombinasjon med ariabelseleksjon for å identifisere topper relatert til hydratformasjon. Molekylformler ble bestemt og toppene ble antatt å være relatert til asfaltener, naftener og polyetylenglykol. For å bistå karakteriseringen av oljene ble infrarød spektroskopi inkludert med FT-ICR MS i en multiblokk analyse for å predikere tettheten til råoljene. To forskjellige strategier for datafusjonering ble testet og sekvensiell fusjonering av blokkene oppnådde den høyeste prediksjonsnøyaktigheten både før og etter reduksjon av datasettene med bruk av variabelseleksjon. Ettersom råoljer har så kompleks sammensetning, er prøvene ofte veldig forskjellige og mange metoder er ikke egnet for å håndtere store variasjoner eller ikke-lineariteter mellom prøvene. Hierarchical cluster-based partial least squares regression (HCPLSR) grupperer dataene og lager lokale modeller for hver gruppe. HC-PLSR kan dermed håndtere ikke-lineariteter mellom gruppene, men siden PLSR er en lokal modell må dataene fortsatt være lokalt lineære. HC-PLSR ble derfor utvidet til convolutional neural networks (HC-CNN) og recurrent neural networks (HC-RNN) og support vector regression (HC-SVR). Disse dyp læring metodene utkonkurrerte HC-PLSR for et datasett som predikerte gjennomsnittlig molekylvekt fra hydrolyserte råmaterialer. Analysen av FT-ICR MS spektre viste at spektrene inneholder veldig mye informasjon. Disse store mengdene med data kan forstyrre prediksjonsmodeller, men bruken av variabelseleksjon motvirket denne effekten. Flere metoder fra maskinlæring og multivariat statistikk har blitt vist å være nyttige for prediksjon av flere parametere from FT-ICR MS data ved bruk av både klassifisering og regresjon

Adaptive Algorithms for Intelligent Acoustic Interfaces

Modern speech communications are evolving towards a new direction which involves users in a more perceptive way. That is the immersive experience, which may be considered as the “last-mile” problem of telecommunications. One of the main feature of immersive communications is the distant-talking, i.e. the hands-free (in the broad sense) speech communications without bodyworn or tethered microphones that takes place in a multisource environment where interfering signals may degrade the communication quality and the intelligibility of the desired speech source. In order to preserve speech quality intelligent acoustic interfaces may be used. An intelligent acoustic interface may comprise multiple microphones and loudspeakers and its peculiarity is to model the acoustic channel in order to adapt to user requirements and to environment conditions. This is the reason why intelligent acoustic interfaces are based on adaptive filtering algorithms. The acoustic path modelling entails a set of problems which have to be taken into account in designing an adaptive filtering algorithm. Such problems may be basically generated by a linear or a nonlinear process and can be tackled respectively by linear or nonlinear adaptive algorithms. In this work we consider such modelling problems and we propose novel effective adaptive algorithms that allow acoustic interfaces to be robust against any interfering signals, thus preserving the perceived quality of desired speech signals. As regards linear adaptive algorithms, a class of adaptive filters based on the sparse nature of the acoustic impulse response has been recently proposed. We adopt such class of adaptive filters, named proportionate adaptive filters, and derive a general framework from which it is possible to derive any linear adaptive algorithm. Using such framework we also propose some efficient proportionate adaptive algorithms, expressly designed to tackle problems of a linear nature. On the other side, in order to address problems deriving from a nonlinear process, we propose a novel filtering model which performs a nonlinear transformations by means of functional links. Using such nonlinear model, we propose functional link adaptive filters which provide an efficient solution to the modelling of a nonlinear acoustic channel. Finally, we introduce robust filtering architectures based on adaptive combinations of filters that allow acoustic interfaces to more effectively adapt to environment conditions, thus providing a powerful mean to immersive speech communications

Adaptive Algorithms for Intelligent Acoustic Interfaces

Modern speech communications are evolving towards a new direction which involves users in a more perceptive way. That is the immersive experience, which may be considered as the “last-mile” problem of telecommunications. One of the main feature of immersive communications is the distant-talking, i.e. the hands-free (in the broad sense) speech communications without bodyworn or tethered microphones that takes place in a multisource environment where interfering signals may degrade the communication quality and the intelligibility of the desired speech source. In order to preserve speech quality intelligent acoustic interfaces may be used. An intelligent acoustic interface may comprise multiple microphones and loudspeakers and its peculiarity is to model the acoustic channel in order to adapt to user requirements and to environment conditions. This is the reason why intelligent acoustic interfaces are based on adaptive filtering algorithms. The acoustic path modelling entails a set of problems which have to be taken into account in designing an adaptive filtering algorithm. Such problems may be basically generated by a linear or a nonlinear process and can be tackled respectively by linear or nonlinear adaptive algorithms. In this work we consider such modelling problems and we propose novel effective adaptive algorithms that allow acoustic interfaces to be robust against any interfering signals, thus preserving the perceived quality of desired speech signals. As regards linear adaptive algorithms, a class of adaptive filters based on the sparse nature of the acoustic impulse response has been recently proposed. We adopt such class of adaptive filters, named proportionate adaptive filters, and derive a general framework from which it is possible to derive any linear adaptive algorithm. Using such framework we also propose some efficient proportionate adaptive algorithms, expressly designed to tackle problems of a linear nature. On the other side, in order to address problems deriving from a nonlinear process, we propose a novel filtering model which performs a nonlinear transformations by means of functional links. Using such nonlinear model, we propose functional link adaptive filters which provide an efficient solution to the modelling of a nonlinear acoustic channel. Finally, we introduce robust filtering architectures based on adaptive combinations of filters that allow acoustic interfaces to more effectively adapt to environment conditions, thus providing a powerful mean to immersive speech communications