NSF CAREER: Scalable Learning and Adaptation with Intelligent Techniques and Neural Networks for Reconfiguration and Survivability of Complex Systems

The NSF CAREER program is a premier program that emphasizes the importance the foundation places on the early development of academic careers solely dedicated to stimulating the discovery process in which the excitement of research enriched by inspired teaching and enthusiastic learning. This paper describes the research and education experiences gained by the principal investigator and his research collaborators and students as a result of a NSF CAREER proposal been awarded by the power, control and adaptive networks (PCAN) program of the electrical, communications and cyber systems division, effective June 1, 2004. In addition, suggestions on writing a winning NSF CAREER proposal are presented

MIMO Beam-Forming with Neural Network Channel Prediction Trained by a Novel PSO-EA-DEPSO Algorithm

A new hybrid algorithm based on particle swarm optimization (PSO), evolutionary algorithm (EA), and differential evolution (DE) is presented for training a recurrent neural network (RNN) for multiple-input multiple-output (MIMO) channel prediction. The hybrid algorithm is shown to be superior in performance to PSO and differential evolution PSO (DEPSO) for different channel environments. The received signal-to-noise ratio is derived for un-coded and beam-forming MIMO systems to see how the RNN error affects the performance. This error is shown to deteriorate the accuracy of the weak singular modes, making beam-forming more desirable. Bit error rate simulations are performed to validate these results

A Recurrent Neural Network Survival Model: Predicting Web User Return Time

The size of a website's active user base directly affects its value. Thus, it is important to monitor and influence a user's likelihood to return to a site. Essential to this is predicting when a user will return. Current state of the art approaches to solve this problem come in two flavors: (1) Recurrent Neural Network (RNN) based solutions and (2) survival analysis methods. We observe that both techniques are severely limited when applied to this problem. Survival models can only incorporate aggregate representations of users instead of automatically learning a representation directly from a raw time series of user actions. RNNs can automatically learn features, but can not be directly trained with examples of non-returning users who have no target value for their return time. We develop a novel RNN survival model that removes the limitations of the state of the art methods. We demonstrate that this model can successfully be applied to return time prediction on a large e-commerce dataset with a superior ability to discriminate between returning and non-returning users than either method applied in isolation.Comment: Accepted into ECML PKDD 2018; 8 figures and 1 tabl

Time Series Forecasting for Outdoor Temperature using Nonlinear Autoregressive Neural Network Models

Weather forecasting is a challenging time series forecasting problem because of its dynamic, continuous, data-intensive, chaotic and irregular behavior. At present, enormous time series forecasting techniques exist and are widely adapted. However, competitive research is still going on to improve the methods and techniques for accurate forecasting. This research article presents the time series forecasting of the metrological parameter, i.e., temperature with NARX (Nonlinear Autoregressive with eXogenous input) based ANN (Artificial Neural Network). In this research work, several time series dependent Recurrent NARX-ANN models are developed and trained with dynamic parameter settings to find the optimum network model according to its desired forecasting task. Network performance is analyzed on the basis of its Mean Square Error (MSE) value over training, validation and test data sets. In order to perform the forecasting for next 4,8 and 12 steps horizon, the model with less MSE is chosen to be the most accurate temperature forecaster. Unlike one step ahead prediction, multi-step ahead forecasting is more difficult and challenging problem to solve due to its underlying additional complexity. Thus, the empirical findings in this work provide valuable suggestions for the parameter settings of NARX model specifically the selection of hidden layer size and autoregressive lag terms in accordance with an appropriate multi-step ahead time series forecasting

Compressor valve failure detection and prognostics

Reciprocating compressors are commonly used machinery for industrial applications. Unscheduled downtime and maintenance activity on the compressors causes considerable loss in throughput and efficiency of a plant. Of all the failures that cause unscheduled downtime in reciprocating compressors, valve related causes are predominant. Most of the failures associated with the valves are tracked to the failure of moving elements within the valve. Achieving higher reliability of critical reciprocating systems requires continuously monitoring the system and performing dynamic analysis of the sensory data for valve fault diagnosis. Continuous monitoring will improve the time and cost to repair through keeping a constant vigil for failure events. Though there has been a good amount of work done for condition monitoring of compressors, there has been very little work on detecting and predicting valve failures. The objective of this thesis is to research detection and prediction of valve failures by wavelet analysis, logistic regression and neural network analysis of pressure and temperature signals, which are the most common measurements on a reciprocating compressor system. Valve failures are seeded on a reciprocating compressor testbed that is instrumented with only temperature and pressure sensor order emulate the reciprocating compressor systems used in the industry. The parameters are measured on a continuous basis and baselines are established for normal (or acceptable) behavior and failure (or fault) condition. Deviation of the system from the normal condition and the time for the system to reach the fault mode is quantified with the help of the above mentioned tools. --Abstract, page iii

Applications of swarm, evolutionary and quantum algorithms in system identification and digital filter design

The thesis focuses on the application of computational intelligence (CI) techniques for two problems - system identification and digital filter design. In system identification, different case studies have been carried out with equal or reduced number of orders as the original system and also in identifying a blackbox model. Lowpass, Highpass, Bandpass and Bandstop FIR and Lowpass IIR filters have been designed using three algorithms using two different fitness functions. Particle Swarm Optimization (PSO), Differential Evolution based PSO (DEPSO) and PSO with Quantum Infusion (PSO-QI) algorithms have been applied in this work --Abstract, page iii

Multiple-input multiple-output wireless communications with imperfect channel knowledge

In the first work a recurrent neural network (RNN) is employed for MIMO channel prediction. A novel PSO-EA-DEPSO off-line training algorithm is presented and is shown to outperform PSO, PSO-EA, and DEPSO. This predictor is shown to be robust to varying channel scenarios. New expressions for the received SNR, array gain, average probability of error, and diversity gain are derived. Next, a new expression for the outage capacity of a MIMO system with no CSI at the transmitter and an estimate at the receiver is presented. Since the outage capacity is a function of the first and second moments of the mutual information, new closed form approximations are derived at low and high effective SNR. Also at low effective SNR a new result for the outage capacity is presented. Finally, the outage capacity for a frequency selective channel is derived. This is followed by a MIMO RNN predictor that operates online. A single RNN is constructed to predict all of the MIMO sub-channels instantaneously. The extended Kalman filter (EKF) and real-time recurrent learning (RTRL) algorithms are applied to compare the MSE of the prediction error. A new expression for the channel estimation error of a continuously varying MIMO channel is derived next. The optimal amount of time to send training pilots is investigated for different channel scenarios. Special cases of the new expression for the channel estimation error lead to previously established results. The last work investigates the performance of a MIMO aeronautical system in a two- ray ground reflection scenario. The ergodic capacity is analyzed when the altitude, horizontal displacement, antenna separation, and aircraft velocity are varied --Abstract, page iv

Combinations of time series forecasts: when and why are they beneficial?.

Time series forecasting has a long track record in many application areas. In forecasting research, it has been illustrated that finding an individual algorithm that works best for all possible scenarios is hopeless. Therefore, instead of striving to design a single superior algorithm, current research efforts have shifted towards gaining a deeper understanding of the reasons a forecasting method may perform well in some conditions whilst it may fail in others. This thesis provides a number of contributions to this matter. Traditional empirical evaluations are discussed from a novel point of view, questioning the benefit of using sophisticated forecasting methods without domain knowledge. An own empirical study focusing on relevant off-the shelf forecasting and forecast combination methods underlines the competitiveness of relatively simple methods in practical applications. Furthermore, meta-features of time series are extracted to automatically find and exploit a link between application specific data characteristics and forecasting performance using meta-learning. Finally, the approach of extending the set of input forecasts by diversifying functional approaches, parameter sets and data aggregation level used for learning is discussed, relating characteristics of the resulting forecasts to different error decompositions for both individual methods and combinations. Advanced combination structures are investigated in order to take advantage of the knowledge on the forecast generation processes. Forecasting is a crucial factor in airline revenue management; forecasting of the anticipated booking, cancellation and no-show numbers has a direct impact on general planning of routes and schedules, capacity control for fareclasses and overbooking limits. In a collaboration with Lufthansa Systems in Berlin, experiments in the thesis are conducted on an airline data set with the objective of improving the current net booking forecast by modifying one of its components, the cancellation forecast. To also compare results achieved of the methods investigated here with the current state-of-the-art in forecasting research, some experiments also use data sets of two recent forecasting competitions, thus being able to provide a link between academic research and industrial practice