Learning from Long-Tailed Noisy Data with Sample Selection and Balanced Loss

The success of deep learning depends on large-scale and well-curated training data, while data in real-world applications are commonly long-tailed and noisy. Many methods have been proposed to deal with long-tailed data or noisy data, while a few methods are developed to tackle long-tailed noisy data. To solve this, we propose a robust method for learning from long-tailed noisy data with sample selection and balanced loss. Specifically, we separate the noisy training data into clean labeled set and unlabeled set with sample selection, and train the deep neural network in a semi-supervised manner with a novel balanced loss based on model bias. Experiments on benchmarks demonstrate that our method outperforms existing state-of-the-art methods

Application of neural network to study share price volatility.

by Lam King Wan.Thesis (M.B.A.)--Chinese University of Hong Kong, 1999.Includes bibliographical references (leaves 72-73).ABSTRACT --- p.ii.TABLE OF CONTENTS --- p.iv.SectionChapter I. --- OBJECTIVE --- p.1Chapter II. --- INTRODUCTION --- p.3The principal investment risk --- p.3Effect of risk on investment --- p.4Investors' concern for investment risk --- p.6Chapter III. --- THE INPUT PARAMETERS --- p.9Chapter IV. --- LITERATURE REVIEW --- p.15What is an artificial neural network? --- p.15What is a neuron? --- p.16Biological versus artificial neuron --- p.16Operation of a neural network --- p.17Neural network paradigm --- p.20Feedforward as the most suitable form of neural network --- p.22Capability of neural network --- p.23The learning process --- p.25Testing the network --- p.29Neural network computing --- p.29Neural network versus conventional computer --- p.30Neural network versus a knowledge based system --- p.32Strength of neural network --- p.34Weaknesses of neural network --- p.35Chapter V. --- NEURAL NETWORK AS A TOOL FOR INVESTMENT ANALYSIS --- p.38Neural network in financial applications --- p.38Trading in the stock market --- p.41Why neural network could outperform in the stock market? --- p.43Applications of neural network --- p.45Chapter VI. --- BUILDING THE NEURAL NETWORK MODEL --- p.47Implementation process --- p.48Step 1´ؤ Problem specification --- p.49Step 2 ´ؤ Data collection --- p.51Step 3 ´ؤ Data analysis and transformation --- p.55Step 4 ´ؤ Training data set extraction --- p.58Step 5 ´ؤ Selection of network architecture --- p.60Step 6 ´ؤ Selection of training algorithm --- p.62Step 7 ´ؤ Training the network --- p.64Step 8 ´ؤ Model deployment --- p.65Chapter 7 --- RESULT AND CONCLUSION --- p.67Result --- p.67Conclusion --- p.69BIBLIOGRAPHY --- p.7

Multi-criteria Evolution of Neural Network Topologies: Balancing Experience and Performance in Autonomous Systems

Majority of Artificial Neural Network (ANN) implementations in autonomous systems use a fixed/user-prescribed network topology, leading to sub-optimal performance and low portability. The existing neuro-evolution of augmenting topology or NEAT paradigm offers a powerful alternative by allowing the network topology and the connection weights to be simultaneously optimized through an evolutionary process. However, most NEAT implementations allow the consideration of only a single objective. There also persists the question of how to tractably introduce topological diversification that mitigates overfitting to training scenarios. To address these gaps, this paper develops a multi-objective neuro-evolution algorithm. While adopting the basic elements of NEAT, important modifications are made to the selection, speciation, and mutation processes. With the backdrop of small-robot path-planning applications, an experience-gain criterion is derived to encapsulate the amount of diverse local environment encountered by the system. This criterion facilitates the evolution of genes that support exploration, thereby seeking to generalize from a smaller set of mission scenarios than possible with performance maximization alone. The effectiveness of the single-objective (optimizing performance) and the multi-objective (optimizing performance and experience-gain) neuro-evolution approaches are evaluated on two different small-robot cases, with ANNs obtained by the multi-objective optimization observed to provide superior performance in unseen scenarios

Learning icons appearance similarity

Selecting an optimal set of icons is a crucial step in the pipeline of visual design to structure and navigate through content. However, designing the icons sets is usually a difficult task for which expert knowledge is required. In this work, to ease the process of icon set selection to the users, we propose a similarity metric which captures the properties of style and visual identity. We train a Siamese Neural Network with an online dataset of icons organized in visually coherent collections that are used to adaptively sample training data and optimize the training process. As the dataset contains noise, we further collect human-rated information on the perception of icon's similarity which will be used for evaluating and testing the proposed model. We present several results and applications based on searches, kernel visualizations and optimized set proposals that can be helpful for designers and non-expert users while exploring large collections of icons.Comment: 12 pages, 11 figure

Neural Network Based Inferential Model For Ethane Steam Cracking Furnace

The product yield distribution of ethane steam cracking is typically obtained using analysers and lab sampling. Since both methods take time to produce results, primarily depending on them to determine main product yield will hinder immediate control action on the process. In order to resolve this issue, an inferential sensor is required. In this study, a neural network based inferential model is developed. The ethane steam cracking process has been modelled using ASPEN Plus and validated with industrial data taken from literature. The relative error (RE) of the model outputs obtained are less than 10%. The ASPEN Plus model is used for input variable selection, nonlinearity assessment, and data generation for neural network modelling. The input variable selection study found that five variables are significantly influential to the ethane and ethylene yields, namely reactor pressure, coil outlet temperature, steam-hydrocarbon ratio, feed composition, and fuel composition. Nonlinearity assessment of the process shows that the process exhibit asymmetrical response and input multiplicities characteristics, and thus, can be classified as a nonlinear process. Data generated from the ASPEN Plus model is used for training, validation, and testing. Two methods have been used to generate the data which are sequential excitation and simultaneous excitation. Four variables are individually excited and combined to make a sequential excitation profile. Data from sequential excitation is divided into training and validation while data from simultaneous excitation is used solely for testing. Three neural network model, namely the Feedforward Neural Network (FFNN), the Generalized Regression Neural Network (GRNN), and the Extreme Learning Machine Neural Network (ELM-NN) are developed and they are evaluated in terms of prediction accuracy and computational time. The evaluation results show that ELM-NN prediction accuracy is higher than FFNN and GRNN. To train, the best model for ELM-NN, GRNN, and FFNN models require 0.0068 seconds, 0.35 seconds, and 12 seconds respectively. In terms of computation time of new set of input data sample, all three models require less than 0.05 seconds to compute one sample of data. However, computation time of the trained GRNN model increases exponentially with the increasing amount of data samples in a batch while for trained FFNN and trained ELM-NN model, the increment is not significant. Out of the three models, the ELM-NN gives the best performance in terms of prediction accuracy and computational time. The R2 of the ELM-NN model is 91.3% and 82.6% for ethane and ethylene yield respectively. The model requires 0.0068 seconds to train and 0.0001 seconds to compute ethane yield and ethylene yields from a new set of input data. This makes the model suitable for applications in real time inferential control system

Would credit scoring work for Islamic finance? A neural network approach

Purpose – The main aim of this paper is to distinguish whether the decision making process of the Islamic financial houses in the UK can be improved through the use of credit scoring modeling techniques as opposed to the currently used judgmental approaches. Subsidiary aims are to identify how scoring models can reclassify accepted applicants who later are considered as having bad credit and how many of the rejected applicants are later considered as having good credit; and highlight significant variables that are crucial in terms of accepting and rejecting applicants which can further aid the decision making process. Design/methodology/approach – A real data-set of 487 applicants are used consisting of 336 accepted credit applications and 151 rejected credit applications make to an Islamic finance house in the UK. In order to build the proposed scoring models, the data-set is divided into training and hold-out sub-set. The training sub-set is used to build the scoring models and the hold-out sub-set is used to test the predictive capabilities of the scoring models.70 percent of the overall applicants will be used for the training sub-set and 30 percent will be used for the testing sub-set. Three statistical modeling techniques namely Discriminant Analysis (DA), Logistic Regression (LR) and Multi-layer Perceptron (MP) neural network are used to build the proposed scoring models. Findings – Our findings reveal that the LR model has the highest Correct Classification (CC) rate in the training sub-set whereas MP outperforms other techniques and has the highest CC rate in the hold-out sub-set. MP also outperforms other techniques in terms of predicting the rejected credit applications and has the lowest Misclassification Cost (MC) above other techniques. In addition, results from MP models show that monthly expenses, age and marital status are identified as the key factors affecting the decision making process. Research limitations/implications – Although our sample is small and restricted to an Islamic Finance house in the UK the results are robust. Future research could consider enlarging the sample in the UK and also internationally allowing for cultural differences to be identified. The results indicate that the scoring models can be of great benefit to Islamic finance houses in regards to their decision making processes of accepting and rejecting new credit applications and thus improve their efficiency and effectiveness. Originality/value –Our contribution is the first to apply credit scoring modeling techniques in Islamic Finance. Also in building a scoring model our application applies a different approach by using accepted and rejected credit applications instead of good and bad credit histories. This identifies opportunity costs of misclassifying credit applications as rejected

A Very Brief Introduction to Machine Learning With Applications to Communication Systems

Given the unprecedented availability of data and computing resources, there is widespread renewed interest in applying data-driven machine learning methods to problems for which the development of conventional engineering solutions is challenged by modelling or algorithmic deficiencies. This tutorial-style paper starts by addressing the questions of why and when such techniques can be useful. It then provides a high-level introduction to the basics of supervised and unsupervised learning. For both supervised and unsupervised learning, exemplifying applications to communication networks are discussed by distinguishing tasks carried out at the edge and at the cloud segments of the network at different layers of the protocol stack

A selective learning method to improve the generalization of multilayer feedforward neural networks.

Multilayer feedforward neural networks with backpropagation algorithm have been used successfully in many applications. However, the level of generalization is heavily dependent on the quality of the training data. That is, some of the training patterns can be redundant or irrelevant. It has been shown that with careful dynamic selection of training patterns, better generalization performance may be obtained. Nevertheless, generalization is carried out independently of the novel patterns to be approximated. In this paper, we present a learning method that automatically selects the training patterns more appropriate to the new sample to be predicted. This training method follows a lazy learning strategy, in the sense that it builds approximations centered around the novel sample. The proposed method has been applied to three different domains: two artificial approximation problems and a real time series prediction problem. Results have been compared to standard backpropagation using the complete training data set and the new method shows better generalization abilities.Publicad