MixRL: Data Mixing Augmentation for Regression using Reinforcement Learning

Data augmentation is becoming essential for improving regression accuracy in critical applications including manufacturing and finance. Existing techniques for data augmentation largely focus on classification tasks and do not readily apply to regression tasks. In particular, the recent Mixup techniques for classification rely on the key assumption that linearity holds among training examples, which is reasonable if the label space is discrete, but has limitations when the label space is continuous as in regression. We show that mixing examples that either have a large data or label distance may have an increasingly-negative effect on model performance. Hence, we use the stricter assumption that linearity only holds within certain data or label distances for regression where the degree may vary by each example. We then propose MixRL, a data augmentation meta learning framework for regression that learns for each example how many nearest neighbors it should be mixed with for the best model performance using a small validation set. MixRL achieves these objectives using Monte Carlo policy gradient reinforcement learning. Our experiments conducted both on synthetic and real datasets show that MixRL significantly outperforms state-of-the-art data augmentation baselines. MixRL can also be integrated with other classification Mixup techniques for better results.Comment: 15 pages, 9 figures, 7 table

Threshold Effects of the Public Capital Productivity : An International Panel Smooth Transition Approach

Using a non linear panel data model we examine the threshold effects in the productivity of the public capital stocks for a panel of 21 OECD countries observed over 1965-2001. Using the so-called "augmented production function" approach, we estimate various specifications of a Panel Smooth Threshold Regression (PSTR) model recently developed by Gonzalez, Teräsvirta and Van Dijk (2004). One of our main results is the existence of strong threshold effects in the relationship between output and private and public inputs : whatever the transition mechanism specified, tests strongly reject the linearity assumption. Moreover this model allows cross-country heterogeneity and time instability of the productivity without specification of an ex-ante classification over individuals. Consequently it is posible to give estimates of productivity coefficients for both private and public capital stocks at any time and for each countries in the sample. Finally we proposed estimates of individual time varying elasticities that are much more reasonable than those previously published.Public Capital ; Panel Smooth ; Threshold Regression Models

Distance-based discriminant analysis method and its applications

This paper proposes a method of finding a discriminative linear transformation that enhances the data's degree of conformance to the compactness hypothesis and its inverse. The problem formulation relies on inter-observation distances only, which is shown to improve non-parametric and non-linear classifier performance on benchmark and real-world data sets. The proposed approach is suitable for both binary and multiple-category classification problems, and can be applied as a dimensionality reduction technique. In the latter case, the number of necessary discriminative dimensions can be determined exactly. Also considered is a kernel-based extension of the proposed discriminant analysis method which overcomes the linearity assumption of the sought discriminative transformation imposed by the initial formulation. This enhancement allows the proposed method to be applied to non-linear classification problems and has an additional benefit of being able to accommodate indefinite kernel

Audio Set classification with attention model: A probabilistic perspective

This paper investigates the classification of the Audio Set dataset. Audio Set is a large scale weakly labelled dataset of sound clips. Previous work used multiple instance learning (MIL) to classify weakly labelled data. In MIL, a bag consists of several instances, and a bag is labelled positive if at least one instances in the audio clip is positive. A bag is labelled negative if all the instances in the bag are negative. We propose an attention model to tackle the MIL problem and explain this attention model from a novel probabilistic perspective. We define a probability space on each bag, where each instance in the bag has a trainable probability measure for each class. Then the classification of a bag is the expectation of the classification output of the instances in the bag with respect to the learned probability measure. Experimental results show that our proposed attention model modeled by fully connected deep neural network obtains mAP of 0.327 on Audio Set dataset, outperforming the Google's baseline of 0.314 and recurrent neural network of 0.325.Comment: Accepted by ICASSP 201