9,461 research outputs found
An Ordered Lasso and Sparse Time-Lagged Regression
We consider regression scenarios where it is natural to impose an order
constraint on the coefficients. We propose an order-constrained version of
L1-regularized regression for this problem, and show how to solve it
efficiently using the well-known Pool Adjacent Violators Algorithm as its
proximal operator. The main application of this idea is time-lagged regression,
where we predict an outcome at time t from features at the previous K time
points. In this setting it is natural to assume that the coefficients decay as
we move farther away from t, and hence the order constraint is reasonable.
Potential applications include financial time series and prediction of dynamic
patient out- comes based on clinical measurements. We illustrate this idea on
real and simulated data.Comment: 15 pages, 6 figure
FReLU: Flexible Rectified Linear Units for Improving Convolutional Neural Networks
Rectified linear unit (ReLU) is a widely used activation function for deep
convolutional neural networks. However, because of the zero-hard rectification,
ReLU networks miss the benefits from negative values. In this paper, we propose
a novel activation function called \emph{flexible rectified linear unit
(FReLU)} to further explore the effects of negative values. By redesigning the
rectified point of ReLU as a learnable parameter, FReLU expands the states of
the activation output. When the network is successfully trained, FReLU tends to
converge to a negative value, which improves the expressiveness and thus the
performance. Furthermore, FReLU is designed to be simple and effective without
exponential functions to maintain low cost computation. For being able to
easily used in various network architectures, FReLU does not rely on strict
assumptions by self-adaption. We evaluate FReLU on three standard image
classification datasets, including CIFAR-10, CIFAR-100, and ImageNet.
Experimental results show that the proposed method achieves fast convergence
and higher performances on both plain and residual networks
- …