An Ordered Lasso and Sparse Time-Lagged Regression

We consider regression scenarios where it is natural to impose an order constraint on the coefficients. We propose an order-constrained version of L1-regularized regression for this problem, and show how to solve it efficiently using the well-known Pool Adjacent Violators Algorithm as its proximal operator. The main application of this idea is time-lagged regression, where we predict an outcome at time t from features at the previous K time points. In this setting it is natural to assume that the coefficients decay as we move farther away from t, and hence the order constraint is reasonable. Potential applications include financial time series and prediction of dynamic patient out- comes based on clinical measurements. We illustrate this idea on real and simulated data.Comment: 15 pages, 6 figure

FReLU: Flexible Rectified Linear Units for Improving Convolutional Neural Networks

Rectified linear unit (ReLU) is a widely used activation function for deep convolutional neural networks. However, because of the zero-hard rectification, ReLU networks miss the benefits from negative values. In this paper, we propose a novel activation function called \emph{flexible rectified linear unit (FReLU)} to further explore the effects of negative values. By redesigning the rectified point of ReLU as a learnable parameter, FReLU expands the states of the activation output. When the network is successfully trained, FReLU tends to converge to a negative value, which improves the expressiveness and thus the performance. Furthermore, FReLU is designed to be simple and effective without exponential functions to maintain low cost computation. For being able to easily used in various network architectures, FReLU does not rely on strict assumptions by self-adaption. We evaluate FReLU on three standard image classification datasets, including CIFAR-10, CIFAR-100, and ImageNet. Experimental results show that the proposed method achieves fast convergence and higher performances on both plain and residual networks

The Dynamics of Public Opinion in Complex Networks

This paper studies the problem of public opinion formation and concentrates on the interplays among three factors: individual attributes, environmental influences and information flow. We present a simple model to analyze the dynamics of four types of networks. Our simulations suggest that regular communities establish not only local consensus, but also global diversity in public opinions. However, when small world networks, random networks, or scale-free networks model social relationships, the results are sensitive to the elasticity coefficient of environmental influences and the average connectivity of the type of network. For example, a community with a higher average connectivity has a higher probability of consensus. Yet, it is misleading to predict results merely based on the characteristic path length of networks. In the process of changing environmental influences and average connectivity, sensitive areas are discovered in the system. By sensitive areas we mean that interior randomness emerges and we cannot predict unequivocally how many opinions will remain upon reaching equilibrium. We also investigate the role of authoritative individuals in information control. While enhancing average connectivity facilitates the diffusion of the authoritative opinion, it makes individuals subject to disturbance from non-authorities as well. Thus, a moderate average connectivity may be preferable because then the public will most likely form an opinion that is parallel with the authoritative one. In a community with a scale-free structure, the influence of authoritative individuals keeps constant with the change of the average connectivity. Provided that the influence of individuals is proportional to the number of their acquaintances, the smallest percentage of authorities is required for a controlled consensus in a scale free network. This study shows that the dynamics of public opinion varies from community to community due to the different degree of impressionability of people and the distinct social network structure of the community.Public Opinion, Complex Network, Consensus, Agent-Based Model