Proximal Online Gradient is Optimum for Dynamic Regret

In online learning, the dynamic regret metric chooses the reference (optimal) solution that may change over time, while the typical (static) regret metric assumes the reference solution to be constant over the whole time horizon. The dynamic regret metric is particularly interesting for applications such as online recommendation (since the customers' preference always evolves over time). While the online gradient method has been shown to be optimal for the static regret metric, the optimal algorithm for the dynamic regret remains unknown. In this paper, we show that proximal online gradient (a general version of online gradient) is optimum to the dynamic regret by showing that the proved lower bound matches the upper bound that slightly improves existing upper bound

Enabling Covariance-Based Feedback in Massive MIMO: A User Classification Approach

In this paper, we propose a novel channel feedback scheme for frequency division duplexing massive multi-input multi-output systems. The concept uses the notion of user statistical separability which was hinted in several prior works in the massive antenna regime but not fully exploited so far. We here propose a hybrid statistical-instantaneous feedback scheme based on a user classification mechanism where the classification metric derives from a rate bound analysis. According to classification results, a user either operates on a statistical feedback mode or instantaneous mode. Our results illustrate the sum rate advantages of our scheme under a global feedback overhead constraint.Comment: 5 pages, 4 figures, conference paper, 2018 Asilomar Conference on Signals, Systems, and Computer

Constructing black holes in Einstein-Maxwell-scalar theory

Exact black hole solutions in the Einstein-Maxwell-scalar theory are constructed. They are the extensions of dilaton black holes in de Sitter or anti de Sitter universe. As a result, except for a scalar potential, a coupling function between the scalar field and the Maxwell invariant is present. Then the corresponding Smarr formula and the first law of thermodynamics are investigated.Comment: 25 pages,8 figure

The Second Order Linear Model

We study a fundamental class of regression models called the second order linear model (SLM). The SLM extends the linear model to high order functional space and has attracted considerable research interest recently. Yet how to efficiently learn the SLM under full generality using nonconvex solver still remains an open question due to several fundamental limitations of the conventional gradient descent learning framework. In this study, we try to attack this problem from a gradient-free approach which we call the moment-estimation-sequence (MES) method. We show that the conventional gradient descent heuristic is biased by the skewness of the distribution therefore is no longer the best practice of learning the SLM. Based on the MES framework, we design a nonconvex alternating iteration process to train a $d$-dimension rank-$k$ SLM within $O(kd)$ memory and one-pass of the dataset. The proposed method converges globally and linearly, achieves $\epsilon$ recovery error after retrieving $O[k^{2}d\cdot\mathrm{polylog}(kd/\epsilon)]$ samples. Furthermore, our theoretical analysis reveals that not all SLMs can be learned on every sub-gaussian distribution. When the instances are sampled from a so-called $\tau$-MIP distribution, the SLM can be learned by $O(p/\tau^{2})$ samples where $p$ and $\tau$ are positive constants depending on the skewness and kurtosis of the distribution. For non-MIP distribution, an addition diagonal-free oracle is necessary and sufficient to guarantee the learnability of the SLM. Numerical simulations verify the sharpness of our bounds on the sampling complexity and the linear convergence rate of our algorithm

Nonconvex One-bit Single-label Multi-label Learning

We study an extreme scenario in multi-label learning where each training instance is endowed with a single one-bit label out of multiple labels. We formulate this problem as a non-trivial special case of one-bit rank-one matrix sensing and develop an efficient non-convex algorithm based on alternating power iteration. The proposed algorithm is able to recover the underlying low-rank matrix model with linear convergence. For a rank-$k$ model with $d_1$ features and $d_2$ classes, the proposed algorithm achieves $O(\epsilon)$ recovery error after retrieving $O(k^{1.5}d_1 d_2/\epsilon)$ one-bit labels within $O(kd)$ memory. Our bound is nearly optimal in the order of $O(1/\epsilon)$. This significantly improves the state-of-the-art sampling complexity of one-bit multi-label learning. We perform experiments to verify our theory and evaluate the performance of the proposed algorithm

A Covariance-Based Hybrid Channel Feedback in FDD Massive MIMO Systems

In this paper, a novel covariance-based channel feedback mechanism is investigated for frequency division duplexing (FDD) massive multi-input multi-output (MIMO) systems. The concept capitalizes on the notion of user statistical separability which was hinted in several prior works in the massive antenna regime but not fully exploited so far. We here propose a hybrid statistical-instantaneous feedback mechanism where the users are separated into two classes of feedback design based on their channel covariance. Under the hybrid framework, each user either operates on a statistical feedback mode or quantized instantaneous channel feedback mode depending on their so-called statistical isolability. The key challenge lies in the design of a covariance-aware classification algorithm which can handle the complex mutual interactions between all users. The classification is derived from rate bound principles. A suitable precoding method is also devised under the mixed statistical and instantaneous feedback model. Simulations are performed to validate our analytical results and illustrate the sum rate advantages of the proposed feedback scheme under a global feedback overhead constraint.Comment: 31 pages, 9 figure

Walking behavior in a circular arena modified by pulsed light stimulation in Drosophila melanogaster w1118 line

The Drosophila melanogaster white-eyed w1118 line serves as a blank control, allowing genetic recombination of any gene of interest along with a readily recognizable marker. w1118 flies display behavioral susceptibility to environmental stimulation such as light. It is of great importance to characterize the behavioral performance of w1118 flies because this would provide a baseline from which the effect of the gene of interest could be differentiated. Little work has been performed to characterize the walking behavior in adult w1118 flies. Here we show that pulsed light stimulation increased the regularity of walking trajectories of w1118 flies in circular arenas. We statistically modeled the distribution of distances to center and extracted the walking structures of w1118 flies. Pulsed light stimulation redistributed the time proportions for individual walking structures. Specifically, pulsed light stimulation reduced the episodes of crossing over the central region of the arena. An addition of four genomic copies of mini-white, a common marker gene for eye color, mimicked the effect of pulsed light stimulation in reducing crossing in a circular arena. The reducing effect of mini-white was copy-number-dependent. These findings highlight the rhythmic light stimulation-evoked modifications of walking behavior in w1118 flies and an unexpected behavioral consequence of mini-white in transgenic flies carrying w1118 isogenic background.Comment: 27 pages, 6 figures, research articl

Creation of Ghost Illusions Using Metamaterials in Wave Dynamics

The creation of wave-dynamic illusion functionality is of great interests to various scientific communities, which can potentially transform an actual perception into the pre-controlled perception, thus empowering unprecedented applications in the advanced-material science, camouflage, cloaking, optical and/or microwave cognition, and defense security, etc. By using the space transformation theory and engineering capability of metamaterials, we propose and realize a functional ghost illusion device, which is capable of creating wave-dynamic virtual ghost images off the original object's position under the illumination of electromagnetic waves. The scattering signature of the object is thus ghosted and perceived as multiple ghost targets with different geometries and compositions. The ghost-illusion material, being inhomogeneous and anisotropic, was realized by thousands of varying unit cells working at non-resonance. The experimental demonstration of the ghost illusion validates our theory of scattering metamorphosis and opens a novel avenue to the wave-dynamic illusion, cognitive deception, manipulate strange light or matter behaviors, and design novel optical and microwave devices.Comment: 19 pages, 6 figure

GESF: A Universal Discriminative Mapping Mechanism for Graph Representation Learning

Graph embedding is a central problem in social network analysis and many other applications, aiming to learn the vector representation for each node. While most existing approaches need to specify the neighborhood and the dependence form to the neighborhood, which may significantly degrades the flexibility of representation, we propose a novel graph node embedding method (namely GESF) via the set function technique. Our method can 1) learn an arbitrary form of representation function from neighborhood, 2) automatically decide the significance of neighbors at different distances, and 3) be applied to heterogeneous graph embedding, which may contain multiple types of nodes. Theoretical guarantee for the representation capability of our method has been proved for general homogeneous and heterogeneous graphs and evaluation results on benchmark data sets show that the proposed GESF outperforms the state-of-the-art approaches on producing node vectors for classification tasks.Comment: 18 page

Decentralized Online Learning: Take Benefits from Others' Data without Sharing Your Own to Track Global Trend

Decentralized Online Learning (online learning in decentralized networks) attracts more and more attention, since it is believed that Decentralized Online Learning can help the data providers cooperatively better solve their online problems without sharing their private data to a third party or other providers. Typically, the cooperation is achieved by letting the data providers exchange their models between neighbors, e.g., recommendation model. However, the best regret bound for a decentralized online learning algorithm is \Ocal{n\sqrt{T}}, where $n$ is the number of nodes (or users) and $T$ is the number of iterations. This is clearly insignificant since this bound can be achieved \emph{without} any communication in the networks. This reminds us to ask a fundamental question: \emph{Can people really get benefit from the decentralized online learning by exchanging information?} In this paper, we studied when and why the communication can help the decentralized online learning to reduce the regret. Specifically, each loss function is characterized by two components: the adversarial component and the stochastic component. Under this characterization, we show that decentralized online gradient (DOG) enjoys a regret bound \Ocal{n\sqrt{T}G + \sqrt{nT}\sigma}, where $G$ measures the magnitude of the adversarial component in the private data (or equivalently the local loss function) and $\sigma$ measures the randomness within the private data. This regret suggests that people can get benefits from the randomness in the private data by exchanging private information. Another important contribution of this paper is to consider the dynamic regret -- a more practical regret to track users' interest dynamics. Empirical studies are also conducted to validate our analysis.Comment: Second version: revise Assumption 1 (there is a typo in the first version); add experiments (see Figure 2); revise Algorithm 1 in a more clear wa