Online Tensor Learning: Computational and Statistical Trade-offs, Adaptivity and Optimal Regret

We investigate a generalized framework for estimating latent low-rank tensors in an online setting, encompassing both linear and generalized linear models. This framework offers a flexible approach for handling continuous or categorical variables. Additionally, we investigate two specific applications: online tensor completion and online binary tensor learning. To address these challenges, we propose the online Riemannian gradient descent algorithm, which demonstrates linear convergence and the ability to recover the low-rank component under appropriate conditions in all applications. Furthermore, we establish a precise entry-wise error bound for online tensor completion. Notably, our work represents the first attempt to incorporate noise in the online low-rank tensor recovery task. Intriguingly, we observe a surprising trade-off between computational and statistical aspects in the presence of noise. Increasing the step size accelerates convergence but leads to higher statistical error, whereas a smaller step size yields a statistically optimal estimator at the expense of slower convergence. Moreover, we conduct regret analysis for online tensor regression. Under the fixed step size regime, a fascinating trilemma concerning the convergence rate, statistical error rate, and regret is observed. With an optimal choice of step size we achieve an optimal regret of $O(\sqrt{T})$. Furthermore, we extend our analysis to the adaptive setting where the horizon T is unknown. In this case, we demonstrate that by employing different step sizes, we can attain a statistically optimal error rate along with a regret of $O(\log T)$. To validate our theoretical claims, we provide numerical results that corroborate our findings and support our assertions

Provable Sample-Efficient Sparse Phase Retrieval Initialized by Truncated Power Method

We study the sparse phase retrieval problem, recovering an $s$-sparse length-$n$ signal from $m$ magnitude-only measurements. Two-stage non-convex approaches have drawn much attention in recent studies for this problem. Despite non-convexity, many two-stage algorithms provably converge to the underlying solution linearly when appropriately initialized. However, in terms of sample complexity, the bottleneck of those algorithms often comes from the initialization stage. Although the refinement stage usually needs only $m=\Omega(s\log n)$ measurements, the widely used spectral initialization in the initialization stage requires $m=\Omega(s^2\log n)$ measurements to produce a desired initial guess, which causes the total sample complexity order-wisely more than necessary. To reduce the number of measurements, we propose a truncated power method to replace the spectral initialization for non-convex sparse phase retrieval algorithms. We prove that $m=\Omega(\bar{s} s\log n)$ measurements, where $\bar{s}$ is the stable sparsity of the underlying signal, are sufficient to produce a desired initial guess. When the underlying signal contains only very few significant components, the sample complexity of the proposed algorithm is $m=\Omega(s\log n)$ and optimal. Numerical experiments illustrate that the proposed method is more sample-efficient than state-of-the-art algorithms

Dynamic Multi-Arm Bandit Game Based Multi-Agents Spectrum Sharing Strategy Design

For a wireless avionics communication system, a Multi-arm bandit game is mathematically formulated, which includes channel states, strategies, and rewards. The simple case includes only two agents sharing the spectrum which is fully studied in terms of maximizing the cumulative reward over a finite time horizon. An Upper Confidence Bound (UCB) algorithm is used to achieve the optimal solutions for the stochastic Multi-Arm Bandit (MAB) problem. Also, the MAB problem can also be solved from the Markov game framework perspective. Meanwhile, Thompson Sampling (TS) is also used as benchmark to evaluate the proposed approach performance. Numerical results are also provided regarding minimizing the expectation of the regret and choosing the best parameter for the upper confidence bound

Investigating the Impact of Corporate Governance on the Relationship between Related Party Transactions and Earning Management: Empirical Evidence from Chinese Private Listed Companies

Abstract Earnings manipulation and related party transactions (RPT) are common problems and hot topics in earlier academic research. The prior literature on the relationship between RPT and earning management (EM) has developed mature and comprehensive. However, previous empirical research on the relationship between RPT and EM lack considering the concept of CG. Therefore, this research refer CG quality to further analyze and discuss the relationship between RPT and accrual-based EM. The purpose of this research is to analyze the relationship between related party transactions and earning management, and further discuss the impact of different corporate governance environments on the relationship between related party transactions and earnings management in Chinese private listed companies. This research utilizes quantitative research method to analyze the issue between RPT, EM and CG based on the data of 1134 private listed companies from 2016 to 2018. This research uses modify Jones model to measure the level of accrual-based EM and constructs CG index by combining five factors of CG to measure the quality of CG. The results prove that there is a positive relationship between RPT and EM, and the quality of CG has a negative impact on the relationship between RPT and EM. This means that good corporate governance environment helps to alleviate the relationship between RPT and EM. Therefore, management and regulators should pay more attention to establishing a more complete corporate governance system to fundamentally alleviate and improve improper connected transactions and earnings management behavior, whatever from the perspective of the company or the capital market supervision. Key words: related party transactions; earning management; corporate governance; Chinese private listed companie

Nonconvex Stochastic Bregman Proximal Gradient Method with Application to Deep Learning

The widely used stochastic gradient methods for minimizing nonconvex composite objective functions require the Lipschitz smoothness of the differentiable part. But the requirement does not hold true for problem classes including quadratic inverse problems and training neural networks. To address this issue, we investigate a family of stochastic Bregman proximal gradient (SBPG) methods, which only require smooth adaptivity of the differentiable part. SBPG replaces the upper quadratic approximation used in SGD with the Bregman proximity measure, resulting in a better approximation model that captures the non-Lipschitz gradients of the nonconvex objective. We formulate the vanilla SBPG and establish its convergence properties under nonconvex setting without finite-sum structure. Experimental results on quadratic inverse problems testify the robustness of SBPG. Moreover, we propose a momentum-based version of SBPG (MSBPG) and prove it has improved convergence properties. We apply MSBPG to the training of deep neural networks with a polynomial kernel function, which ensures the smooth adaptivity of the loss function. Experimental results on representative benchmarks demonstrate the effectiveness and robustness of MSBPG in training neural networks. Since the additional computation cost of MSBPG compared with SGD is negligible in large-scale optimization, MSBPG can potentially be employed as an universal open-source optimizer in the future.Comment: 37 page

Distributed Power System Virtual Inertia Implemented by Grid-Connected Power Converters

Renewable energy sources (RESs), e.g., wind and solar photovoltaics, have been increasingly used to meet worldwide growing energy demands and reduce greenhouse gas emissions. However, RESs are normally coupled to the power grid through fast-response power converters without any inertia, leading to decreased power system inertia. As a result, the grid frequency may easily go beyond the acceptable range under severe frequency events, resulting in undesirable load-shedding, cascading failures, or even large-scale blackouts. To address the ever-decreasing inertia issue, this paper proposes the concept of distributed power system virtual inertia, which can be implemented by grid-connected power converters. Without modifications of system hardware, power system inertia can be emulated by the energy stored in the dc-link capacitors of grid-connected power converters. By regulating the dc-link voltages in proportional to the grid frequency, the dc-link capacitors are aggregated into an extremely large equivalent capacitor serving as an energy buffer for frequency support. Furthermore, the limitation of virtual inertia, together with its design parameters, is identified. Finally, the feasibility of the proposed concept is validated through simulation and experimental results, which indicate that 12.5% and 50% improvements of the frequency nadir and rate of change of frequency can be achieved.NRF (Natl Research Foundation, S’pore)Accepted versio