On adaptive stochastic heavy ball momentum for solving linear systems

The stochastic heavy ball momentum (SHBM) method has gained considerable popularity as a scalable approach for solving large-scale optimization problems. However, one limitation of this method is its reliance on prior knowledge of certain problem parameters, such as singular values of a matrix. In this paper, we propose an adaptive variant of the SHBM method for solving stochastic problems that are reformulated from linear systems using user-defined distributions. Our adaptive SHBM (ASHBM) method utilizes iterative information to update the parameters, addressing an open problem in the literature regarding the adaptive learning of momentum parameters. We prove that our method converges linearly in expectation, with a better convergence rate compared to the basic method. Notably, we demonstrate that the deterministic version of our ASHBM algorithm can be reformulated as a variant of the conjugate gradient (CG) method, inheriting many of its appealing properties, such as finite-time convergence. Consequently, the ASHBM method can be further generalized to develop a brand-new framework of the stochastic CG (SCG) method for solving linear systems. Our theoretical results are supported by numerical experiments

On the convergence analysis of the greedy randomized Kaczmarz method

In this paper, we analyze the greedy randomized Kaczmarz (GRK) method proposed in Bai and Wu (SIAM J. Sci. Comput., 40(1):A592--A606, 2018) for solving linear systems. We develop more precise greedy probability criteria to effectively select the working row from the coefficient matrix. Notably, we prove that the linear convergence of the GRK method is deterministic and demonstrate that using a tighter threshold parameter can lead to a faster convergence rate. Our result revises existing convergence analyses, which are solely based on the expected error by realizing that the iterates of the GRK method are random variables. Consequently, we obtain an improved iteration complexity for the GRK method. Moreover, the Polyak's heavy ball momentum technique is incorporated to improve the performance of the GRK method. We propose a refined convergence analysis, compared with the technique used in Loizou and Richt\'{a}rik (Comput. Optim. Appl., 77(3):653--710, 2020), of momentum variants of randomized iterative methods, which shows that the proposed GRK method with momentum (mGRK) also enjoys a deterministic linear convergence. Numerical experiments show that the mGRK method is more efficient than the GRK method

Fast stochastic dual coordinate descent algorithms for linearly constrained convex optimization

The problem of finding a solution to the linear system $Ax = b$ with certain minimization properties arises in numerous scientific and engineering areas. In the era of big data, the stochastic optimization algorithms become increasingly significant due to their scalability for problems of unprecedented size. This paper focuses on the problem of minimizing a strongly convex function subject to linear constraints. We consider the dual formulation of this problem and adopt the stochastic coordinate descent to solve it. The proposed algorithmic framework, called fast stochastic dual coordinate descent, utilizes sampling matrices sampled from user-defined distributions to extract gradient information. Moreover, it employs Polyak's heavy ball momentum acceleration with adaptive parameters learned through iterations, overcoming the limitation of the heavy ball momentum method that it requires prior knowledge of certain parameters, such as the singular values of a matrix. With these extensions, the framework is able to recover many well-known methods in the context, including the randomized sparse Kaczmarz method, the randomized regularized Kaczmarz method, the linearized Bregman iteration, and a variant of the conjugate gradient (CG) method. We prove that, with strongly admissible objective function, the proposed method converges linearly in expectation. Numerical experiments are provided to confirm our results.Comment: arXiv admin note: text overlap with arXiv:2305.0548

The Backward Euler Fully Discrete Finite Volume Method for the Problem of Purely Longitudinal Motion of a Homogeneous Bar

We present a linear backward Euler fully discrete finite volume method for the initial-boundary-value problem of purely longitudinal motion of a homogeneous bar and an give optimal order error estimates in L2 and H1 norms. Furthermore, we obtain the superconvergence error estimate of the generalized projection of the solution u in H1 norm. Numerical experiment illustrates the convergence and stability of this scheme

Improvement of provably secure self-certified proxy convertible authenticated encryption scheme

By integrating self-certified public-key systems and the designated verifier proxy signature with message recovery, Wu and Lin proposed the first self-certified proxy convertible authenticated encryption (SP-CAE) scheme and its variants based on discrete logarithm problem (DLP) in 2009. Though their schemes are claimed provably secure, we demonstrate that their schemes are existentially forgeable under adaptive chosen warrants, unconfidentiable and verifiable under adaptive chosen messages and designated verifiers. Then we propose a provably secure SP-CAE scheme in the random oracle model

Monitoring hourly night-time light by an unmanned aerial vehicle and its implications to satellite remote sensing

Satellite-observed night-time light in urban areas has been widely used as an indicator for socioeconomic development and light pollution. Up to present, the diurnal dynamics of city light during the night, which are important to understand the nature of human activity and the underlying variables explaining night-time brightness, have hardly been investigated by remote sensing techniques due to limitation of the revisit time and spatial resolution of available satellites. In this study, we employed a consumer-grade unmanned aerial vehicle (UAV) to monitor city light in a study area located in Wuhan City, China, from 8:08 PM, April 15, 2019 to 5:08 AM, April 16, 2019, with an hourly temporal resolution. By using three ground-based Sky Quality Meters (SQMs), we found that the UAV-recorded light brightness was consistent with the ground luminous intensity measured by the SQMs in both the spatial (R = 0.72) and temporal dimensions (R > 0.94), and that the average city light brightness was consistent with the sky brightness in the temporal dimension (R = 0.98), indicating that UAV images can reliably monitor the city's night-time brightness. The temporal analysis showed that different locations had different patterns of temporal changes in their night-time brightness, implying that inter-calibration of two kinds of satellite images with different overpass times would be a challenge. Combining an urban function map of 18 classes and the hourly UAV images, we found that urban functions differed in their temporal light dynamics. For example, the outdoor sports field lost 97.28% of its measured brightness between 8: 08 PM – 4:05 AM, while an administrative building only lost 4.56%, and the entire study area lost 61.86% of its total brightness. Within our study area, the period between 9:06 PM and 10:05 PM was the period with largest amount of light loss. The spectral analysis we conducted showed that city light colors were different in some urban functions, with the major road being the reddest region at 8:08 PM and becoming even redder at 4:05 AM. This preliminary study indicates that UAVs are a good tool to investigate city light at night, and that city light is very complex in both of the temporal and spatial dimensions, requiring comprehensive investigation using more advanced UAV techniques, and emphasizing the need for geostationary platforms for night-time light sensors