Dictionary optimization for representing sparse signals using Rank-One Atom Decomposition (ROAD)

Dictionary learning has attracted growing research interest during recent years. As it is a bilinear inverse problem, one typical way to address this problem is to iteratively alternate between two stages: sparse coding and dictionary update. The general principle of the alternating approach is to fix one variable and optimize the other one. Unfortunately, for the alternating method, an ill-conditioned dictionary in the training process may not only introduce numerical instability but also trap the overall training process towards a singular point. Moreover, it leads to difficulty in analyzing its convergence, and few dictionary learning algorithms have been proved to have global convergence. For the other bilinear inverse problems, such as short-and-sparse deconvolution (SaSD) and convolutional dictionary learning (CDL), the alternating method is still a popular choice. As these bilinear inverse problems are also ill-posed and complicated, they are tricky to handle. Additional inner iterative methods are usually required for both of the updating stages, which aggravates the difficulty of analyzing the convergence of the whole learning process. It is also challenging to determine the number of iterations for each stage, as over-tuning any stage will trap the whole process into a local minimum that is far from the ground truth. To mitigate the issues resulting from the alternating method, this thesis proposes a novel algorithm termed rank-one atom decomposition (ROAD), which intends to recast a bilinear inverse problem into an optimization problem with respect to a single variable, that is, a set of rank-one matrices. Therefore, the resulting algorithm is one stage, which minimizes the sparsity of the coefficients while keeping the data consistency constraint throughout the whole learning process. Inspired by recent advances in applying the alternating direction method of multipliers (ADMM) to nonconvex nonsmooth problems, an ADMM solver is adopted to address ROAD problems, and a lower bound of the penalty parameter is derived to guarantee a convergence in the augmented Lagrangian despite nonconvexity of the optimization formulation. Compared to two-stage dictionary learning methods, ROAD simplifies the learning process, eases the difficulty of analyzing convergence, and avoids the singular point issue. From a practical point of view, ROAD reduces the number of tuning parameters required in other benchmark algorithms. Numerical tests reveal that ROAD outperforms other benchmark algorithms in both synthetic data tests and single image super-resolution applications. In addition to dictionary learning, the ROAD formulation can also be extended to solve the SaSD and CDL problems. ROAD can still be employed to recast these problems into a one-variable optimization problem. Numerical tests illustrate that ROAD has better performance in estimating convolutional kernels compared to the latest SaSD and CDL algorithms.Open Acces

The origins of covenanting thought and resistance : c.1580-1638

Until quite recently it has been argued that the Scottish Reformation of 1560 removed the trappings of Catholicism from the kirk, but retained the old machinery of ecclesiastical government. Since the 1970s, however, this notion has been placed under increasing pressure by an alternative interpretation which suggests the Reformation rejected episcopal government in favour of a conciliar form of kirk polity. This study, by adopting as its basis the more recent interpretation of the Reformation noted above, proposes the view that the genesis of the presbyterian polity of c. 1580 lies in the thought and intent of the reformers of 1560. The prevalent historiographical view that the hybrid polity of 'bishop-in-presbytery (established in 1610) represented a popular restoration - rather than a stoutly resisted introduction - of an erastian episcopate is therefore challenged. In particular, resistance to the new regime emanated from the lairds, merchants and professional classes of Scottish society, and thus the role of this 'middling group in supporting presbyterianism features prominently in this work. The role of women in the events of the period is likewise discussed, as historiography (in Scotland at least) has neglected their important contribution to the maintenance of resistance during these key years. The thought and actions of two prominent Scottish presbyterian exiles - Alexander Leighton and Robert Durie - worried the king on his English doorstep, and the contribution which these two men made to covenanting thought and resistance, particularly in the 1620s and 1630s, is also examined. Archibald Johnston of Wariston played a major role in the revolution of 1637, and the motivations which led him to become the architect of revolution in 1637 are examined. The overall theme of the thesis is one of continuity of thought and resistance, and thus the thesis looks finally in detail at the nature and process of presbyterian protest and petition from c. 1580 to 1637

Dictionary Learning with BLOTLESS Update

Algorithms for learning a dictionary to sparsely represent a given dataset typically alternate between sparse coding and dictionary update stages. Methods for dictionary update aim to minimise expansion error by updating dictionary vectors and expansion coefficients given patterns of non-zero coefficients obtained in the sparse coding stage. We propose a block total least squares (BLOTLESS) algorithm for dictionary update. BLOTLESS updates a block of dictionary elements and the corresponding sparse coefficients simultaneously. In the error free case, three necessary conditions for exact recovery are identified. Lower bounds on the number of training data are established so that the necessary conditions hold with high probability. Numerical simulations show that the bounds approximate well the number of training data needed for exact dictionary recovery. Numerical experiments further demonstrate several benefits of dictionary learning with BLOTLESS update compared with state-of-the-art algorithms especially when the amount of training data is small