4,695 research outputs found
Let's Make Block Coordinate Descent Go Fast: Faster Greedy Rules, Message-Passing, Active-Set Complexity, and Superlinear Convergence
Block coordinate descent (BCD) methods are widely-used for large-scale
numerical optimization because of their cheap iteration costs, low memory
requirements, amenability to parallelization, and ability to exploit problem
structure. Three main algorithmic choices influence the performance of BCD
methods: the block partitioning strategy, the block selection rule, and the
block update rule. In this paper we explore all three of these building blocks
and propose variations for each that can lead to significantly faster BCD
methods. We (i) propose new greedy block-selection strategies that guarantee
more progress per iteration than the Gauss-Southwell rule; (ii) explore
practical issues like how to implement the new rules when using "variable"
blocks; (iii) explore the use of message-passing to compute matrix or Newton
updates efficiently on huge blocks for problems with a sparse dependency
between variables; and (iv) consider optimal active manifold identification,
which leads to bounds on the "active set complexity" of BCD methods and leads
to superlinear convergence for certain problems with sparse solutions (and in
some cases finite termination at an optimal solution). We support all of our
findings with numerical results for the classic machine learning problems of
least squares, logistic regression, multi-class logistic regression, label
propagation, and L1-regularization
Introduction: unmarried and unknown: urban men and women in the low countries since the early modern period
This essay introduces a special issue on The Lure of the City that examines the attraction of towns to unmarried men and women in the Low Countries during the early modern period and the nineteenth century. The issue has the relation between singles and cities as its main focus. Singles were present in disproportionately large numbers in urban areas, but the question is why? This essay sets out the historiographical framework for the contributions in the issue, discusses the sources and methodologies used, and provides a brief overview of the evolution of singleness in the Low Countries. The contributions all demonstrate the relevance of a comparative approach. It is revealed that towns and cities not only attracted but also created singles, that they offered different opportunities for different groups of unmarried people and that they affected men and women differently. Finally, it appears that not every town and city was attractive to men and women without a spouse
- …