Sharp analysis of low-rank kernel matrix approximations

We consider supervised learning problems within the positive-definite kernel framework, such as kernel ridge regression, kernel logistic regression or the support vector machine. With kernels leading to infinite-dimensional feature spaces, a common practical limiting difficulty is the necessity of computing the kernel matrix, which most frequently leads to algorithms with running time at least quadratic in the number of observations n, i.e., O(n^2). Low-rank approximations of the kernel matrix are often considered as they allow the reduction of running time complexities to O(p^2 n), where p is the rank of the approximation. The practicality of such methods thus depends on the required rank p. In this paper, we show that in the context of kernel ridge regression, for approximations based on a random subset of columns of the original kernel matrix, the rank p may be chosen to be linear in the degrees of freedom associated with the problem, a quantity which is classically used in the statistical analysis of such methods, and is often seen as the implicit number of parameters of non-parametric estimators. This result enables simple algorithms that have sub-quadratic running time complexity, but provably exhibit the same predictive performance than existing algorithms, for any given problem instance, and not only for worst-case situations

Convex Analysis and Optimization with Submodular Functions: a Tutorial

Set-functions appear in many areas of computer science and applied mathematics, such as machine learning, computer vision, operations research or electrical networks. Among these set-functions, submodular functions play an important role, similar to convex functions on vector spaces. In this tutorial, the theory of submodular functions is presented, in a self-contained way, with all results shown from first principles. A good knowledge of convex analysis is assumed

Gender Is a Natural Kind with a Historical Essence

Traditional debate on the metaphysics of gender has been a contrast of essentialist and social-constructionist positions. The standard reaction to this opposition is that neither position alone has the theoretical resources required to satisfy an equitable politics. This has caused a number of theorists to suggest ways in which gender is unified on the basis of social rather than biological characteristics but is “real” or “objective” nonetheless – a position I term social objectivism. This essay begins by making explicit the motivations for, and central assumptions of, social objectivism. I then propose that gender is better understood as a real kind with a historical essence, analogous to the biologist’s claim that species are historical entities. I argue that this proposal achieves a better solution to the problems that motivate social objectivism. Moreover, the account is consistent with a post-positivist understanding of the classificatory practices employed within the natural and social sciences

Structured sparsity-inducing norms through submodular functions

Sparse methods for supervised learning aim at finding good linear predictors from as few variables as possible, i.e., with small cardinality of their supports. This combinatorial selection problem is often turned into a convex optimization problem by replacing the cardinality function by its convex envelope (tightest convex lower bound), in this case the L1-norm. In this paper, we investigate more general set-functions than the cardinality, that may incorporate prior knowledge or structural constraints which are common in many applications: namely, we show that for nondecreasing submodular set-functions, the corresponding convex envelope can be obtained from its \lova extension, a common tool in submodular analysis. This defines a family of polyhedral norms, for which we provide generic algorithmic tools (subgradients and proximal operators) and theoretical results (conditions for support recovery or high-dimensional inference). By selecting specific submodular functions, we can give a new interpretation to known norms, such as those based on rank-statistics or grouped norms with potentially overlapping groups; we also define new norms, in particular ones that can be used as non-factorial priors for supervised learning

The Influence of Plant Dispersion on Movement Patterns of the Colorado Potato Beetle, \u3ci\u3eLeptinotarsa Decemlineata\u3c/i\u3e (Coleoptera: Chrysomelidae)

The influence of plant dispersion on movements of the Colorado potato beetle, Leptinotarsa decemlineata (Say) (Coleoptera: Chrysomelidae), was studied with mark-recapture techniques. Beetles released between potato monocultures, polycultures with two additional non-host plant species, and polycultures with five additional non-host species, randomly colonized the three types of plots. Releases between different arrangements of potted host and non-host plants showed (1) greater beetle colonization and greater length of time spent on potato plants growing alone than on potato plants surrounded by non-host vegetation, and (2) no effect of potato plant density on colonization or tenure time. Overall, there was a 65~ recapture rate; beetles consistently stayed on the same plants they originally colonized, often for at least five days after release

AdaBatch: Efficient Gradient Aggregation Rules for Sequential and Parallel Stochastic Gradient Methods

We study a new aggregation operator for gradients coming from a mini-batch for stochastic gradient (SG) methods that allows a significant speed-up in the case of sparse optimization problems. We call this method AdaBatch and it only requires a few lines of code change compared to regular mini-batch SGD algorithms. We provide a theoretical insight to understand how this new class of algorithms is performing and show that it is equivalent to an implicit per-coordinate rescaling of the gradients, similarly to what Adagrad methods can do. In theory and in practice, this new aggregation allows to keep the same sample efficiency of SG methods while increasing the batch size. Experimentally, we also show that in the case of smooth convex optimization, our procedure can even obtain a better loss when increasing the batch size for a fixed number of samples. We then apply this new algorithm to obtain a parallelizable stochastic gradient method that is synchronous but allows speed-up on par with Hogwild! methods as convergence does not deteriorate with the increase of the batch size. The same approach can be used to make mini-batch provably efficient for variance-reduced SG methods such as SVRG

Fundamental Reform of Income Tax: In How Far Can the Assessment Basis Be Broadened and Tax Law Simplified?

Intensive discussion is now underway on the tax reform concept put forward by Paul Kirchhof. Analyses based on extrapolations of individual tax return data from the income tax statistics show that ending the main tax concessions and allowances would not be enough to compensate for the loss of revenue from lowering the top rate of tax to 25%. Moreover, the importance of simplifying the tax system is being exaggerated in public discussion. A much simpler tax system is neither necessarily efficient nor fair. Politicians must look for reasonable compromises here. With the appointment of Paul Kirchhof, Professor of Tax Law and former Judge at the Constitutional Court, to the Union parties' competence team the discussion on fundamental reform and simplification of the German income and corporate tax system has intensified. Kirchhof has put forward the most far-reaching proposal for tax reform of recent years in a concept developed with his research group on the Federal tax code. He wants to see an almost flat rate income tax of 25% on all taxable income over euro 18 000; in return, all tax concessions and exemptions related to specific types of income would be dropped, while lump sums would be allowed for some income-related expenses and operating expenditure. The tax regulations would also be tightened and their application simplified by thoroughly systematizing and redrafting the income tax laws. DIW Berlin carried out a study of this and other proposed reforms in April 2004 in regard to the revenue they would yield and their distribution effects, as well as their effects on the supply of labour. The main conclusion was that a clear drop in the rates of tax, particularly in the upper incomes range, would cause considerable loss of revenue, and that this could not be made good by broadening the tax base or stimulating growth. The proposals by Paul Kirchhof, as well as the concept put forward by the Free Democrats, would mean that tax payers on high incomes would pay very much less tax, not only in absolute terms but also in relation to their incomes, than tax payers on average earnings, so these proposals would also lead to greater inequality of income. In view of the current discussion on the scope for broadening the tax base a consideration of the main concessions and allowances is of interest. These are shown in the tax statistics or can be estimated from (table). An extensive and representative random sample was taken from the income tax statistics for 1998 - the latest year for which data is as yet available - and the key features that are relevant for taxation policy were extrapolated to the year 2005.4 According to the forecast 29 million tax payers will be liable for income tax in 2005, of whom 14.8 million will be single and 14.2 million married couples taxed on their joint incomes.5 Simulation calculations of the income tax charged for the 2005 tax year using DIW Berlin's income tax micro-simulation model are in line with the current tax revenue and current estimates of tax. Revenue from income tax charged will be euro 171.4 billions, and revenue from non-assessed nonassessed wage tax euro 15.1 billions.