Tracking the gradients using the Hessian: A new look at variance reducing stochastic methods

Our goal is to improve variance reducing stochastic methods through better control variates. We first propose a modification of SVRG which uses the Hessian to track gradients over time, rather than to recondition, increasing the correlation of the control variates and leading to faster theoretical convergence close to the optimum. We then propose accurate and computationally efficient approximations to the Hessian, both using a diagonal and a low-rank matrix. Finally, we demonstrate the effectiveness of our method on a wide range of problems.Comment: 17 pages, 2 figures, 1 tabl

Twenty million years of continuous deformation along theKarakorum fault, western Tibet: A thermochronological analysis.

The role of the Karakorum fault zone (KFZ) is debated. South of 33°N, ongoing dextral-oblique slip along the SW edge of the Gar basin exhumes metamorphic and magmatic rocks of the Ayilari range. Minerals have recorded a continuum of deformation from temperatures >600–400°C down to 20 Ma of deformation along the fault. Greenschist facies deformation superimposed upon the medium- to high-grade deformation marks a kinematic change from pure dextral to dextral-normal motion associated with the onset of rapid cooling. At the regional scale, the coexistence of transtension in the Gar basin with transpression documented along the Pangong range farther north suggests another example of the ‘‘zipper tectonics'' model developed along the Red River fault. The kinematic shift induced the rise of the Ayilari range starting at 16–12 Ma and the incision of major river courses. The Indus River might have become captive of the relief at this time. The river's 120 km of apparent offset implies dextral motion at a long-term rate of ca 8.5 ± 1.5 mm/yr

Rare Species Support Vulnerable Functions in High-Diversity Ecosystems

Around the world, the human-induced collapses of populations and species have triggered a sixth mass extinction crisis, with rare species often being the first to disappear. Although the role of species diversity in the maintenance of ecosystem processes has been widely investigated, the role of rare species remains controversial. A critical issue is whether common species insure against the loss of functions supported by rare species. This issue is even more critical in species-rich ecosystems where high functional redundancy among species is likely and where it is thus often assumed that ecosystem functioning is buffered against species loss. Here, using extensive datasets of species occurrences and functional traits from three highly diverse ecosystems (846 coral reef fishes, 2,979 alpine plants, and 662 tropical trees), we demonstrate that the most distinct combinations of traits are supported predominantly by rare species both in terms of local abundance and regional occupancy. Moreover, species that have low functional redundancy and are likely to support the most vulnerable functions, with no other species carrying similar combinations of traits, are rarer than expected by chance in all three ecosystems. For instance, 63% and 98% of fish species that are likely to support highly vulnerable functions in coral reef ecosystems are locally and regionally rare, respectively. For alpine plants, 32% and 89% of such species are locally and regionally rare, respectively. Remarkably, 47% of fish species and 55% of tropical tree species that are likely to support highly vulnerable functions have only one individual per sample on average. Our results emphasize the importance of rare species conservation, even in highly diverse ecosystems, which are thought to exhibit high functional redundancy. Rare species offer more than aesthetic, cultural, or taxonomic diversity value; they disproportionately increase the potential breadth of functions provided by ecosystems across spatial scales. As such, they are likely to insure against future uncertainty arising from climate change and the ever-increasing anthropogenic pressures on ecosystems. Our results call for a more detailed understanding of the role of rarity and functional vulnerability in ecosystem functioning

SGD: General Analysis and Improved Rates

International audienceWe propose a general yet simple theorem describing the convergence of SGD under the arbitrary sampling paradigm. Our theorem describes the convergence of an infinite array of variants of SGD, each of which is associated with a specific probability law governing the data selection rule used to form minibatches. This is the first time such an analysis is performed, and most of our variants of SGD were never explicitly considered in the literature before. Our analysis relies on the recently introduced notion of expected smoothness and does not rely on a uniform bound on the variance of the stochastic gradients. By specializing our theorem to different mini-batching strategies , such as sampling with replacement and independent sampling, we derive exact expressions for the stepsize as a function of the mini-batch size. With this we can also determine the mini-batch size that optimizes the total complexity, and show explicitly that as the variance of the stochastic gradient evaluated at the minimum grows, so does the optimal mini-batch size. For zero variance, the optimal mini-batch size is one. Moreover, we prove insightful stepsize-switching rules which describe when one should switch from a constant to a decreasing stepsize regime

Ocean Colour Radiometry and Water Quality

In coastal and marine waters, the variables associated with a defined 'quality status' can change over a wide range of spatial and temporal scales, leading to substantial logistic and economic difficulties to monitor them on a regular basis. Realistically, the approach to coastal water management is to undertake ongoing assessments of the population and condition of the marine flora and fauna, as well as what might be termed 'water quality'. This approach usually employs in situ techniques which rarely sustain the repeat frequency and spatial coverage that is required to support an effective management practice in vulnerable areas, where more intensive monitoring scheme needs to be in place. However, several of the water quality indicators are directly or indirectly related to the colour or reflectance of the water, and therefore, potentially accessible from satellite remote sensing.JRC.H.3-Global environement monitorin