Small steps and giant leaps: Minimal Newton solvers for Deep Learning

We propose a fast second-order method that can be used as a drop-in replacement for current deep learning solvers. Compared to stochastic gradient descent (SGD), it only requires two additional forward-mode automatic differentiation operations per iteration, which has a computational cost comparable to two standard forward passes and is easy to implement. Our method addresses long-standing issues with current second-order solvers, which invert an approximate Hessian matrix every iteration exactly or by conjugate-gradient methods, a procedure that is both costly and sensitive to noise. Instead, we propose to keep a single estimate of the gradient projected by the inverse Hessian matrix, and update it once per iteration. This estimate has the same size and is similar to the momentum variable that is commonly used in SGD. No estimate of the Hessian is maintained. We first validate our method, called CurveBall, on small problems with known closed-form solutions (noisy Rosenbrock function and degenerate 2-layer linear networks), where current deep learning solvers seem to struggle. We then train several large models on CIFAR and ImageNet, including ResNet and VGG-f networks, where we demonstrate faster convergence with no hyperparameter tuning. Code is available

The role of bacteria in pine wilt disease: insights from microbiome analysis.

Pine Wilt Disease (PWD) has a significant impact on Eurasia pine forests. The microbiome of the nematode (the primary cause of the disease), its insect vector, and the host tree may be relevant for the disease mechanism. The aim of this study was to characterize these microbiomes, from three PWD-affected areas in Portugal, using Denaturing Gradient Gel Electrophoresis, 16S rRNA gene pyrosequencing, and a functional inference-based approach (PICRUSt). The bacterial community structure of the nematode was significantly different from the infected trees but closely related to the insect vector, supporting the hypothesis that the nematode microbiome might be in part inherited from the insect. Sampling location influenced mostly the tree microbiome (P < 0.05). Genes related both with plant growth promotion and phytopathogenicity were predicted for the tree microbiome. Xenobiotic degradation functions were predicted in the nematode and insect microbiomes. Phytotoxin biosynthesis was also predicted for the nematode microbiome, supporting the theory of a direct contribution of the microbiome to tree-wilting. This is the first study that simultaneously characterized the nematode, tree and insect-vector microbiomes from the same affected areas, and overall the results support the hypothesis that the PWD microbiome plays an important role in the disease's development

Situating Sound: The Space and Time of the Dancehall Session

This research situates the multiple body of the Jamaica Dancehall "Crowd" (audience) in the intensities of the Sound System Session. This is a heterogeneous "acoustic space," and discontinuous ritual time, in which sexual expression and orientation, and racial attitudes, diverge from Jamaican norms. This essay proceeds to account for the propagation of this temporality and spatiality in terms of the electromechanical processes of the Sound System "Set" (equipment), that is control, power and transduction. It looks firstly at the Sound Engineers' sensorimotor engineering technique of compensation for monitoring and manipulating the auditory performance of the Set. Secondly it discusses the sociocultural procedures of the cutting and mixing of the music the Selector plays in the Session. The essay identifies these practices and procedures as the basic elements for many cultural, cybernetic, linguistic, or communication systems. In conclusion, it is suggested that for the Engineers' and Selectors' instrumental techniques to be affective and effective they have to be brought into a proportional relationship with the Crowd's experience. The Crew does this through their embodied experience and expert evaluative judgment - which is considered as an example of analogical, rather than logical, rationality

Sonic Dominance and the Reggae Sound System Session

Sound connects people; it draws us together. It was Count Basie who drew me to one the editors of this volume. He was playing Lester Leaps In. And it was the sound of the music that pulled me in through a half-open door. Portuguese trans. https://revistaecopos.eco.ufrj.br/eco_po

Supermassive black holes as the regulators of star formation in central galaxies

We present a relationship between the black hole mass, stellar mass, and star formation rate of a diverse group of 91 galaxies with dynamically-measured black hole masses. For our sample of galaxies with a variety of morphologies and other galactic properties, we find that the specific star formation rate is a smoothly decreasing function of the ratio between black hole mass and stellar mass, or what we call the specific black hole mass. In order to explain this relation, we propose a physical framework where the gradual suppression of a galaxy's star formation activity results from the adjustment to an increase in specific black hole mass and, accordingly, an increase in the amount of heating. From this framework, it follows that at least some galaxies with intermediate specific black hole masses are in a steady state of partial quiescence with intermediate specific star formation rates, implying that both transitioning and steady-state galaxies live within this region known as the "green valley." With respect to galaxy formation models, our results present an important diagnostic with which to test various prescriptions of black hole feedback and its effects on star formation activity.Comment: 15 pages, 4 figures, 2 tables. Accepted for publication in The Astrophysical Journa

Learning feed-forward one-shot learners

One-shot learning is usually tackled by using generative models or discriminative embeddings. Discriminative methods based on deep learning, which are very effective in other learning scenarios, are ill-suited for one-shot learning as they need large amounts of training data. In this paper, we propose a method to learn the parameters of a deep model in one shot. We construct the learner as a second deep network, called a learnet, which predicts the parameters of a pupil network from a single exemplar. In this manner we obtain an efficient feed-forward one-shot learner, trained end-to-end by minimizing a one-shot classification objective in a learning to learn formulation. In order to make the construction feasible, we propose a number of factorizations of the parameters of the pupil network. We demonstrate encouraging results by learning characters from single exemplars in Omniglot, and by tracking visual objects from a single initial exemplar in the Visual Object Tracking benchmark.Comment: The first three authors contributed equally, and are listed in alphabetical orde

End-to-end representation learning for Correlation Filter based tracking

The Correlation Filter is an algorithm that trains a linear template to discriminate between images and their translations. It is well suited to object tracking because its formulation in the Fourier domain provides a fast solution, enabling the detector to be re-trained once per frame. Previous works that use the Correlation Filter, however, have adopted features that were either manually designed or trained for a different task. This work is the first to overcome this limitation by interpreting the Correlation Filter learner, which has a closed-form solution, as a differentiable layer in a deep neural network. This enables learning deep features that are tightly coupled to the Correlation Filter. Experiments illustrate that our method has the important practical benefit of allowing lightweight architectures to achieve state-of-the-art performance at high framerates.Comment: To appear at CVPR 201