Small steps and giant leaps: Minimal Newton solvers for Deep Learning

We propose a fast second-order method that can be used as a drop-in replacement for current deep learning solvers. Compared to stochastic gradient descent (SGD), it only requires two additional forward-mode automatic differentiation operations per iteration, which has a computational cost comparable to two standard forward passes and is easy to implement. Our method addresses long-standing issues with current second-order solvers, which invert an approximate Hessian matrix every iteration exactly or by conjugate-gradient methods, a procedure that is both costly and sensitive to noise. Instead, we propose to keep a single estimate of the gradient projected by the inverse Hessian matrix, and update it once per iteration. This estimate has the same size and is similar to the momentum variable that is commonly used in SGD. No estimate of the Hessian is maintained. We first validate our method, called CurveBall, on small problems with known closed-form solutions (noisy Rosenbrock function and degenerate 2-layer linear networks), where current deep learning solvers seem to struggle. We then train several large models on CIFAR and ImageNet, including ResNet and VGG-f networks, where we demonstrate faster convergence with no hyperparameter tuning. Code is available

Laser Embossing of Micro-and Submicrometer Surface Structures in Copper

Micro- and submicrometer structures have been transferred from nickel foils into solid copper surfaces by laser microembossing. The developed arrangement for laser microembossing allows a large-area replication using multi- pulse laser scanning scheme, guaranties a low contamination of the embossed surface and enables the utilization of thick workpieces. In the micrometer range the replicated patterns feature a high accuracy regarding the shape. A significant difference between the master and the replication pattern could be observed for the laser embossing of submicrometer patterns. In conclusion, the results show that the proposed laser embossing process is a promising method with a number of applications in microengineering

BKT-like transition in the Potts model on an inhomogeneous annealed network

We solve the ferromagnetic q-state Potts model on an inhomogeneous annealed network which mimics a random recursive graph. We find that this system has the inverted Berezinskii--Kosterlitz--Thouless (BKT) phase transition for any $q \geq 1$, including the values $q \geq 3$, where the Potts model normally shows a first order phase transition. We obtain the temperature dependences of the order parameter, specific heat, and susceptibility demonstrating features typical for the BKT transition. We show that in the entire normal phase, both the distribution of a linear response to an applied local field and the distribution of spin-spin correlations have a critical, i.e. power-law, form.Comment: 7 pages, 3 figure

Verhaltenstherapeutische Behandlung eines Patienten mit beginnender Alzheimer-Demenz

The following case report shows a behavioral treatment of a 53-year-old Patient with Alzheimer's Disease (AD), The treatment includes procedures described in the Behavioral Competency Training (VKT) for patients with early-stage AD developed by Ehrhardt and associates. The program consists of about 20 weekly treatment sessions in which the patient's resources are activated and patient's coping is supported. The therapy and the intervention are debated in the light of the ongoing discussion about a state-of-the-art therapy for AD

Half the entanglement in critical systems is distillable from a single specimen

We establish that the leading critical scaling of the single-copy entanglement is exactly one half of the entropy of entanglement of a block in critical infinite spin chains in a general setting, using methods of conformal field theory. Conformal symmetry imposes that the single-copy entanglement for critical many-body systems scales as E_1(\rho_L)=(c/6) \log L- (c/6) (\pi^2/\log L) + O(1/L), where L is the number of constituents in a block of an infinite chain and c corresponds to the central charge. This proves that from a single specimen of a critical chain, already half the entanglement can be distilled compared to the rate that is asymptotically available. The result is substantiated by a quantitative analysis for all translationally invariant quantum spin chains corresponding to general isotropic quasi-free fermionic models. An analytic example of the XY model shows that away from criticality the above simple relation is only maintained near the quantum phase transition point.Comment: 4 pages RevTeX, 1 figure, final versio