2,062 research outputs found
Small steps and giant leaps: Minimal Newton solvers for Deep Learning
We propose a fast second-order method that can be used as a drop-in
replacement for current deep learning solvers. Compared to stochastic gradient
descent (SGD), it only requires two additional forward-mode automatic
differentiation operations per iteration, which has a computational cost
comparable to two standard forward passes and is easy to implement. Our method
addresses long-standing issues with current second-order solvers, which invert
an approximate Hessian matrix every iteration exactly or by conjugate-gradient
methods, a procedure that is both costly and sensitive to noise. Instead, we
propose to keep a single estimate of the gradient projected by the inverse
Hessian matrix, and update it once per iteration. This estimate has the same
size and is similar to the momentum variable that is commonly used in SGD. No
estimate of the Hessian is maintained. We first validate our method, called
CurveBall, on small problems with known closed-form solutions (noisy Rosenbrock
function and degenerate 2-layer linear networks), where current deep learning
solvers seem to struggle. We then train several large models on CIFAR and
ImageNet, including ResNet and VGG-f networks, where we demonstrate faster
convergence with no hyperparameter tuning. Code is available
Uncertainty and sensitivity analyses of the complete program system UFOMOD and of selected submodels
Laser Embossing of Micro-and Submicrometer Surface Structures in Copper
Micro- and submicrometer structures have been transferred from nickel foils into solid copper surfaces by laser microembossing. The developed arrangement for laser microembossing allows a large-area replication using multi- pulse laser scanning scheme, guaranties a low contamination of the embossed surface and enables the utilization of thick workpieces. In the micrometer range the replicated patterns feature a high accuracy regarding the shape. A significant difference between the master and the replication pattern could be observed for the laser embossing of submicrometer patterns. In conclusion, the results show that the proposed laser embossing process is a promising method with a number of applications in microengineering
BKT-like transition in the Potts model on an inhomogeneous annealed network
We solve the ferromagnetic q-state Potts model on an inhomogeneous annealed
network which mimics a random recursive graph. We find that this system has the
inverted Berezinskii--Kosterlitz--Thouless (BKT) phase transition for any , including the values , where the Potts model normally shows
a first order phase transition. We obtain the temperature dependences of the
order parameter, specific heat, and susceptibility demonstrating features
typical for the BKT transition. We show that in the entire normal phase, both
the distribution of a linear response to an applied local field and the
distribution of spin-spin correlations have a critical, i.e. power-law, form.Comment: 7 pages, 3 figure
Verhaltenstherapeutische Behandlung eines Patienten mit beginnender Alzheimer-Demenz
The following case report shows a behavioral treatment of a 53-year-old Patient with Alzheimer's Disease (AD), The treatment includes procedures described in the Behavioral Competency Training (VKT) for patients with early-stage AD developed by Ehrhardt and associates. The program consists of about 20 weekly treatment sessions in which the patient's resources are activated and patient's coping is supported. The therapy and the intervention are debated in the light of the ongoing discussion about a state-of-the-art therapy for AD
Half the entanglement in critical systems is distillable from a single specimen
We establish that the leading critical scaling of the single-copy
entanglement is exactly one half of the entropy of entanglement of a block in
critical infinite spin chains in a general setting, using methods of conformal
field theory. Conformal symmetry imposes that the single-copy entanglement for
critical many-body systems scales as E_1(\rho_L)=(c/6) \log L- (c/6)
(\pi^2/\log L) + O(1/L), where L is the number of constituents in a block of an
infinite chain and c corresponds to the central charge. This proves that from a
single specimen of a critical chain, already half the entanglement can be
distilled compared to the rate that is asymptotically available. The result is
substantiated by a quantitative analysis for all translationally invariant
quantum spin chains corresponding to general isotropic quasi-free fermionic
models. An analytic example of the XY model shows that away from criticality
the above simple relation is only maintained near the quantum phase transition
point.Comment: 4 pages RevTeX, 1 figure, final versio
- …