On-Line Learning Theory of Soft Committee Machines with Correlated Hidden Units - Steepest Gradient Descent and Natural Gradient Descent -

The permutation symmetry of the hidden units in multilayer perceptrons causes the saddle structure and plateaus of the learning dynamics in gradient learning methods. The correlation of the weight vectors of hidden units in a teacher network is thought to affect this saddle structure, resulting in a prolonged learning time, but this mechanism is still unclear. In this paper, we discuss it with regard to soft committee machines and on-line learning using statistical mechanics. Conventional gradient descent needs more time to break the symmetry as the correlation of the teacher weight vectors rises. On the other hand, no plateaus occur with natural gradient descent regardless of the correlation for the limit of a low learning rate. Analytical results support these dynamics around the saddle point.Comment: 7 pages, 6 figure

Effect of manganese doping on the size effect of lead zirconate titanate thin films and the extrinsic nature of dead layers

We have investigated the size effect in lead zirconate titanate (PZT) thin films with a range of manganese (Mn) doping concentrations. We found that the size effect in the conventional Pt/PZT/Pt thin-film capacitors could be systematically reduced and almost completely eliminated by increasing Mn doping concentration. The interfacial layer at the electrode-film interface appears to disappear almost entirely for the PZT films with 2% Mn doping levels, confirmed by the fits using the conventional in-series capacitor model. Our work indicates that the size effect in ferroelectrics is extrinsic in nature, supporting the work by Saad et al. Other implications of our results have also been discussed. By comparing a variety of experimental studies in the literature we propose a scenario that the dead layer between PZT (or barium strontium titanate, BST) and metal electrodes such as Pt and Au might have a defective pyrochlore/fluorite structure (possibly with a small portion of ferroelectric perovskite phase).Comment: 21 pages, 6 figure