On-Line Learning Theory of Soft Committee Machines with Correlated Hidden Units - Steepest Gradient Descent and Natural Gradient Descent -

The permutation symmetry of the hidden units in multilayer perceptrons causes the saddle structure and plateaus of the learning dynamics in gradient learning methods. The correlation of the weight vectors of hidden units in a teacher network is thought to affect this saddle structure, resulting in a prolonged learning time, but this mechanism is still unclear. In this paper, we discuss it with regard to soft committee machines and on-line learning using statistical mechanics. Conventional gradient descent needs more time to break the symmetry as the correlation of the teacher weight vectors rises. On the other hand, no plateaus occur with natural gradient descent regardless of the correlation for the limit of a low learning rate. Analytical results support these dynamics around the saddle point.Comment: 7 pages, 6 figure

Statistical Mechanics of Nonlinear On-line Learning for Ensemble Teachers

We analyze the generalization performance of a student in a model composed of nonlinear perceptrons: a true teacher, ensemble teachers, and the student. We calculate the generalization error of the student analytically or numerically using statistical mechanics in the framework of on-line learning. We treat two well-known learning rules: Hebbian learning and perceptron learning. As a result, it is proven that the nonlinear model shows qualitatively different behaviors from the linear model. Moreover, it is clarified that Hebbian learning and perceptron learning show qualitatively different behaviors from each other. In Hebbian learning, we can analytically obtain the solutions. In this case, the generalization error monotonically decreases. The steady value of the generalization error is independent of the learning rate. The larger the number of teachers is and the more variety the ensemble teachers have, the smaller the generalization error is. In perceptron learning, we have to numerically obtain the solutions. In this case, the dynamical behaviors of the generalization error are non-monotonic. The smaller the learning rate is, the larger the number of teachers is; and the more variety the ensemble teachers have, the smaller the minimum value of the generalization error is.Comment: 13 pages, 9 figure

Analysis of dropout learning regarded as ensemble learning

Deep learning is the state-of-the-art in fields such as visual object recognition and speech recognition. This learning uses a large number of layers, huge number of units, and connections. Therefore, overfitting is a serious problem. To avoid this problem, dropout learning is proposed. Dropout learning neglects some inputs and hidden units in the learning process with a probability, p, and then, the neglected inputs and hidden units are combined with the learned network to express the final output. We find that the process of combining the neglected hidden units with the learned network can be regarded as ensemble learning, so we analyze dropout learning from this point of view.Comment: 9 pages, 8 figures, submitted to Conferenc

On the Prospects for Laser Cooling of TlF

We measure the upper state lifetime and two ratios of vibrational branching fractions f_{v'v} on the B^{3}\Pi_{1}(v') - X^{1}\Sigma^{+}(v) transition of TlF. We find the B state lifetime to be 99(9) ns. We also determine that the off-diagonal vibrational decays are highly suppressed: f_{01}/f_{00} < 2x10^{-4} and f_{02}/f_{00} = 1.10(6)%, in excellent agreement with their predicted values of f_{01}/f_{00} < 8x10^{-4} and f_{02}/f_{00} = 1.0(2)% based on Franck-Condon factors calculated using Morse and RKR potentials. The implications of these results for the possible laser cooling of TlF and fundamental symmetries experiments are discussed.Comment: 5 pages, 2 figure

Linear response strength functions with iterative Arnoldi diagonalization

We report on an implementation of a new method to calculate RPA strength functions with iterative non-hermitian Arnoldi diagonalization method, which does not explicitly calculate and store the RPA matrix. We discuss the treatment of spurious modes, numerical stability, and how the method scales as the used model space is enlarged. We perform the particle-hole RPA benchmark calculations for double magic nucleus 132Sn and compare the resulting electromagnetic strength functions against those obtained within the standard RPA.Comment: 9 RevTeX pages, 11 figures, submitted to Physical Review

Minimizing Unsatisfaction in Colourful Neighbourhoods

Colouring sparse graphs under various restrictions is a theoretical problem of significant practical relevance. Here we consider the problem of maximizing the number of different colours available at the nodes and their neighbourhoods, given a predetermined number of colours. In the analytical framework of a tree approximation, carried out at both zero and finite temperatures, solutions obtained by population dynamics give rise to estimates of the threshold connectivity for the incomplete to complete transition, which are consistent with those of existing algorithms. The nature of the transition as well as the validity of the tree approximation are investigated.Comment: 28 pages, 12 figures, substantially revised with additional explanatio

Effect of Turbocharger Compression Ratio on Performance of the Spark-Ignition Internal Combustion Engine

Internal Combustion Engines (ICE) are one of the most important engineering applications that operate based on the conversion of chemical energy from fuel into thermal energy as a result of direct combustion. The obtained thermal energy is then turned into kinetic energy to derive various means of transportation, such as marine, air, and land vehicles. The efficiency of ICE today is considered in the range of the intermediate level, and various improvements are being made to enhance its efficiency. The turbocharger can support the ICE, which works by increasing the pressure in the engine to enhance its efficiency. In this investigation, the effect of the turbocharger pressure on ICE performance was studied in the range of 2 to 10 bar. It was found that the increase in turbocharger pressure enhanced the pressure inside the engine, positively affecting engine efficiency indicators. Therefore, the increase in turbocharger pressure is directly proportional to the ICE efficiency. Doi: 10.28991/ESJ-2022-06-03-04 Full Text: PD

Dynamical replica theoretic analysis of CDMA detection dynamics

We investigate the detection dynamics of the Gibbs sampler for code-division multiple access (CDMA) multiuser detection. Our approach is based upon dynamical replica theory which allows an analytic approximation to the dynamics. We use this tool to investigate the basins of attraction when phase coexistence occurs and examine its efficacy via comparison with Monte Carlo simulations.Comment: 18 pages, 2 figure

Impact of layer defects in ferroelectric thin films

Based on a modified Ising model in a transverse field we demonstrate that defect layers in ferroelectric thin films, such as layers with impurities, vacancies or dislocations, are able to induce a strong increase or decrease of the polarization depending on the variation of the exchange interaction within the defect layers. A Green's function technique enables us to calculate the polarization, the excitation energy and the critical temperature of the material with structural defects. Numerically we find the polarization as function of temperature, film thickness and the interaction strengths between the layers. The theoretical results are in reasonable accordance to experimental datas of different ferroelectric thin films.Comment: 17 pages, 8 figure