Dynamics of Learning with Restricted Training Sets I: General Theory

We study the dynamics of supervised learning in layered neural networks, in the regime where the size $p$ of the training set is proportional to the number $N$ of inputs. Here the local fields are no longer described by Gaussian probability distributions and the learning dynamics is of a spin-glass nature, with the composition of the training set playing the role of quenched disorder. We show how dynamical replica theory can be used to predict the evolution of macroscopic observables, including the two relevant performance measures (training error and generalization error), incorporating the old formalism developed for complete training sets in the limit $\alpha=p/N\to\infty$ as a special case. For simplicity we restrict ourselves in this paper to single-layer networks and realizable tasks.Comment: 39 pages, LaTe

On-Line Learning Theory of Soft Committee Machines with Correlated Hidden Units - Steepest Gradient Descent and Natural Gradient Descent -

The permutation symmetry of the hidden units in multilayer perceptrons causes the saddle structure and plateaus of the learning dynamics in gradient learning methods. The correlation of the weight vectors of hidden units in a teacher network is thought to affect this saddle structure, resulting in a prolonged learning time, but this mechanism is still unclear. In this paper, we discuss it with regard to soft committee machines and on-line learning using statistical mechanics. Conventional gradient descent needs more time to break the symmetry as the correlation of the teacher weight vectors rises. On the other hand, no plateaus occur with natural gradient descent regardless of the correlation for the limit of a low learning rate. Analytical results support these dynamics around the saddle point.Comment: 7 pages, 6 figure

Driver Distraction Identification with an Ensemble of Convolutional Neural Networks

The World Health Organization (WHO) reported 1.25 million deaths yearly due to road traffic accidents worldwide and the number has been continuously increasing over the last few years. Nearly fifth of these accidents are caused by distracted drivers. Existing work of distracted driver detection is concerned with a small set of distractions (mostly, cell phone usage). Unreliable ad-hoc methods are often used.In this paper, we present the first publicly available dataset for driver distraction identification with more distraction postures than existing alternatives. In addition, we propose a reliable deep learning-based solution that achieves a 90% accuracy. The system consists of a genetically-weighted ensemble of convolutional neural networks, we show that a weighted ensemble of classifiers using a genetic algorithm yields in a better classification confidence. We also study the effect of different visual elements in distraction detection by means of face and hand localizations, and skin segmentation. Finally, we present a thinned version of our ensemble that could achieve 84.64% classification accuracy and operate in a real-time environment.Comment: arXiv admin note: substantial text overlap with arXiv:1706.0949

Impact of nitrogen regime on fatty acid profiles of Desmodesmus quadricaudatus and Chlorella sp. and ability to produce biofuel

Abstract Microalgae have emerged as one of the most promising sources for fatty acid production. Since the various fatty acid profiles (chain length, degree of unsaturation, and branching of the chain) of the different sources influence biodiesel fuel properties, it is important to possess data on how the presence of NaNO3 as nitrogen source can influence the profile of produced fatty acids from algae. The fatty acid profiles of Desmodesmus quadricaudatus and Chlorella sp. were detected in pure batch cultures experiments. BG-11 nitrogen free medium and the medium contained 1.5 g NaNO3 l−1 were used in this investigation. At late stationary growth phase in nitrogen free medium, Chlorella sp. produced 58.39% saturated fatty acids and 41.60% unsaturated fatty acids. While in medium contained 1.5 g NaNO3 l−1Chlorella sp. produced 62.08% saturated fatty acids and 37.92% unsaturated fatty acids. In nitrogen free medium D. quadricaudatus produced 66.92% saturated fatty acids and 33.07% unsaturated fatty acids. While in cultures contained 1.5 g NaNO3 l−1D. quadricaudatus produced 51.62% saturated fatty acids and 48.37% unsaturated fatty acids. The fatty acid profile of Chlorella sp. and D. quadricaudatus that isolated from Egyptian water body and grown in nitrogen free medium may be suitable for biodiesel production. The results discussed and compared to fatty acid profiles produced by other algal species

Strain Gradients in Epitaxial Ferroelectrics

X-ray analysis of ferroelectric thin layers of Ba1/2Sr1/2TiO3 with different thickness reveals the presence of internal strain gradients across the film thickness and allows us to propose a functional form for the internal strain profile. We use this to calculate the direct influence of strain gradient, through flexoelectric coupling, on the degradation of the ferroelectric properties of thin films with decreasing thickness, in excellent agreement with the observed behaviour. This work highlights the link between strain relaxation and strain gradients in epitaxial films, and shows the pressing need to avoid strain gradients in order to obtain thin ferroelectrics with bulk-like properties.Comment: 4 pages, 3 embedded figures (1 color), revTex

On the behaviour of solutions of the two-cell cubic autocatalator reaction model

Approximate solutions of the initial value problem for reaction diffusion equations in two regions (cells) are obtained. The system is considered here with two chemical species, species $A$ and the autocatalyst $B$. The reaction is taken to be cubic in the autocatalysis in the first region with linear exchange through $A$. In the first region, the autocatalyst is taken to decay linearly. Approximate solutions are found through the Picard iterative sequence of solutions. The space and time variations of the concentration of the species $A$ and $B$ are evaluated in the two regions. The oscillation of the concentrations in times has been observed in different locations. This phenomena is stepped out for relatively large times. Comparison between two consecutive solutions is made. The maximum error estimate is of order $10^{-3}$ for some appropriate time period. At this time level, the solutions obtained are adequate for laboratory simulation experiments to open systems. It is observed that no initiation to travelling waves occurs whenever the initial values of the concentrations of the reactant (or the autocatalysts) are not periodic