Multi-layer Relation Networks

Relational Networks (RN) as introduced by Santoro et al. (2017) have demonstrated strong relational reasoning capabilities with a rather shallow architecture. Its single-layer design, however, only considers pairs of information objects, making it unsuitable for problems requiring reasoning across a higher number of facts. To overcome this limitation, we propose a multi-layer relation network architecture which enables successive refinements of relational information through multiple layers. We show that the increased depth allows for more complex relational reasoning by applying it to the bAbI 20 QA dataset, solving all 20 tasks with joint training and surpassing the state-of-the-art results

A Simple Model of Evolution with Variable System Size

A simple model of biological extinction with variable system size is presented that exhibits a power-law distribution of extinction event sizes. The model is a generalization of a model recently introduced by Newman (Proc. R. Soc. Lond. B265, 1605 (1996). Both analytical and numerical analysis show that the exponent of the power-law distribution depends only marginally on the growth rate $g$ at which new species enter the system and is equal to the one of the original model in the limit $g\to\infty$. A critical growth rate $g_c$ can be found below which the system dies out. Under these model assumptions stable ecosystems can only exist if the regrowth of species is sufficiently fast.Comment: 5 pages, RevTeX, with 5 figures, revised version accepted for publication in Phys. Rev.

Predicting economic growth with classical physics and human biology

We collect and analyze the data for working time, life expectancy, and the pair output and infrastructure of industrializing nations. During S-functional recovery from disaster the pair's time shifts yield 25 years for the infrastructure's physical lifetime. At G7 level the per capita outputs converge and the time shifts identify a heritable quantity with a reaction time of 62 years. It seems to control demand and the spare time required for enjoying G7 affluence. The sum of spare and working time is fixed by the universal flow of time. This yields analytic solutions for equilibrium, recovery, and long-term evolution for all six variables with biologically stabilized parameters

The physics of business cycles and inflation

We analyse four consecutive cycles observed in the USA for employment and inflation. They are driven by three oil price shocks and an intended interest rate shock. Non-linear coupling between the rate equations for consumer products as prey and consumers as predators provides the required instability, but its natural damping is too high for spontaneous cycles. Extending the Lotka-Volterra equations with a small term for collective anticipation yields a second analytic solution without damping. It predicts the base period, phase shifts, and the sensitivity to shocks for all six cyclic variables correctly

Recursive Autoconvolution for Unsupervised Learning of Convolutional Neural Networks

In visual recognition tasks, such as image classification, unsupervised learning exploits cheap unlabeled data and can help to solve these tasks more efficiently. We show that the recursive autoconvolution operator, adopted from physics, boosts existing unsupervised methods by learning more discriminative filters. We take well established convolutional neural networks and train their filters layer-wise. In addition, based on previous works we design a network which extracts more than 600k features per sample, but with the total number of trainable parameters greatly reduced by introducing shared filters in higher layers. We evaluate our networks on the MNIST, CIFAR-10, CIFAR-100 and STL-10 image classification benchmarks and report several state of the art results among other unsupervised methods.Comment: 8 pages, accepted to International Joint Conference on Neural Networks (IJCNN 2017

On the boundedness of an iteration involving points on the hypersphere

For a finite set of points $X$ on the unit hypersphere in $\mathbb{R}^d$ we consider the iteration $u_{i+1}=u_i+\chi_i$, where $\chi_i$ is the point of $X$ farthest from $u_i$. Restricting to the case where the origin is contained in the convex hull of $X$ we study the maximal length of $u_i$. We give sharp upper bounds for the length of $u_i$ independently of $X$. Precisely, this upper bound is infinity for $d\ge 3$ and $\sqrt2$ for $d=2$

Gas-induced friction and diffusion of rigid rotors

We derive the Boltzmann equation for the rotranslational dynamics of an arbitrary convex rigid body in a rarefied gas. It yields as a limiting case the Fokker-Planck equation accounting for friction, diffusion, and nonconservative drift forces and torques. We provide the rotranslational friction and diffusion tensors for specular and diffuse reflection off particles with spherical, cylindrical, and cuboidal shape, and show that the theory describes thermalization, photophoresis, and the inverse Magnus effect in the free molecular regime.Comment: 13 pages, corrected typos, extended caption of Fig.

Committees of deep feedforward networks trained with few data

Deep convolutional neural networks are known to give good results on image classification tasks. In this paper we present a method to improve the classification result by combining multiple such networks in a committee. We adopt the STL-10 dataset which has very few training examples and show that our method can achieve results that are better than the state of the art. The networks are trained layer-wise and no backpropagation is used. We also explore the effects of dataset augmentation by mirroring, rotation, and scaling

The phase response of the cortical slow oscillation

Cortical slow oscillations occur in the mammalian brain during deep sleep and have been shown to contribute to memory consolidation, an effect that can be enhanced by electrical stimulation. As the precise underlying working mechanisms are not known it is desired to develop and analyze computational models of slow oscillations and to study the response to electrical stimuli. In this paper we employ the conductance based model of Compte et al. [J Neurophysiol 89, 2707] to study the effect of electrical stimulation. The population response to electrical stimulation depends on the timing of the stimulus with respect to the state of the slow oscillation. First, we reproduce the experimental results of electrical stimulation in ferret brain slices by Shu et al. [Nature 423, 288] from the conductance based model. We then numerically obtain the phase response curve for the conductance based network model to quantify the network's response to weak stimuli. Our results agree with experiments in vivo and in vitro that show that sensitivity to stimulation is weaker in the up than in the down state. However, we also find that within the up state stimulation leads to a shortening of the up state, or phase advance, whereas during the up-down transition a prolongation of up states is possible, resulting in a phase delay. Finally, we compute the phase response curve for the simple mean-field model by Ngo et al. [Europhys Lett 89, 68002] and find that the qualitative shape of the PRC is preserved, despite its different mechanism for the generation of slow oscillations

A physical theory of economic growth

Economic growth is unpredictable unless demand is quantified. We solve this problem by introducing the demand for unpaid spare time and a user quantity named human capacity. It organizes and amplifies spare time required for enjoying affluence like physical capital, the technical infrastructure for production, organizes and amplifies working time for supply. The sum of annual spare and working time is fixed by the universal flow of time. This yields the first macroeconomic equilibrium condition. Both storable quantities form stabilizing feedback loops. They are driven with the general and technical knowledge embodied with parts of the supply by education and construction. Linear amplification yields S-functions as only analytic solutions. Destructible physical capital controls medium-term recoveries from disaster. Indestructible human capacity controls the collective long-term industrial evolution. It is immune even to world wars and runs from 1800 to date parallel to the unisex life expectancy in the pioneering nations. This is the first quantitative information on long-term demand. The theory is self-consistent. It reproduces all peaceful data from 1800 to date without adjustable parameter. It has full forecasting power since the decisive parameters are constants of the human species. They predict an asymptotic maximum for the economic level per capita. Long-term economic growth appears as a part of natural science