Integration of Leaky-Integrate-and-Fire-Neurons in Deep Learning Architectures

Up to now, modern Machine Learning is mainly based on fitting high dimensional functions to enormous data sets, taking advantage of huge hardware resources. We show that biologically inspired neuron models such as the Leaky-Integrate-and-Fire (LIF) neurons provide novel and efficient ways of information encoding. They can be integrated in Machine Learning models, and are a potential target to improve Machine Learning performance. Thus, we derived simple update-rules for the LIF units from the differential equations, which are easy to numerically integrate. We apply a novel approach to train the LIF units supervisedly via backpropagation, by assigning a constant value to the derivative of the neuron activation function exclusively for the backpropagation step. This simple mathematical trick helps to distribute the error between the neurons of the pre-connected layer. We apply our method to the IRIS blossoms image data set and show that the training technique can be used to train LIF neurons on image classification tasks. Furthermore, we show how to integrate our method in the KERAS (tensorflow) framework and efficiently run it on GPUs. To generate a deeper understanding of the mechanisms during training we developed interactive illustrations, which we provide online. With this study we want to contribute to the current efforts to enhance Machine Intelligence by integrating principles from biology

Sparsity through evolutionary pruning prevents neuronal networks from overfitting

Modern Machine learning techniques take advantage of the exponentially rising calculation power in new generation processor units. Thus, the number of parameters which are trained to resolve complex tasks was highly increased over the last decades. However, still the networks fail - in contrast to our brain - to develop general intelligence in the sense of being able to solve several complex tasks with only one network architecture. This could be the case because the brain is not a randomly initialized neural network, which has to be trained by simply investing a lot of calculation power, but has from birth some fixed hierarchical structure. To make progress in decoding the structural basis of biological neural networks we here chose a bottom-up approach, where we evolutionarily trained small neural networks in performing a maze task. This simple maze task requires dynamical decision making with delayed rewards. We were able to show that during the evolutionary optimization random severance of connections lead to better generalization performance of the networks compared to fully connected networks. We conclude that sparsity is a central property of neural networks and should be considered for modern Machine learning approaches

Data from: A remote-controlled observatory for behavioural and ecological research: a case study on emperor penguins

1. Long-term photographic recordings of animal populations provide unique insights in ecological and evolutionary processes. However, image acquisition at remote locations under harsh climatic conditions is highly challenging. 2. We present a robust, energetically self-sufficient and remote-controlled observatory designed to operate year-round in the Antarctic at temperatures below -50 °C and wind speeds above 150 km/h. The observatory is equipped with multiple overview cameras and a high resolution steerable camera with a telephoto lens for capturing images with high spatial and temporal resolution. 3. Our observatory has been in operation since 2013 to investigate an emperor penguin (Aptenodytes forsteri) colony at Atka Bay near the German Neumayer III research station. Data recorded by this observatory give novel biological insights in animal life cycle and demographic trends, but also in collective and individual behaviour. As an example, we present data showing how wind speed and direction influence movements of the entire colony and of individual penguins. We also estimate daily fluctuations in the total number of individuals present at the breeding site. 4. Our results demonstrate that remote-controlled observation systems can bridge the gap between remote sensing, simple time-lapse recording setups, and on-site observations by human investigators to collect unique biological datasets of undisturbed animal populations

The desmin mutation R349P increases contractility and fragility of stem cell-generated muscle micro-tissues

Aims Desminopathies comprise hereditary myopathies and cardiomyopathies caused by mutations in the intermediate filament protein desmin that lead to severe and often lethal degeneration of striated muscle tissue. Animal and single cell studies hinted that this degeneration process is associated with massive ultrastructural defects correlating with increased susceptibility of the muscle to acute mechanical stress. The underlying mechanism of mechanical susceptibility, and how muscle degeneration develops over time, however, has remained elusive. Methods Here, we investigated the effect of a desmin mutation on the formation, differentiation, and contractile function of in vitro-engineered three-dimensional micro-tissues grown from muscle stem cells (satellite cells) isolated from heterozygous R349P desmin knock-in mice. Results Micro-tissues grown from desmin-mutated cells exhibited spontaneous unsynchronised contractions, higher contractile forces in response to electrical stimulation, and faster force recovery compared with tissues grown from wild-type cells. Within 1 week of culture, the majority of R349P desmin-mutated tissues disintegrated, whereas wild-type tissues remained intact over at least three weeks. Moreover, under tetanic stimulation lasting less than 5 s, desmin-mutated tissues partially or completely ruptured, whereas wild-type tissues did not display signs of damage. Conclusions Our results demonstrate that the progressive degeneration of desmin-mutated micro-tissues is closely linked to extracellular matrix fibre breakage associated with increased contractile forces and unevenly distributed tensile stress. This suggests that the age-related degeneration of skeletal and cardiac muscle in patients suffering from desminopathies may be similarly exacerbated by mechanical damage from high-intensity muscle contractions. We conclude that micro-tissues may provide a valuable tool for studying the organization of myocytes and the pathogenic mechanisms of myopathies