PredProp: Bidirectional Stochastic Optimization with Precision Weighted Predictive Coding

We present PredProp, a method for bidirectional, parallel and local optimisation of weights, activities and precision in neural networks. PredProp jointly addresses inference and learning, scales learning rates dynamically and weights gradients by the curvature of the loss function by optimizing prediction error precision. PredProp optimizes network parameters with Stochastic Gradient Descent and error forward propagation based strictly on prediction errors and variables locally available to each layer. Neighboring layers optimise shared activity variables so that prediction errors can propagate forward in the network, while predictions propagate backwards. This process minimises the negative Free Energy, or evidence lower bound of the entire network. We show that networks trained with PredProp resemble gradient based predictive coding when the number of weights between neighboring activity variables is one. In contrast to related work, PredProp generalizes towards backward connections of arbitrary depth and optimizes precision for any deep network architecture. Due to the analogy between prediction error precision and the Fisher information for each layer, PredProp implements a form of Natural Gradient Descent. When optimizing DNN models, layer-wise PredProp renders the model a bidirectional predictive coding network. Alternatively DNNs can parameterize the weights between two activity variables. We evaluate PredProp for dense DNNs on simple inference, learning and combined tasks. We show that, without an explicit sampling step in the network, PredProp implements a form of variational inference that allows to learn disentangled embeddings from low amounts of data and leave evaluation on more complex tasks and datasets to future work

PredNet and Predictive Coding: A Critical Review

PredNet, a deep predictive coding network developed by Lotter et al., combines a biologically inspired architecture based on the propagation of prediction error with self-supervised representation learning in video. While the architecture has drawn a lot of attention and various extensions of the model exist, there is a lack of a critical analysis. We fill in the gap by evaluating PredNet both as an implementation of the predictive coding theory and as a self-supervised video prediction model using a challenging video action classification dataset. We design an extended model to test if conditioning future frame predictions on the action class of the video improves the model performance. We show that PredNet does not yet completely follow the principles of predictive coding. The proposed top-down conditioning leads to a performance gain on synthetic data, but does not scale up to the more complex real-world action classification dataset. Our analysis is aimed at guiding future research on similar architectures based on the predictive coding theory

3D printing of sacrificial templates into hierarchical porous materials

Hierarchical porous materials are widespread in nature and find an increasing number of applications as catalytic supports, biological scaffolds and lightweight structures. Recent advances in additive manufacturing and 3D printing technologies have enabled the digital fabrication of porous materials in the form of lattices, cellular structures and foams across multiple length scales. However, current approaches do not allow for the fast manufacturing of bulk porous materials featuring pore sizes that span broadly from macroscopic dimensions down to the nanoscale. Here, ink formulations are designed and investigated to enable 3D printing of hierarchical materials displaying porosity at the nano-, micro- and macroscales. Pores are generated upon removal of nanodroplets and microscale templates present in the initial ink. Using particles to stabilize the droplet templates is key to obtain Pickering nanoemulsions that can be 3D printed through direct ink writing. The combination of such self-assembled templates with the spatial control offered by the printing process allows for the digital manufacturing of hierarchical materials exhibiting thus far inaccessible multiscale porosity and complex geometries.ISSN:2045-232

LocText: relation extraction of protein localizations to assist database curation

Abstract Background The subcellular localization of a protein is an important aspect of its function. However, the experimental annotation of locations is not even complete for well-studied model organisms. Text mining might aid database curators to add experimental annotations from the scientific literature. Existing extraction methods have difficulties to distinguish relationships between proteins and cellular locations co-mentioned in the same sentence. Results LocText was created as a new method to extract protein locations from abstracts and full texts. LocText learned patterns from syntax parse trees and was trained and evaluated on a newly improved LocTextCorpus. Combined with an automatic named-entity recognizer, LocText achieved high precision (P = 86%±4). After completing development, we mined the latest research publications for three organisms: human (Homo sapiens), budding yeast (Saccharomyces cerevisiae), and thale cress (Arabidopsis thaliana). Examining 60 novel, text-mined annotations, we found that 65% (human), 85% (yeast), and 80% (cress) were correct. Of all validated annotations, 40% were completely novel, i.e. did neither appear in the annotations nor the text descriptions of Swiss-Prot. Conclusions LocText provides a cost-effective, semi-automated workflow to assist database curators in identifying novel protein localization annotations. The annotations suggested through text-mining would be verified by experts to guarantee high-quality standards of manually-curated databases such as Swiss-Prot

Maternal exposure to ambient air pollution and risk of early childhood cancers: A population-based study in Ontario, Canada

Background: There are increasing concerns regarding the role of exposure to ambient air pollution during pregnancy in the development of early childhood cancers. Objective: This population based study examined whether prenatal and early life (<1year of age) exposures to ambient air pollutants, including nitrogen dioxide (NO2) and particulate matter with aerodynamic diameters ≤2.5μm (PM2.5), were associated with selected common early childhood cancers in Canada. Methods: 2,350,898 singleton live births occurring between 1988 and 2012 were identified in the province of Ontario, Canada. We assigned temporally varying satellite-derived estimates of PM2.5 and land-use regression model estimates of NO2 to maternal residences during pregnancy. Incident cases of 13 subtypes of pediatric cancers among children up to age 6 until 2013 were ascertained through administrative health data linkages. Associations of trimester-specific, overall pregnancy and first year of life exposures were evaluated using Cox proportional hazards models, adjusting for potential confounders. Results: A total of 2044 childhood cancers were identified. Exposure to PM2.5, per interquartile range increase, over the entire pregnancy, and during the first trimester was associated with an increased risk of astrocytoma (hazard ratio (HR) per 3.9μg/m3=1.38 (95% CI: 1.01, 1.88) and, HR per 4.0μg/m3 =1.40 (95% CI: 1.05-1.86), respectively). We also found a positive association between first trimester NO2 and acute lymphoblastic leukemia (ALL) (HR=1.20 (95% CI: 1.02-1.41) per IQR (13.3ppb)). Conclusions: In this population-based study in the largest province of Canada, results suggest an association between exposure to ambient air pollution during pregnancy, especially in the first trimester and an increased risk of astrocytoma and ALL. Further studies are required to replicate the findings of this study with adjustment for important individual-level confounders