Stochastic expectation propagation

Expectation propagation (EP) is a deterministic approximation algorithm that is often used to perform approximate Bayesian parameter learning. EP approximates the full intractable posterior distribution through a set of local approximations that are iteratively refined for each datapoint. EP can offer analytic and computational advantages over other approximations, such as Variational Inference (VI), and is the method of choice for a number of models. The local nature of EP appears to make it an ideal candidate for performing Bayesian learning on large models in large-scale dataset settings. However, EP has a crucial limitation in this context: the number of approximating factors needs to increase with the number of data-points, N, which often entails a prohibitively large memory overhead. This paper presents an extension to EP, called stochastic expectation propagation (SEP), that maintains a global posterior approximation (like VI) but updates it in a local way (like EP). Experiments on a number of canonical learning problems using synthetic and real-world datasets indicate that SEP performs almost as well as full EP, but reduces the memory consumption by a factor of $N$. SEP is therefore ideally suited to performing approximate Bayesian learning in the large model, large dataset setting

Deep Gaussian processes for regression using approximate expectation propagation

Deep Gaussian processes (DGPs) are multi-layer hierarchical generalisations of Gaussian processes (GPs) and are formally equivalent to neural networks with multiple, infinitely wide hidden layers. DGPs are nonparametric probabilistic models and as such are arguably more flexible, have a greater capacity to generalise, and provide better calibrated uncertainty estimates than alternative deep models. This paper develops a new approximate Bayesian learning scheme that enables DGPs to be applied to a range of medium to large scale regression problems for the first time. The new method uses an approximate Expectation Propagation procedure and a novel and efficient extension of the probabilistic backpropagation algorithm for learning. We evaluate the new method for non-linear regression on eleven real-world datasets, showing that it always outperforms GP regression and is almost always better than state-of-the-art deterministic and sampling-based approximate inference methods for Bayesian neural networks. As a by-product, this work provides a comprehensive analysis of six approximate Bayesian methods for training neural networks

Black-Box α-divergence minimization

Black-box alpha (BB-α) is a new approximate inference method based on the minimization of α-divergences. BB-α scales to large datasets because it can be implemented using stochastic gradient descent. BB-α can be applied to complex probabilistic models with little effort since it only requires as input the likelihood function and its gradients. These gradients can be easily obtained using automatic differentiation. By changing the divergence parameter α, the method is able to interpolate between variational Bayes (VB) (α → 0) and an algorithm similar to expectation propagation (EP) (α = 1). Experiments on probit regression and neural network regression and classification problems show that BB-a with non-standard settings of α, such as α = 0.5, usually produces better predictions than with α → 0 (VB) or α = 1 (EP).JMHL acknowledges support from the Rafael del Pino Foundation. YL thanks the Schlumberger Foundation Faculty for the Future fellowship on supporting her PhD study. MR acknowledges support from UK Engineering and Physical Sciences Research Council (EPSRC) grant EP/L016516/1 for the University of Cambridge Centre for Doctoral Training, the Cambridge Centre for Analysis. TDB thanks Google for funding his European Doctoral Fellowship. DHL acknowledge support from Plan National I+D+i, Grant TIN2013-42351-P and TIN2015- 70308-REDT, and from Comunidad de Madrid, Grant S2013/ICE-2845 CASI-CAM-CM. RET thanks EPSRC grant #EP/L000776/1 and #EP/M026957/1

Sequence tutor: Conservative fine-tuning of sequence generation models with KL-control

This paper proposes a general method for improving the structure and quality of sequences generated by a recurrent neural network (RNN), while maintaining information originally learned from data, as well as sample diversity. An RNN is first pre-trained on data using maximum likelihood estimation (MLE), and the probability distribution over the next token in the sequence learned by this model is treated as a prior policy. Another RNN is then trained using reinforcement learning (RL) to generate higher-quality outputs that account for domain-specific incentives while retaining proximity to the prior policy of the MLE RNN. To formalize this objective, we derive novel off-policy RL methods for RNNs from KL-control. The effectiveness of the approach is demonstrated on two applications; 1) generating novel musical melodies, and 2) computational molecular generation. For both problems, we show that the proposed method improves the desired properties and structure of the generated sequences, while maintaining information learned from data

Deterministic variational inference for robust Bayesian neural networks

Bayesian neural networks (BNNs) hold great promise as a flexible and principled solution to deal with uncertainty when learning from finite data. Among approaches to realize probabilistic inference in deep neural networks, variational Bayes (VB) is theoretically grounded, generally applicable, and computationally efficient. With wide recognition of potential advantages, why is it that variational Bayes has seen very limited practical use for BNNs in real applications? We argue that variational inference in neural networks is fragile: successful implementations require careful initialization and tuning of prior variances, as well as controlling the variance of Monte Carlo gradient estimates. We provide two innovations that aim to turn VB into a robust inference tool for Bayesian neural networks: first, we introduce a novel deterministic method to approximate moments in neural networks, eliminating gradient variance; second, we introduce a hierarchical prior for parameters and a novel Empirical Bayes procedure for automatically selecting prior variances. Combining these two innovations, the resulting method is highly efficient and robust. On the application of heteroscedastic regression we demonstrate good predictive performance over alternative approaches

Understanding the role of imidazolium-based ionic liquids in the electrochemical CO2 reduction reaction

The development of efficient CO 2 capture and utilization technologies driven by renewable energy sources is mandatory to reduce the impact of climate change. Herein, seven imidazolium-based ionic liquids (ILs) with different anions and cations were tested as catholytes for the CO2 electrocatalytic reduction to CO over Ag electrode. Relevant activity and stability, but different selectivities for CO2 reduction or the side H 2 evolution were observed. Density functional theory results show that depending on the IL anions the CO 2 is captured or converted. Acetate anions (being strong Lewis bases) enhance CO2 capture and H2 evolution, while fluorinated anions (being weaker Lewis bases) favour the CO2 electroreduction. Differently from the hydrolytically unstable 1-butyl-3-methylimidazolium tetrafluoroborate, 1-Butyl-3-Methylimidazolium Triflate was the most promising IL, showing the highest Faradaic efficiency to CO (>95%), and up to 8 h of stable operation at high current rates (−20 mA & −60 mA), which opens the way for a prospective process scale-up

Cost-effectiveness of introducing a rotavirus vaccine in developing countries: The case of Mexico

<p>Abstract</p> <p>Background</p> <p>In developing countries rotavirus is the leading cause of severe diarrhoea and diarrhoeal deaths in children under 5. Vaccination could greatly alleviate that burden, but in Mexico as in most low- and middle-income countries the decision to add rotavirus vaccine to the national immunisation program will depend heavily on its cost-effectiveness and affordability. The objective of this study was to assess the cost-effectiveness of including the pentavalent rotavirus vaccine in Mexico's national immunisation program.</p> <p>Methods</p> <p>A cost-effectiveness model was developed from the perspective of the health system, modelling the vaccination of a hypothetical birth cohort of 2 million children monitored from birth through 60 months of age. It compares the cost and disease burden of rotavirus in an unvaccinated cohort of children with one vaccinated as recommended at 2, 4, and 6 months.</p> <p>Results</p> <p>Including the pentavalent vaccine in the national immunisation program could prevent 71,464 medical visits (59%), 5,040 hospital admissions (66%), and 612 deaths from rotavirus gastroenteritis (70%). At US$10 per dose and a cost of administration of US$13.70 per 3-dose regimen, vaccination would cost US$122,058 per death prevented, US$4,383 per discounted life-year saved, at a total net cost of US$74.7 million dollars to the health care system. Key variables influencing the results were, in order of importance, case fatality, vaccine price, vaccine efficacy, serotype prevalence, and annual loss of efficacy. The results are also very sensitive to the discount rate assumed when calculated per life-year saved.</p> <p>Conclusion</p> <p>At prices below US$15 per dose, the cost per life-year saved is estimated to be lower than one GNP per capita and hence highly cost effective by the WHO Commission on Macroeconomics and Health criteria. The cost-effectiveness estimates are highly dependent upon the mortality in the absence of the vaccine, which suggests that the vaccine is likely to be significantly more cost-effective among poorer populations and among those with less access to prompt medical care – such that poverty reduction programs would be expected to reduce the future cost-effectiveness of the vaccine.</p

Methodological guide for ethnobotanical study of forest species in Amazonian and related communities

The ethnobotanical study is carried out due to the accelerated loss of traditional knowledge, forest degradation and natural habitats in Amazonian communities. This work proposes a methodological guide that facilitates accessibility for obtaining information in the ethnobotanical study of superior lignified and ruderal species for medicinal purposes and other uses. A survey was made that proposes the technique Interlocutor - Medium - Interlocutor for the dialogue, with a scientific character and the handling of the survey in three stages: application, validation and generalization, obtaining the ethnobotanical information that the Amazonian communities treasure in agreement with their ethnicity of origin. The results demonstrated the effectiveness in the application of the survey through the relationship established between the researcher and community leaders, ancestral healers, farmers, housewives, people of different ages and other members with interest in the subject, facilitating accessibility for the location and identification of superior lignified and ruderal species for medicinal purposes and other uses

Scaling properties of protein family phylogenies

One of the classical questions in evolutionary biology is how evolutionary processes are coupled at the gene and species level. With this motivation, we compare the topological properties (mainly the depth scaling, as a characterization of balance) of a large set of protein phylogenies with a set of species phylogenies. The comparative analysis shows that both sets of phylogenies share remarkably similar scaling behavior, suggesting the universality of branching rules and of the evolutionary processes that drive biological diversification from gene to species level. In order to explain such generality, we propose a simple model which allows us to estimate the proportion of evolvability/robustness needed to approximate the scaling behavior observed in the phylogenies, highlighting the relevance of the robustness of a biological system (species or protein) in the scaling properties of the phylogenetic trees. Thus, the rules that govern the incapability of a biological system to diversify are equally relevant both at the gene and at the species level.Comment: Replaced with final published versio

Optimizing CIGB-300 intralesional delivery in locally advanced cervical cancer

Background:We conducted a phase 1 trial in patients with locally advanced cervical cancer by injecting 0.5 ml of the CK2-antagonist CIGB-300 in two different sites on tumours to assess tumour uptake, safety, pharmacodynamic activity and identify the recommended dose.Methods:Fourteen patients were treated with intralesional injections containing 35 or 70 mg of CIGB-300 in three alternate cycles of three consecutive days each before standard chemoradiotherapy. Tumour uptake was determined using 99 Tc-radiolabelled peptide. In situ B23/nucleophosmin was determined by immunohistochemistry.Results:Maximum tumour uptake for CIGB-300 70-mg dose was significantly higher than the one observed for 35 mg: 16.1±8.9 vs 31.3±12.9 mg (P=0.01). Both, AUC 24h and biological half-life were also significantly higher using 70 mg of CIGB-300 (P<0.001). Unincorporated CIGB-300 diffused rapidly to blood and was mainly distributed towards kidneys, and marginally in liver, lungs, heart and spleen. There was no DLT and moderate allergic-like reactions were the most common systemic side effect with strong correlation between unincorporated CIGB-300 and histamine levels in blood. CIGB-300, 70 mg, downregulated B23/nucleophosmin (P=0.03) in tumour specimens.Conclusion:Intralesional injections of 70 mg CIGB-300 in two sites (0.5 ml per injection) and this treatment plan are recommended to be evaluated in phase 2 studies.Fil: Sarduy, M. R.. Medical-surgical Research Center; CubaFil: García, I.. Centro de Ingeniería Genética y Biotecnología; CubaFil: Coca, M. A.. Clinical Investigation Center; CubaFil: Perera, A.. Clinical Investigation Center; CubaFil: Torres, L. A.. Clinical Investigation Center; CubaFil: Valenzuela, C. M.. Centro de Ingeniería Genética y Biotecnología; CubaFil: Baladrón, I.. Centro de Ingeniería Genética y Biotecnología; CubaFil: Solares, M.. Hospital Materno Ramón González Coro; CubaFil: Reyes, V.. Center For Genetic Engineering And Biotechnology Havana; CubaFil: Hernández, I.. Isotope Center; CubaFil: Perera, Y.. Centro de Ingeniería Genética y Biotecnología; CubaFil: Martínez, Y. M.. Medical-surgical Research Center; CubaFil: Molina, L.. Medical-surgical Research Center; CubaFil: González, Y. M.. Medical-surgical Research Center; CubaFil: Ancízar, J. A.. Centro de Ingeniería Genética y Biotecnología; CubaFil: Prats, A.. Clinical Investigation Center; CubaFil: González, L.. Centro de Ingeniería Genética y Biotecnología; CubaFil: Casacó, C. A.. Clinical Investigation Center; CubaFil: Acevedo, B. E.. Centro de Ingeniería Genética y Biotecnología; CubaFil: López Saura, P. A.. Centro de Ingeniería Genética y Biotecnología; CubaFil: Alonso, Daniel Fernando. Universidad Nacional de Quilmes; ArgentinaFil: Gómez, R.. Elea Laboratories; ArgentinaFil: Perea Rodríguez, S. E.. Center For Genetic Engineering And Biotechnology Havana; Cuba. Centro de Ingeniería Genética y Biotecnología; Cub