Learning Expressive Priors for Generalization and Uncertainty Estimation in Neural Networks

In this work, we propose a novel prior learning method for advancing generalization and uncertainty estimation in deep neural networks. The key idea is to exploit scalable and structured posteriors of neural networks as informative priors with generalization guarantees. Our learned priors provide expressive probabilistic representations at large scale, like Bayesian counterparts of pre-trained models on ImageNet, and further produce non-vacuous generalization bounds. We also extend this idea to a continual learning framework, where the favorable properties of our priors are desirable. Major enablers are our technical contributions: (1) the sums-of-Kronecker-product computations, and (2) the derivations and optimizations of tractable objectives that lead to improved generalization bounds. Empirically, we exhaustively show the effectiveness of this method for uncertainty estimation and generalization

Auxora versus standard of care for the treatment of severe or critical COVID-19 pneumonia: results from a randomized controlled trial

BACKGROUND: Calcium release-activated calcium (CRAC) channel inhibitors stabilize the pulmonary endothelium and block proinflammatory cytokine release, potentially mitigating respiratory complications observed in patients with COVID-19. This study aimed to investigate the safety and efficacy of Auxora, a novel, intravenously administered CRAC channel inhibitor, in adults with severe or critical COVID-19 pneumonia. METHODS: A randomized, controlled, open-label study of Auxora was conducted in adults with severe or critical COVID-19 pneumonia. Patients were randomized 2:1 to receive three doses of once-daily Auxora versus standard of care (SOC) alone. The primary objective was to assess the safety and tolerability of Auxora. Following FDA guidance, study enrollment was halted early to allow for transition to a randomized, blinded, placebo-controlled study. RESULTS: In total, 17 patients with severe and three with critical COVID-19 pneumonia were randomized to Auxora and nine with severe and one with critical COVID-19 pneumonia to SOC. Similar proportions of patients receiving Auxora and SOC experienced ≥ 1 adverse event (75% versus 80%, respectively). Fewer patients receiving Auxora experienced serious adverse events versus SOC (30% versus 50%, respectively). Two patients (10%) receiving Auxora and two (20%) receiving SOC died during the 30 days after randomization. Among patients with severe COVID-19 pneumonia, the median time to recovery with Auxora was 5 days versus 12 days with SOC; the recovery rate ratio was 1.87 (95% CI, 0.72, 4.89). Invasive mechanical ventilation was needed in 18% of patients with severe COVID-19 pneumonia receiving Auxora versus 50% receiving SOC (absolute risk reduction = 32%; 95% CI, - 0.07, 0.71). Outcomes measured by an 8-point ordinal scale were significantly improved for patients receiving Auxora, especially for patients with a baseline PaO(2)/FiO(2) = 101-200. CONCLUSIONS: Auxora demonstrated a favorable safety profile in patients with severe or critical COVID-19 pneumonia and improved outcomes in patients with severe COVID-19 pneumonia. These results, however, are limited by the open-label study design and small patient population resulting from the early cessation of enrollment in response to regulatory guidance. The impact of Auxora on respiratory complications in patients with severe COVID-19 pneumonia will be further assessed in a planned randomized, blinded, placebo-controlled study. TRIAL REGISTRATION: ClinicalTrials.gov, NCT04345614

Laparoscopic Drainage of a Hepatic Echinococcal Cyst: A Case Report

The Echinococcus granulosus tapeworm causes hepatic echinococcosis. It is endemic in the Mediterranean region, Middle East, and South America. Human infection is secondary to accidental consumption of ova in feces. Absorption through the bowel wall and entrance into the portal circulation leads to liver infection. This case involves a 34 y/o Moroccan male with an echinococcal liver cyst. His chief complaint was RUQ pain. The patient was treated with albendazole and praziquantel. His PMH and PSH was noncontributory. Patient was not on any other medications. ROS was otherwise unremarkable. The patient was AF VSS. He was tender to palpation in RUQ. Liver function tests were normal. Echinococcal titers were positive. CT demonstrated a large cystic lesion in the right lobe of the liver measuring 13.5 cm in diameter. The patient underwent successful laparoscopic drainage and excision of echinococcal cyst. Final pathology demonstrated degenerating parasites (E. granulosus) of echinococcal cyst

Combining deep learning and physical models for solar nowcasting

Sudden changes in solar irradiance on a local scale can significantly influence solar power generation. This intermittent characteristic of the solar resource is mainly caused by passing clouds and represents a challenge when solar energy is integrated into the power system. By making use of intra hour nowcasts (very short-term forecasts), changing conditions on solar irradiance can be anticipated, allowing optimized power plant operation and grid integration. All-sky imagers, capturing sky conditions at high spatial and temporal resolution, can be the basis of such nowcasting systems. However, the benefit of these nowcasting systems heavily depends on the accuracy of the predictions. In a previous work, a hybrid model combining physics-based and persistence nowcasts has proven to be advantageous. In this work, we present a novel deep learning (DL) model based on the transformer architecture for solar irradiance nowcasts and show that integrating this model into the hybrid model further improves the nowcast quality significantly. While the physics-based nowcasts are derived from a pipeline of processing steps to model clouds and anticipating their impact on solar irradiance, the DL model is completely data-driven and trained end-to-end using sequences of sky images and groundbased irradiance measurements as input. For comparison to the literature, evaluation is carried out on a benchmark dataset of 2019 from the same site. First, the nowcast quality of the DL model is analyzed independently on standard forecasting error metrics like root mean square error (RMSE), mean absolute error (MAE), mean bias error (MBE) and forecast skill. For computing the forecast skill, we used the so-called smart persistence (SP) as reference model. Reaching scores of over 28%, the DL model itself already outperforms the previous hybrid model in terms of RMSE. Next, the hybrid model, combining physics-based, DL and SP nowcasts, is evaluated on the same dataset using the same metrics. Compared to the previous hybrid model, the new hybrid model shows significant improvement over all metrics

Progressive Bayesian Neural Networks

Uncertainty estimates are crucial in many deep learning problems, e.g. for active learning or safety-critical applications. While Bayesian deep learning provides a framework to generate uncertainty estimates for deep learning models, it requires a well-specified prior which is in general unknown. This work aims to use large-scale datasets to learn an informative prior over the parameters of a neural network which can then be used in subsequent tasks to create better uncertainty estimations and tighter generalization bounds. The model uses scalable Laplace approximations to enable working with large-scale networks and datasets with little computational overhead compared to standard deep learning. Altogether, this transforms the problem of defining high-dimensional prior distributions with complex interactions between different weights to finding related datasets. To improve the generalization bounds for Laplace approximation, a novel method to scale the curvature using PAC-Bayesian bounds is proposed. For this, an approximate upper bound of the training error is derived for Laplace approximation that is optimized with respect to the curvature scales. Empirically, the learned prior needs less temperature scaling than isotropic Gaussian priors and produces similarly accurate predictions and uncertainty estimations. Moreover, non-vacuous generalization bounds are obtained for a LeNet-5 architecture on the NotMNIST dataset. In particular, the curvature scaling improves the bounds by up to 23 percent points while the empirically learned prior tightens the bound compared to isotropic Gaussian priors by an average of nine percent points, resulting in an upper bound of the generalization error of 65% on the NotMNIST dataset. Additionally, we introduce Progressive Bayesian Neural Networks (PBNN) that combine the learned prior with progressive neural networks to learn sequentially incoming tasks without catastrophic forgetting. Using an empirically learned prior on the ImageNet dataset, PBNN improve the accuracy and uncertainty on a large-scale robotics dataset compared to progressive neural networks and their variation with MC dropout. Moreover, we present a more accurate Kronecker-factorization of the Fisher Information Matrix (FIM) as an alternative to the widely adopted Kronecker-Factored Approximate Curvature (K-FAC). For this, we transform the optimal Kronecker-factored approximation of the FIM into a best rank-one approximation problem and solve this problem with a novel scalable version of the well-known power (iteration) method. In a proof-of-concept experiment, we show that the proposed algorithm can achieve more accurate estimates of the true FIM when compared to the K-FAC method

Sewing for Hitler? The clothing industry during the ‘Third Reich’

The article deals with the history of the clothing industry during the 'Third Reich'. It discusses the development of the industry and the room for manoeuvre by the example of three companies, Seidensticker, Hugo Boss, and Bierbaum-Proenen. The article makes the point that the Nazi's economic policy brought about severe restraints for the clothing industry, which seems to be typical for consumer goods industries as a whole during the 'Third Reich'

Handbook on loss reserving

This handbook presents the basic aspects of actuarial loss reserving. Besides the traditional methods, it also includes a description of more recent ones and a discussion of certain problems occurring in actuarial practice, like inflation, scarce data, large claims, slow loss development, the use of market statistics, the need for simulation techniques and the task of calculating best estimates and ranges of future losses. In property and casualty insurance the provisions for payment obligations from losses that have occurred but have not yet been settled usually constitute the largest item on the liabilities side of an insurer's balance sheet. For this reason, the determination and evaluation of these loss reserves is of considerable economic importance for every property and casualty insurer. Actuarial students, academics as well as practicing actuaries will benefit from this overview of the most important actuarial methods of loss reserving by developing an understanding of the underlying stochastic models and how to practically solve some problems which may occur in actuarial practice

Kronecker-Factored Optimal Curvature

The current scalable Bayesian methods for Deep Neural Networks (DNNs) often rely on the Fisher Information Matrix (FIM). For the tractable computations of the FIM, the Kronecker-Factored Approximate Curvature (K-FAC) method is widely adopted, which approximates the true FIM by a layer-wise block-diagonal matrix, and each diagonal block is then Kronecker-factored. In this paper, we propose an alternative formulation to obtain the Kronecker-factored FIM. The key insight is to cast the given FIM computations into an optimization problem over the sums of Kronecker products. In particular, we prove that this formulation is equivalent to the best rank-one approximation problem, where the well-known power iteration method is guaranteed to converge to an optimal rank-one solution - resulting in our novel algorithm: the Kronecker-Factored Optimal Curvature (K-FOC). In a proof-of-concept experiment, we show that the proposed algorithm can achieve more accurate estimates of the true FIM when compared to the K-FAC method