Binary domain generalization for sparsifying binary neural networks

Binary neural networks (BNNs) are an attractive solution for developing and deploying deep neural network (DNN)-based applications in resource constrained devices. Despite their success, BNNs still suffer from a fixed and limited compression factor that may be explained by the fact that existing pruning methods for full-precision DNNs cannot be directly applied to BNNs. In fact, weight pruning of BNNs leads to performance degradation, which suggests that the standard binarization domain of BNNs is not well adapted for the task. This work proposes a novel more general binary domain that extends the standard binary one that is more robust to pruning techniques, thus guaranteeing improved compression and avoiding severe performance losses. We demonstrate a closed-form solution for quantizing the weights of a full-precision network into the proposed binary domain. Finally, we show the flexibility of our method, which can be combined with other pruning strategies. Experiments over CIFAR-10 and CIFAR-100 demonstrate that the novel approach is able to generate efficient sparse networks with reduced memory usage and run-time latency, while maintaining performance.Comment: Accepted as conference paper at ECML PKDD 202

Elastic Registration of Geodesic Vascular Graphs

Vascular graphs can embed a number of high-level features, from morphological parameters, to functional biomarkers, and represent an invaluable tool for longitudinal and cross-sectional clinical inference. This, however, is only feasible when graphs are co-registered together, allowing coherent multiple comparisons. The robust registration of vascular topologies stands therefore as key enabling technology for group-wise analyses. In this work, we present an end-to-end vascular graph registration approach, that aligns networks with non-linear geometries and topological deformations, by introducing a novel overconnected geodesic vascular graph formulation, and without enforcing any anatomical prior constraint. The 3D elastic graph registration is then performed with state-of-the-art graph matching methods used in computer vision. Promising results of vascular matching are found using graphs from synthetic and real angiographies. Observations and future designs are discussed towards potential clinical applications

In vitro and in vivo comparison of the anti-staphylococcal efficacy of generic products and the innovator of oxacillin

<p>Abstract</p> <p>Background</p> <p>Oxacillin continues to be an important agent in the treatment of staphylococcal infections; many generic products are available and the only requirement for their approval is demonstration of pharmaceutical equivalence. We tested the assumption that pharmaceutical equivalence predicts therapeutic equivalence by comparing 11 generics with the innovator product in terms of concentration of the active pharmaceutical ingredient (API), minimal inhibitory (MIC) and bactericidal concentrations (MBC), and antibacterial efficacy in the neutropenic mouse thigh infection model.</p> <p>Methods</p> <p>The API in each product was measured by a validated microbiological assay and compared by slope (potency) and intercept (concentration) analysis of linear regressions. MIC and MBC were determined by broth microdilution according to Clinical and Laboratory Standard Institute (CLSI) guidelines. For in vivo efficacy, neutropenic ICR mice were inoculated with a clinical strain of <it>Staphylococcus aureus</it>. The animals had 4.14 ± 0.18 log₁₀CFU/thigh when treatment started. Groups of 10 mice per product received a total dose ranging from 2.93 to 750 mg/kg per day administered q1h. Sigmoidal dose-response curves were generated by nonlinear regression fitted to Hill equation to compute maximum effect (E_max), slope (N), and the effective dose reaching 50% of the E_max(ED₅₀). Based on these results, bacteriostatic dose (BD) and dose needed to kill the first log of bacteria (1LKD) were also determined.</p> <p>Results</p> <p>4 generic products failed pharmaceutical equivalence due to significant differences in potency; however, all products were undistinguishable from the innovator in terms of MIC and MBC. Independently of their status with respect to pharmaceutical equivalence or in vitro activity, all generics failed therapeutic equivalence in vivo, displaying significantly lower E_maxand requiring greater BD and 1LKD, or fitting to a non-sigmoidal model.</p> <p>Conclusions</p> <p>Pharmaceutical or in vitro equivalence did not entail therapeutic equivalence for oxacillin generic products, indicating that criteria for approval deserve review to include evaluation of in vivo efficacy.</p

Do Deep Neural Networks Contribute to Multivariate Time Series Anomaly Detection?

Anomaly detection in time series is a complex task that has been widely studied. In recent years, the ability of unsupervised anomaly detection algorithms has received much attention. This trend has led researchers to compare only learning-based methods in their articles, abandoning some more conventional approaches. As a result, the community in this field has been encouraged to propose increasingly complex learning-based models mainly based on deep neural networks. To our knowledge, there are no comparative studies between conventional, machine learning-based and, deep neural network methods for the detection of anomalies in multivariate time series. In this work, we study the anomaly detection performance of sixteen conventional, machine learning-based and, deep neural network approaches on five real-world open datasets. By analyzing and comparing the performance of each of the sixteen methods, we show that no family of methods outperforms the others. Therefore, we encourage the community to reincorporate the three categories of methods in the anomaly detection in multivariate time series benchmarks

Improved optimization strategies for deep Multi-Task Networks

In Multi-Task Learning (MTL), it is a common practice to train multi-task networks by optimizing an objective function, which is a weighted average of the task-specific objective functions. Although the computational advantages of this strategy are clear, the complexity of the resulting loss landscape has not been studied in the literature. Arguably, its optimization may be more difficult than a separate optimization of the constituting task-specific objectives. In this work, we investigate the benefits of such an alternative, by alternating independent gradient descent steps on the different task-specific objective functions and we formulate a novel way to combine this approach with state-of-the-art optimizers. As the separation of task-specific objectives comes at the cost of increased computational time, we propose a random task grouping as a trade-off between better optimization and computational efficiency. Experimental results over three well-known visual MTL datasets show better overall absolute performance on losses and standard metrics compared to an averaged objective function and other state-of-the-art MTL methods. In particular, our method shows the most benefits when dealing with tasks of different nature and it enables a wider exploration of the shared parameter space. We also show that our random grouping strategy allows to trade-off between these benefits and computational efficiency