Understanding BatchNorm in Ternary Training

Neural networks are comprised of two components, weights andactivation function. Ternary weight neural networks (TNNs) achievea good performance and offer up to 16x compression ratio. TNNsare difficult to train without BatchNorm and there has been no studyto clarify the role of BatchNorm in a ternary network. Benefitingfrom a study in binary networks, we show how BatchNorm helps inresolving the exploding gradients issue

Deep Learning Inference Frameworks for ARM CPU

The deep learning community focuses on training networks for a better accuracy on GPU servers. However, bringing this technology to consumer products requires inference adaptation of suchInstruction networks for low-energy, small-memory, and computationally constrained edge devices. ARM CPU is one of the important components of edge devices, but a clear comparison between the existinginference frameworks is missing. We provide minimal preliminaries about ARM CPU architecture and briefly mention the difference between the existing inference frameworks to evaluate them based on performance versus usability trade-offs

Binary Quantizer

One-bit quantization is a general tool to execute a complex model,such as deep neural networks, on a device with limited resources,such as cell phones. Naively compressing weights into one bityields an extensive accuracy loss. One-bit models, therefore, re-quire careful re-training. Here we introduce a class functions de-vised to be used as a regularizer for re-training one-bit models. Us-ing a regularization function, specifically devised for binary quanti-zation, avoids heuristic touch of the optimization scheme and savesconsiderable coding effort

Fast high-dimensional Bayesian classification and clustering

We introduce a fast approach to classification and clustering applicable to high-dimensional continuous data, based on Bayesian mixture models for which explicit computations are available. This permits us to treat classification and clustering in a single framework, and allows calculation of unobserved class probability. The new classifier is robust to adding noise variables as a drawback of the built-in spike-and-slab structure of the proposed Bayesian model. The usefulness of classification using our method is shown on metabololomic example, and on the Iris data with and without noise variables. Agglomerative hierarchical clustering is used to construct a dendrogram based on the posterior probabilities of particular partitions, to provide a dendrogram with a probabilistic interpretation. An extension to variable selection is proposed which summarises the importance of variables for classification or clustering and has probabilistic interpretation. Having a simple model provides estimation of the model parameters using maximum likelihood and therefore yields a fully automatic algorithm. The new clustering method is applied to metabolomic, microarray, and image data and is studied using simulated data motivated by real datasets. The computational difficulties of the new approach are discussed, solutions for algorithm acceleration are proposed, and the written computer code is briefly analysed. Simulations shows that the quality of the estimated model parameters depends on the parametric distribution assumed for effects, but after fixing the model parameters to reasonable values, the distribution of the effects influences clustering very little. Simulations confirms that the clustering algorithm and the proposed variable selection method is reliable when the model assumptions are wrong. The new approach is compared with the popular Bayesian clustering alternative, MCLUST, fitted on the principal components using two loss functions in which our proposed approach is found to be more efficient in almost every situation

Electrocatalysis at liquid/liquid interfaces

This thesis is devoted to the study of oxygen reduction reaction catalysed by porphyrins at the interface between two immiscible electrolyte solutions (ITIES). Electrochemical and spectrophotometric techniques are introduced to these interfaces in order to gather more information about the transfer mechanism. Furthermore, the reduction of oxygen and the oxidation of decamethylferrocene (DMFc) in 1,2-DCE and production of hydrogen peroxide (H2O2) in the aqueous phase, on the basis of the two-phase reaction controlled by a common ion are investigated. Mass spectrometric measurements were carried out for the 1,2-DCE phase before and after two-phase reaction with an aqueous phase containing acid to indicate the stability of DMFc and the DMFc+ over the course of the two-phase reaction. Density function theory (DFT) computations have been performed based on developed a reaction pathway. Catalytic effect of 5,10,15,20-tetraphenylporphyrinatocobalt(II) [Co(tpp)], 2,3,7,8,12,13,17,18-Octaethyl-porphyrin cobalt(II) (CoOEP) and two free-base porphyrins 5,10,15,20-meso-tetraphenylporphyrin (H2TPP) and 2,3,7,8,12,13,17,18-octaethyl-21H,23H-porphyrin (H2OEP) have been investigated as a catalyst for a two electron reduction of O2 in presence of an electron donor at various pH values at the polarized water|1,2-DCE interface. Using voltammetry, it is possible to drive this catalytic reduction at the interface as a function of the applied potential difference, where aqueous protons and organic electron donors combine to reduce O2. The signal observed corresponds to a proton-coupled electron transfer (PCET) reaction, as no current and no reaction can be observed in the absence of either catalyst, acid or O2. [Co(tpp)] and CoOEP catalysis work like conventional cobalt porphyrins, activating O2 via coordination by the formation of a superoxide structure. The advantages of the present system is that, by controlling the interfacial potential difference, the proton transfer from water to 1,2-DCE can be accurately controlled. Accordingly, the driving force for proton-coupled electron transfer reactions is also effectively harnessed. Assisted proton transfer (APT) reactions were studied across the water|1,2-DCE interface facilitated by two free-base porphyrins such as H2TPP and H2OEP. At a water|1,2-DCE interface, the interfacial formation of di-acid H4TPP2+ and H4OEP2+ are observed by ion-transfer voltammetry and UV-Visible spectroscopy, due to the double protonation of H2TPP and H2OEP at the tertiary nitrogens in the ring. Additionally, "Ionic Partition Diagram" of neutral and ionisable H2TPP compounds is plotted to illustrate the various contributions of H2TPP