Adaptive Normalized Risk-Averting Training For Deep Neural Networks

This paper proposes a set of new error criteria and learning approaches, Adaptive Normalized Risk-Averting Training (ANRAT), to attack the non-convex optimization problem in training deep neural networks (DNNs). Theoretically, we demonstrate its effectiveness on global and local convexity lower-bounded by the standard $L_p$-norm error. By analyzing the gradient on the convexity index $\lambda$, we explain the reason why to learn $\lambda$ adaptively using gradient descent works. In practice, we show how this method improves training of deep neural networks to solve visual recognition tasks on the MNIST and CIFAR-10 datasets. Without using pretraining or other tricks, we obtain results comparable or superior to those reported in recent literature on the same tasks using standard ConvNets + MSE/cross entropy. Performance on deep/shallow multilayer perceptrons and Denoised Auto-encoders is also explored. ANRAT can be combined with other quasi-Newton training methods, innovative network variants, regularization techniques and other specific tricks in DNNs. Other than unsupervised pretraining, it provides a new perspective to address the non-convex optimization problem in DNNs.Comment: AAAI 2016, 0.39%~0.4% ER on MNIST with single 32-32-256-10 ConvNets, code available at https://github.com/cauchyturing/ANRA

Adopting Robustness and Optimality in Fitting and Learning

We generalized a modified exponentialized estimator by pushing the robust-optimal (RO) index $\lambda$ to $-\infty$ for achieving robustness to outliers by optimizing a quasi-Minimin function. The robustness is realized and controlled adaptively by the RO index without any predefined threshold. Optimality is guaranteed by expansion of the convexity region in the Hessian matrix to largely avoid local optima. Detailed quantitative analysis on both robustness and optimality are provided. The results of proposed experiments on fitting tasks for three noisy non-convex functions and the digits recognition task on the MNIST dataset consolidate the conclusions.Comment: arXiv admin note: text overlap with arXiv:1506.0269

Rare variants contribute disproportionately to quantitative trait variation in yeast.

How variants with different frequencies contribute to trait variation is a central question in genetics. We use a unique model system to disentangle the contributions of common and rare variants to quantitative traits. We generated ~14,000 progeny from crosses among 16 diverse yeast strains and identified thousands of quantitative trait loci (QTLs) for 38 traits. We combined our results with sequencing data for 1011 yeast isolates to show that rare variants make a disproportionate contribution to trait variation. Evolutionary analyses revealed that this contribution is driven by rare variants that arose recently, and that negative selection has shaped the relationship between variant frequency and effect size. We leveraged the structure of the crosses to resolve hundreds of QTLs to single genes. These results refine our understanding of trait variation at the population level and suggest that studies of rare variants are a fertile ground for discovery of genetic effects

An assessment of information flow as an enabler to collaboration in the supply chain within a South African context

Organisations need to know more in order to do more for customers. As demand for customer information grows, so to, do the archives in the back office and the servers at the central hub. It is estimated that the amount of data now captured and stored nearly doubles every 12-18 months.(Information Week) The logistics industry is plagued by the very element of changing customer demands, customization thus resulting in the survival of the fittest. The informed customer demands an integrated product offering customised to their needs. This industry has evolved to one where companies need deep pockets to ensure an IT platform capable of meeting the increasing demands of the modern supply chain. The objective of the report is to gain further insight and understanding of how stakeholders within South Africa assess the flow of information as an enabler to greater collaboration within the supply chain. Information flow is one of the many elements that contribute toward greater collaboration, which is a recent trend in supply chain management that focuses on joint planning, coordination and process integration between stakeholders in the supply chain (Spekman et al 1998). Globalisation and the advent of e-commerce Business 2 Business transactions and the Lean production philosophies that are being adopted by more and more industries is demanding real time, data exchange and information flow in order to make the necessary and timely decisions which are required to meet ever changing customer demands. Information is only one of the areas in which tremendous benefit can be derived, this paper considers a thorough literature review of aspects surrounding information flow from a global perspective and assesses the feedback of South African organisations in relation to this theme with the view to providing readers with greater insight to possible opportunities that may exist for improvement in their respective supply chain.Dissertation (MBA)--University of Pretoria, 2010.Gordon Institute of Business Science (GIBS)unrestricte

Coupling Hydrodynamic and Biokinetic Growth Models in Aerated Wastewater Treatment Processes

In this thesis, a coupled hydrodynamic and wastewater biokinetic finite volume based CFD model for an aeration tank in OpenFOAM has been created to understand the effect of the hydrodynamics on the biological processes. A pilot-scale aeration tank that is aerated using fine membrane diffusers along the base has been designed and manufactured. A procedure for conducting lab experiments using an acoustic Doppler velocimeter to record velocity measurements was outlined. A series of aeration tank experiments with flow rates ranging from 18 – 108 L/min through membrane diffuser setups that involved 1 or 3 diffusers were conducted in which ADV velocity measurements were taken and have been used to validate a CFD model. Additionally, it was found that certain diffuser configurations showed pseudo - 2D behaviour such that the recorded data could be used to validate 2D simulations of the aeration tank. A CFD model using the Eulerian-Eulerian multiphase formulation in OpenFOAM was created to replicate the bubble driven fluid flow and free surface effects in the pilot-scale aeration tank. The influence of the inlet conditions, bubble diameter size and bubble dynamic models on the generated results were investigated and compared with the experimental data to validate the modelling choices. As a result, a 2D and 3D CFD model of the aeration tank was defined and validated against the experimental ADV data. Using the results, a procedure for coupling the biokinetics into the hydrodynamics was described in OpenFOAM. The difficulties that arose from transferring a two-phase solution with a free surface to a single-phase solver was outlined and solutions to the issues were defined and assessed. The mass transfer of oxygen into the fluid was modelled and compared with experimental results from the membrane diffuser manufactures to confirm the accuracy of the model. The oxygen mass transfer model was used to assess how the membrane diffuser setup and flow rate impacts the oxygenation of the reactor. It was found that increasing the number of aerating diffusers while keeping the total air flow rate the same significantly increased the oxygenation of the tank in comparison to just increasing the air flow rate which was found to only slightly increase the oxygenation. Additionally, a curve fitting procedure was described to derive a global oxygen transfer rate coefficient and saturation value from the CFD simulation for specific aeration tank setups and assessment of the values found they could give insight to the hydrodynamic behaviour in the reactor. The simulations were further extended to include the biokinetics to describe the biological interactions. A simple biokinetic aeration model was proposed to assess the impact of the hydrodynamics, inlet and outlet locations, and flow rate on the biological processes in tank. It was found that inadequate mixing in the 2D simulation resulted in twice the required amount of time to reach the maximum biomass concentrations compared with the equivalent perfectly mixed reactor. It was shown that the location of the inlet and outlet with the same hydrodynamic flow fields could influence the biological processes. It was found that there was no difference in the biological performance of the 3D reactor with an aerating flow rate of 0.3 and 0.6 L/s such that it would be inefficient to aerate the tank at 0.6 L/s. Finally, the full ASM1 was implemented into the coupled model and compared with the conventional ASM1 model to assess the performance of the aeration tank at producing and removing nitrates and ammonium. It was found that inadequate mixing resulted in reduced efficiency of the reactor at producing and removing nitrates and ammonium, respectively, which would further impact the performance of the sequential rectors

Manganese-catalyzed hydrogenation of amides and polyurethanes : is catalyst inhibition an additional barrier to efficient hydrogenation of amides and their derivatives?

J.L. and A.K. thank the UKRI Future Leaders Fellowship (MR/W007460/1). The authors would like to thank the University of St. Andrews and the EPSRC Centre for Doctoral Training in Critical Resource Catalysis (CRITICAT) for financial support [Ph.D. studentship to C.L.O.; Grant Code: EP/L016419/1].The hydrogenation of amides and other less electrophilic carbonyl derivatives with an N–C═O functionality requires significant improvements in scope and catalytic activity to be a genuinely useful reaction in industry. Here, we report the results of a study that examined whether such reactions are further disadvantaged by nitrogen-containing compounds such as aliphatic amines acting as inhibitors on the catalysts. In this case, an enantiomerically pure manganese catalyst previously established to be efficient in the hydrogenation of ketones, N-aryl-imines, and esters was used as a prototype of a manganese catalyst. This was accomplished by doping a model ester hydrogenation with various nitrogen-containing compounds and monitoring progress. Following from this, a protocol for the catalytic hydrogenation of amides and polyurethanes is described, including the catalytic hydrogenation of an axially chiral amide that resulted in low levels of kinetic resolution. The hypothesis of nitrogen-containing compounds acting as an inhibitor in the catalytic hydrogenation process has also been rationalized by using spectroscopy (high-pressure infrared (IR), nuclear magnetic resonance (NMR)) and mass spectrometry studies.Publisher PDFPeer reviewe