Distributed Training Large-Scale Deep Architectures

Scale of data and scale of computation infrastructures together enable the current deep learning renaissance. However, training large-scale deep architectures demands both algorithmic improvement and careful system configuration. In this paper, we focus on employing the system approach to speed up large-scale training. Via lessons learned from our routine benchmarking effort, we first identify bottlenecks and overheads that hinter data parallelism. We then devise guidelines that help practitioners to configure an effective system and fine-tune parameters to achieve desired speedup. Specifically, we develop a procedure for setting minibatch size and choosing computation algorithms. We also derive lemmas for determining the quantity of key components such as the number of GPUs and parameter servers. Experiments and examples show that these guidelines help effectively speed up large-scale deep learning training

Solving Quadratic Equations with XL on Parallel Architectures - extended version

Solving a system of multivariate quadratic equations (MQ) is an NP-complete problem whose complexity estimates are relevant to many cryptographic scenarios. In some cases it is required in the best known attack; sometimes it is a generic attack (such as for the multivariate PKCs), and sometimes it determines a provable level of security (such as for the QUAD stream ciphers). Under reasonable assumptions, the best way to solve generic MQ systems is the XL algorithm implemented with a sparse matrix solver such as Wiedemann\u27s algorithm. Knowing how much time an implementation of this attack requires gives us a good idea of how future cryptosystems related to MQ can be broken, similar to how implementations of the General Number Field Sieve that factors smaller RSA numbers give us more insight into the security of actual RSA-based cryptosystems. This paper describes such an implementation of XL using the block Wiedemann algorithm. In 5 days we are able to solve a system with 32 variables and 64 equations over $\mathbb{F}_{16}$ (a computation of about $2^{60.3}$ bit operations) on a small cluster of 8 nodes, with 8 CPU cores and 36 GB of RAM in each node. We do not expect system solvers of the F$_4$/F$_5$ family to accomplish this due to their much higher memory demand. Our software also offers implementations for $\mathbb{F}_{2}$ and $\mathbb{F}_{31}$ and can be easily adapted to other small fields. More importantly, it scales nicely for small clusters, NUMA machines, and a combination of both

Temperature Swing Adsorption Process for CO2 Capture Using Polyaniline Solid Sorbent

AbstractTo capture carbon dioxide from power plant flue gas which consists of 15% CO2 and 85% N2, with a temperature swing adsorption (TSA) by using polyaniline solid sorbent as the adsorbent, is explored experimentally and theoretically. First, single component adsorption equilibrium data of carbon dioxide on polyaniline solid sorbent is obtained by using Micro-Balance Thermo D-200. Then isotherm curves and the parameters are obtained by numerical method. The adsorption is expressed by the Langmuir-Freundlich isotherm. After accomplishment of isotherm curves, the breakthrough curve experiment is investigated with single adsorption column. The experiments test the change in adsorbed gas concentration at the outlet by adsorbed gas, CO2, and non-adsorbed gas, helium. Finally, this study accentuates the TSA experiments on CO2 purity and recovery by operation variable discussion which includes feed pressure, adsorption temperature and desorption temperature to find optimal operation condition. The results of optimal operation condition are CO2 purity of 47.65% with a 92.46% recovery

Distribution and associated factors of optic disc diameter and cup-to-disc ratio in an elderly Chinese population

AbstractBackgroundGlaucoma is the second leading cause of blindness worldwide and East Asian people account for almost half of those affected. Vertical elongation of the optic cup is a characteristic feature of glaucoma. However, there is a significant overlap in the vertical cup-to-disc ratio (VCDR) between normal eyes and eyes affected by glaucoma. The purpose of this study was to determine the distribution of VCDR and vertical disc diameter (VDD) and their predictive factors in a population of elderly Chinese residents in Taiwan.MethodsFour hundred and sixty elderly Chinese residents aged 72 years and older in the Shihpai district, Taipei, Taiwan participated in this study. Slit lamp biomicroscopic measurement of the VCDR and VDD after pupil dilation with a 78 diopter lens was performed by one glaucoma specialist. Multiple linear regression analyses were used to fit the best model for independent variables.ResultsThe VCDR was recorded for 438 right eyes and 430 left eyes. After excluding participants with glaucoma, the mean ± SD VCDR was 0.44 ± 0.17 for both eyes, and the 97.5th percentile was 0.8. A greater VCDR was associated with a longer axial length [VCDR = −0.47 + 0.04(axial length)] under multiple regression analysis. The VDD was obtained for 420 right eyes and 406 left eyes. The mean ± SD VDD for all participants was 1.77 ± 0.22 mm for the right eye and 1.79 ± 0.22 mm for the left eye. A higher body mass index (BMI) and a longer axial length were significantly associated with a larger VDD under multiple regression analysis. [VDD = −0.05 + 0.07 (axial length) + 0.06 (obesity); if BMI <24, then obesity = 0; if BMI ≥24, then obesity = 1]. A larger VDD was associated with a larger VCDR (p < 0.001) and the VCDR could be predicted by the equation VCDR = −0.07 + 0.3VDD.ConclusionA greater VCDR was related to a longer axial length. A greater VDD was related to a higher BMI and a longer axial length

Granzyme G is expressed in the two-cell stage mouse embryo and is required for the maternal-zygotic transition

<p>Abstract</p> <p>Background</p> <p>Detailed knowledge of the molecular and cellular mechanisms that direct spatial and temporal gene expression in pre-implantation embryos is critical for understanding the control of the maternal-zygotic transition and cell differentiation in early embryonic development. In this study, twenty-three clones, expressed at different stages of early mouse development, were identified using differential display reverse transcription polymerase chain reaction (DDRT-PCR). One of these clones, which is expressed in 2-cell stage embryos at 48 hr post-hCG injection, shows a perfect sequence homology to the gene encoding the granzyme G protein. The granzyme family members are serine proteases that are present in the secretory granules of cytolytic T lymphocytes. However, the pattern of granzyme G expression and its function in early mouse embryos are entirely unknown.</p> <p>Results</p> <p>Upon the introduction of an antisense morpholino (2 mM) against granzyme G to knock-down endogenous gene function, all embryos were arrested at the 2- to 4-cell stages of egg cleavage, and the <it>de novo </it>synthesis of zygotic RNAs was decreased. The embryonic survival rate was dramatically decreased at the late 2-cell stage when serine protease-specific inhibitors, 0.1 mM 3,4-dichloroisocoumarin (3,4-DCI), and 2 mM phenyl methanesulphonyl fluoride (PMSF), were added to the <it>in vitro </it>embryonic culture medium. Survival was not affected by the addition of 0.5 mM EDTA, a metalloproteinase inhibitor.</p> <p>Conclusion</p> <p>We characterized for the first time the expression and function of <it>granzyme G </it>during early stage embryogenesis. Our data suggest that granzyme G is an important factor in early mouse embryonic development and may play a novel role in the elimination of maternal proteins and the triggering of zygotic gene expression during the maternal-zygotic transition.</p

Mismatched dNTP incorporation by DNA polymerase β does not proceed via globally different conformational pathways†

Understanding how DNA polymerases control fidelity requires elucidation of the mechanisms of matched and mismatched dNTP incorporations. Little is known about the latter because mismatched complexes do not crystallize readily. In this report, we employed small-angle X-ray scattering (SAXS) and structural modeling to probe the conformations of different intermediate states of mammalian DNA polymerase β (Pol β) in its wild-type and an error-prone variant, I260Q. Our structural results indicate that the mismatched ternary complex lies in-between the open and the closed forms, but more closely resembles the open form for WT and the closed form for I260Q. On the basis of molecular modeling, this over-stabilization of mismatched ternary complex of I260Q is likely caused by formation of a hydrogen bonding network between the side chains of Gln260, Tyr296, Glu295 and Arg258, freeing up Asp192 to coordinate MgdNTP. These results argue against recent reports suggesting that mismatched dNTP incorporations follow a conformational path distinctly different from that of matched dNTP incorporation, or that its conformational closing is a major contributor to fidelity