A study on hardware design for high performance artificial neural network by using FPGA and NoC

制度:新 ; 報告番号:甲3421号 ; 学位の種類:博士(工学) ; 授与年月日:2011/9/15 ; 早大学位記番号:新574

You Only Propagate Once: Accelerating Adversarial Training via Maximal Principle

Deep learning achieves state-of-the-art results in many tasks in computer vision and natural language processing. However, recent works have shown that deep networks can be vulnerable to adversarial perturbations, which raised a serious robustness issue of deep networks. Adversarial training, typically formulated as a robust optimization problem, is an effective way of improving the robustness of deep networks. A major drawback of existing adversarial training algorithms is the computational overhead of the generation of adversarial examples, typically far greater than that of the network training. This leads to the unbearable overall computational cost of adversarial training. In this paper, we show that adversarial training can be cast as a discrete time differential game. Through analyzing the Pontryagin's Maximal Principle (PMP) of the problem, we observe that the adversary update is only coupled with the parameters of the first layer of the network. This inspires us to restrict most of the forward and back propagation within the first layer of the network during adversary updates. This effectively reduces the total number of full forward and backward propagation to only one for each group of adversary updates. Therefore, we refer to this algorithm YOPO (You Only Propagate Once). Numerical experiments demonstrate that YOPO can achieve comparable defense accuracy with approximately 1/5 ~ 1/4 GPU time of the projected gradient descent (PGD) algorithm. Our codes are available at https://https://github.com/a1600012888/YOPO-You-Only-Propagate-Once.Comment: Accepted as a conference paper at NeurIPS 201

Hensen's Node from Vitamin A-Deficient Quail Embryo Induces Chick Limb Bud Duplication and Retains Its Normal Asymmetric Expression ofSonic hedgehog(Shh)

AbstractBoth Hensen's node, the organizer center in chick embryo, and exogenous retinoic acid are known to induce limb duplication when grafted or applied to the host chick limb bud. Retinoic acid is known to be present in the node and has been proposed as the putative morphogen for chick limb development. Here, we report that Hensen's node from vitamin A-deficient quail embryo induces limb duplication in the host chick embryo similar to that induced by the node from vitamin A-sufficient control embryos. We also demonstrate that the expression ofSonic hedgehog(Shh), recently shown to be the mediator of polarizing activity in the chick limb bud, is not affected by the endogenous vitamin A status of the embryo. Furthermore, whole-mountin situhybridization revealed asymmetry ofShhexpression in the Hensen's node of both vitamin A-sufficient and -deficient quail embryos. Retinoids were not detectable in the eggs from which vitamin A-deficient embryos were obtained. Extracts from normal embryos induced a level of expression of reporter gene equivalent to the presence of 3.4 pg of active retinoids per embryo, while those from vitamin A-deficient embryos induced a baseline level of reporter gene expression similar to that of the controls. Our studies suggest that endogenous retinoic acid is not involved inShhexpression nor in regulating its asymmetry during normal early avian embryogenesis and support the current view that endogenous retinoic acid may not be a direct morphogen for limb bud duplication