Defensive Dropout for Hardening Deep Neural Networks under Adversarial Attacks

Deep neural networks (DNNs) are known vulnerable to adversarial attacks. That is, adversarial examples, obtained by adding delicately crafted distortions onto original legal inputs, can mislead a DNN to classify them as any target labels. This work provides a solution to hardening DNNs under adversarial attacks through defensive dropout. Besides using dropout during training for the best test accuracy, we propose to use dropout also at test time to achieve strong defense effects. We consider the problem of building robust DNNs as an attacker-defender two-player game, where the attacker and the defender know each others' strategies and try to optimize their own strategies towards an equilibrium. Based on the observations of the effect of test dropout rate on test accuracy and attack success rate, we propose a defensive dropout algorithm to determine an optimal test dropout rate given the neural network model and the attacker's strategy for generating adversarial examples.We also investigate the mechanism behind the outstanding defense effects achieved by the proposed defensive dropout. Comparing with stochastic activation pruning (SAP), another defense method through introducing randomness into the DNN model, we find that our defensive dropout achieves much larger variances of the gradients, which is the key for the improved defense effects (much lower attack success rate). For example, our defensive dropout can reduce the attack success rate from 100% to 13.89% under the currently strongest attack i.e., C&W attack on MNIST dataset.Comment: Accepted as conference paper on ICCAD 201

On the Universal Approximation Property and Equivalence of Stochastic Computing-based Neural Networks and Binary Neural Networks

Large-scale deep neural networks are both memory intensive and computation-intensive, thereby posing stringent requirements on the computing platforms. Hardware accelerations of deep neural networks have been extensively investigated in both industry and academia. Specific forms of binary neural networks (BNNs) and stochastic computing based neural networks (SCNNs) are particularly appealing to hardware implementations since they can be implemented almost entirely with binary operations. Despite the obvious advantages in hardware implementation, these approximate computing techniques are questioned by researchers in terms of accuracy and universal applicability. Also it is important to understand the relative pros and cons of SCNNs and BNNs in theory and in actual hardware implementations. In order to address these concerns, in this paper we prove that the "ideal" SCNNs and BNNs satisfy the universal approximation property with probability 1 (due to the stochastic behavior). The proof is conducted by first proving the property for SCNNs from the strong law of large numbers, and then using SCNNs as a "bridge" to prove for BNNs. Based on the universal approximation property, we further prove that SCNNs and BNNs exhibit the same energy complexity. In other words, they have the same asymptotic energy consumption with the growing of network size. We also provide a detailed analysis of the pros and cons of SCNNs and BNNs for hardware implementations and conclude that SCNNs are more suitable for hardware.Comment: 9 pages, 3 figure

Towards Query-Efficient Black-Box Adversary with Zeroth-Order Natural Gradient Descent

Despite the great achievements of the modern deep neural networks (DNNs), the vulnerability/robustness of state-of-the-art DNNs raises security concerns in many application domains requiring high reliability. Various adversarial attacks are proposed to sabotage the learning performance of DNN models. Among those, the black-box adversarial attack methods have received special attentions owing to their practicality and simplicity. Black-box attacks usually prefer less queries in order to maintain stealthy and low costs. However, most of the current black-box attack methods adopt the first-order gradient descent method, which may come with certain deficiencies such as relatively slow convergence and high sensitivity to hyper-parameter settings. In this paper, we propose a zeroth-order natural gradient descent (ZO-NGD) method to design the adversarial attacks, which incorporates the zeroth-order gradient estimation technique catering to the black-box attack scenario and the second-order natural gradient descent to achieve higher query efficiency. The empirical evaluations on image classification datasets demonstrate that ZO-NGD can obtain significantly lower model query complexities compared with state-of-the-art attack methods.Comment: accepted by AAAI 202

Variable pitch approach for performance improving of straight-bladed VAWT at rated tip speed ratio

This paper presents a new variable pitch (VP) approach to increase the peak power coefficient of the straight-bladed vertical-axis wind turbine (VAWT), by widening the azimuthal angle band of the blade with the highest aerodynamic torque, instead of increasing the highest torque. The new VP-approach provides a curve of pitch angle designed for the blade operating at the rated tip speed ratio (TSR) corresponding to the peak power coefficient of the fixed pitch (FP)-VAWT. The effects of the new approach are exploited by using the double multiple stream tubes (DMST) model and Prandtl’s mathematics to evaluate the blade tip loss. The research describes the effects from six aspects, including the lift, drag, angle of attack (AoA), resultant velocity, torque, and power output, through a comparison between VP-VAWTs and FP-VAWTs working at four TSRs: 4, 4.5, 5, and 5.5. Compared with the FP-blade, the VP-blade has a wider azimuthal zone with the maximum AoA, lift, drag, and torque in the upwind half-cycle, and yields the two new larger maximum values in the downwind half-cycle. The power distribution in the swept area of the turbine changes from an arched shape of the FP-VAWT into the rectangular shape of the VP-VAWT. The new VP-approach markedly widens the highest-performance zone of the blade in a revolution, and ultimately achieves an 18.9% growth of the peak power coefficient of the VAWT at the optimum TSR. Besides achieving this growth, the new pitching method will enhance the performance at TSRs that are higher than current optimal values, and an increase of torque is also generated

Incentivizing emerging market suppliers for responsible international supply chain: Revenue-sharing and government subsidy

Many emerging market suppliers of multinational enterprises (MNEs) have been exposed to social responsibility controversies. These incidents significantly affect MNEs’ operations and emerging economies’ sustainable development. This paper considers a two-tier transnational supply chain model to explore the impact of different participants’ corporate social responsibility (CSR) engagements on their profits and social welfare. We consider two incentive schemes that could enhance emerging market suppliers’ CSR activities: revenue sharing from their buyers and subsidies from their governments. Using the supplier Stackelberg game, we find: 1) transnational operation costs hinder MNEs’ incentive to invest in CSR; 2) suppliers’ CSR activities have a larger impact on the demand for final products and emerging market welfare than MNEs’ activities; 3) suppliers will voluntarily engage in CSR activities, but only at an insufficient level, whereas MNEs revenue-sharing with suppliers and government subsidies to suppliers can improve suppliers’ CSR level; 4) government subsidy improves suppliers’ CSR activities to a larger extent than MNEs’ revenue-sharing. Our study fills the gap in CSR activities along the international supply chain. We also provide critical managerial implications to MNEs and their emerging market suppliers on reducing CSR risk, and policy implications to emerging market governments on realizing sustainable development

Seroprevalence Survey of Avian influenza A (H5) in wild migratory birds in Yunnan Province, Southwestern China

BACKGROUND: Highly pathogenic avian influenza virus (HPAIV) is a highly contagious disease which is a zoonotic pathogen of significant economic and public health concern. The outbreaks caused by HPAIV H5N1 of Asian origin have caused animal and human disease and mortality in several countries of Southeast Asia, such as Bangladesh, Cambodia, China, India, Indonesia, Laos, Myanmar, Thailand and Viet Nam. For the first time since 1961, this HPAIV has also caused extensive mortality in wild birds and has sparked debate of the role wild birds have played in the spread of this virus. Other than confirmed mortality events, little is known of this virus in wild birds. There is no report on the seroprevalence of avian influenza H5 infection in wild migratory birds in Yunnan Province. In this study we examined live wild birds in Yunnan Province for H5 specific antibody to better understand the occurrence of this disease in free living birds. METHODS: Sera from 440 wild birds were collected from in Kunming and Northern Ailaoshan of Yunnan Province, Southwestern China, and assayed for H5 antibodies using the hemagglutination inhibition (HI) assays. RESULTS: The investigation revealed that the seroprevalence of avian influenza H5 was as following: Ciconiiformes 2.6%, Strigiformes 13.04%, Passeriformes 20%, Cuculiformes 21.74%, Gruiformes 0%, Columbiformes 0%, Charadriiformes 0% and Coraciiformes 0%. Statistical analyses showed that there was a significant difference of prevalence between the orders (P < 0.01). Specific avian influenza H5 antibodies were detected in 23 of 440 (5.23%) sera. Mean HI titer 23 positive sera against H5 were 5.4 log(2). CONCLUSIONS: The results of the present survey indicated that the proportion of wild birds had previously infected AIV H5 at other times of the year. To our knowledge, this is the first seroprevalence report of avian influenza H5 infection in wild migratory birds in China’ s southwestern Yunnan Province. The results of the present survey have significant public health concerns

Complete genome sequencing of Pseudomonas syringae pv. actinidiae Biovar 3, P155, kiwifruit pathogen originating from China

Pseudomonas syringae pv. actinidiae is a bacterial pathogen of kiwifruit. Based on the results of the pathogenicity assay, we sequenced the strain Pseudomonas syringae (Psa3) P155 which possesses a series of virulence and resistance genes, CRISPR candidate elements, prophage related sequences, methylation modifications, genomic islands as well as one plasmid. Most importantly, the copper resistance genes copA, copB, copC, copD, and copZ as well as aminoglycoside resistance gene ksgA were identified in strain P155, which would pose a threat to kiwifruit production. The complete sequence we reported here will provide valuable information for a better understanding of the genetic structure and pathogenic characteristics of the genome of P155

STUDY ON THE SETTLEMENT OF TUNNEL BOTTOM AND PRESSURE OF ROCK MASS BASED ON CURVED BEAM ON ELASTIC FOUNDATION THEORY

Tunnel invert is a weak section of curved beam on tunnel foundation, and it is easy to break down. Based on the curved beam theory on elastic foundation, the curved beam model of tunnel invert was established, the displacement equation of tunnel invert under external load was deduced, and the formula of settlement of tunnel bottom and the pressure of rock mass was presented. By means of the calculating formula, the distribution law of settlement of tunnel bottom and pressure of rock mass were obtained when tunnel bottom was strengthened and not strengthened by high pressure jet grouting pile. The final formula in the paper is precise to predict the settlement of tunnel bottom and pressure of rock mass, so it is of great value for tunnel design and construction

Electrical stimulation therapy for peripheral nerve injury

Peripheral nerve injury is common and frequently occurs in extremity trauma patients. The motor and sensory impairment caused by the injury will affect patients' daily life and social work. Surgical therapeutic approaches don't assure functional recovery, which may lead to neuronal atrophy and hinder accelerated regeneration. Rehabilitation is a necessary stage for patients to recover better. A meaningful role in non-pharmacological intervention is played by rehabilitation, through individualized electrical stimulation therapy. Clinical studies have shown that electrical stimulation enhances axon growth during nerve repair and accelerates sensorimotor recovery. According to different effects and parameters, electrical stimulation can be divided into neuromuscular, transcutaneous, and functional electrical stimulation. The therapeutic mechanism of electrical stimulation may be to reduce muscle atrophy and promote muscle reinnervation by increasing the expression of structural protective proteins and neurotrophic factors. Meanwhile, it can modulate sensory feedback and reduce neuralgia by inhibiting the descending pathway. However, there are not many summary clinical application parameters of electrical stimulation, and the long-term effectiveness and safety also need to be further explored. This article aims to explore application methodologies for effective electrical stimulation in the rehabilitation of peripheral nerve injury, with simultaneous consideration for fundamental principles of electrical stimulation and the latest technology. The highlight of this paper is to identify the most appropriate stimulation parameters (frequency, intensity, duration) to achieve efficacious electrical stimulation in the rehabilitation of peripheral nerve injury