Efficient Neural Network Robustness Certification with General Activation Functions

Finding minimum distortion of adversarial examples and thus certifying robustness in neural network classifiers for given data points is known to be a challenging problem. Nevertheless, recently it has been shown to be possible to give a non-trivial certified lower bound of minimum adversarial distortion, and some recent progress has been made towards this direction by exploiting the piece-wise linear nature of ReLU activations. However, a generic robustness certification for general activation functions still remains largely unexplored. To address this issue, in this paper we introduce CROWN, a general framework to certify robustness of neural networks with general activation functions for given input data points. The novelty in our algorithm consists of bounding a given activation function with linear and quadratic functions, hence allowing it to tackle general activation functions including but not limited to four popular choices: ReLU, tanh, sigmoid and arctan. In addition, we facilitate the search for a tighter certified lower bound by adaptively selecting appropriate surrogates for each neuron activation. Experimental results show that CROWN on ReLU networks can notably improve the certified lower bounds compared to the current state-of-the-art algorithm Fast-Lin, while having comparable computational efficiency. Furthermore, CROWN also demonstrates its effectiveness and flexibility on networks with general activation functions, including tanh, sigmoid and arctan.Comment: Accepted by NIPS 2018. Huan Zhang and Tsui-Wei Weng contributed equall

Utility of Washington Early Recognition Center self-report screening questionnaires in the assessment of patients with schizophrenia and bipolar disorder

Early identification and treatment are associated with improved outcomes in bipolar disorder (BPD) and schizophrenia (SCZ). Screening for the presence of these disorders usually involves time-intensive interviews that may not be practical in settings where mental health providers are limited. Thus, individuals at earlier stages of illness are often not identified. The Washington Early Recognition Center Affectivity and Psychosis (WERCAP) screen is a self-report questionnaire originally developed to identify clinical risk for developing bipolar or psychotic disorders. The goal of the current study was to investigate the utility of the WERCAP Screen and two complementary questionnaires, the WERC Stress Screen and the WERC Substance Screen, in identifying individuals with established SCZ or BPD. Participants consisted of 35 BPD and 34 SCZ patients, as well as 32 controls (CON), aged 18–30 years. Univariate analyses were used to test for score differences between groups. Logistic regression and receiver operating characteristic (ROC) curves were used to identify diagnostic predictors. Significant group differences were found for the psychosis section of the WERCAP (pWERCAP; p < 0.001), affective section of the WERCAP (aWERCAP; p = 0.001), and stress severity (p = 0.027). No significant group differences were found in the rates of substance use as measured by the WERC Substance Screen (p = 0.267). Only the aWERCAP and pWERCAP scores were useful predictors of diagnostic category. ROC curve analysis showed the optimal cut point on the aWERCAP to identify BPD among our participant groups was a score of >20 [area under the curve (AUC): 0.87; sensitivity: 0.91; specificity: 0.71], while that for the pWERCAP to identify SCZ was a score of >13 (AUC: 0.89; sensitivity: 0.88; specificity: 0.82). These results indicate that the WERCAP Screen may be useful in screening individuals for BPD and SCZ and that identifying stress and substance-use severity can be rapidly done using self-report questionnaires. Larger studies in undiagnosed individuals will be needed to test the WERCAP Screen’s ability to identify mania or psychosis in the community

Towards Fast Computation of Certified Robustness for ReLU Networks

Verifying the robustness property of a general Rectified Linear Unit (ReLU) network is an NP-complete problem [Katz, Barrett, Dill, Julian and Kochenderfer CAV17]. Although finding the exact minimum adversarial distortion is hard, giving a certified lower bound of the minimum distortion is possible. Current available methods of computing such a bound are either time-consuming or delivering low quality bounds that are too loose to be useful. In this paper, we exploit the special structure of ReLU networks and provide two computationally efficient algorithms Fast-Lin and Fast-Lip that are able to certify non-trivial lower bounds of minimum distortions, by bounding the ReLU units with appropriate linear functions Fast-Lin, or by bounding the local Lipschitz constant Fast-Lip. Experiments show that (1) our proposed methods deliver bounds close to (the gap is 2-3X) exact minimum distortion found by Reluplex in small MNIST networks while our algorithms are more than 10,000 times faster; (2) our methods deliver similar quality of bounds (the gap is within 35% and usually around 10%; sometimes our bounds are even better) for larger networks compared to the methods based on solving linear programming problems but our algorithms are 33-14,000 times faster; (3) our method is capable of solving large MNIST and CIFAR networks up to 7 layers with more than 10,000 neurons within tens of seconds on a single CPU core. In addition, we show that, in fact, there is no polynomial time algorithm that can approximately find the minimum $\ell_1$ adversarial distortion of a ReLU network with a $0.99\ln n$ approximation ratio unless $\mathsf{NP}$=$\mathsf{P}$, where $n$ is the number of neurons in the network.Comment: Tsui-Wei Weng and Huan Zhang contributed equall

Age-related changes to macrophages are detrimental to fracture healing in mice.

The elderly population suffers from higher rates of complications during fracture healing that result in increased morbidity and mortality. Inflammatory dysregulation is associated with increased age and is a contributing factor to the myriad of age-related diseases. Therefore, we investigated age-related changes to an important cellular regulator of inflammation, the macrophage, and the impact on fracture healing outcomes. We demonstrated that old mice (24 months) have delayed fracture healing with significantly less bone and more cartilage compared to young mice (3 months). The quantity of infiltrating macrophages into the fracture callus was similar in old and young mice. However, RNA-seq analysis demonstrated distinct differences in the transcriptomes of macrophages derived from the fracture callus of old and young mice, with an up-regulation of M1/pro-inflammatory genes in macrophages from old mice as well as dysregulation of other immune-related genes. Preventing infiltration of the fracture site by macrophages in old mice improved healing outcomes, with significantly more bone in the calluses of treated mice compared to age-matched controls. After preventing infiltration by macrophages, the macrophages remaining within the fracture callus were collected and examined via RNA-seq analysis, and their transcriptome resembled macrophages from young calluses. Taken together, infiltrating macrophages from old mice demonstrate detrimental age-related changes, and depleting infiltrating macrophages can improve fracture healing in old mice

Survival Analysis Using Auxiliary Variables Via Nonparametric Multiple Imputation

We develop an approach, based on multiple imputation, that estimates the marginal survival distribution in survival analysis using auxiliary variable to recover information for censored observations. To conduct the imputation, we use two working survival model to define the nearest neighbor imputing risk set. One model is for the event times and the other for the censoring times. Based on the imputing risk set, two nonparametric multiple imputation methods are considered: risk set imputation, and Kaplan-Meier estimator. For both methods a future event or censoring time is imputed for each censored observation. With a categorical auxiliary variable, we show that with a large number of imputes the estimates from the Kaplan-Meier imputation method correspond to the weighted Kaplan-Meier estimator. We also show that the Kaplan-Meier imputation method is robust to misspecification of either one of the two working models. In a simulation study with the time independent and time dependent auxiliary variables, we compare the multiple imputation approaches with an inverse probability of censoring weighted method. We show that all approaches can reduce bias due to dependent censoring and improve the efficiency. We apply the approaches to AIDS clinical trial data comparing ZDV and placebo, in which CD4 count is the time-dependent auxiliary variable

Spatial Effects in Energy-Efficient Residential HVAC Technology Adoption

Preprint; final version published as: Noonan, D. S., Hsieh, L.-H. C., & Matisoff, D. (2013). Spatial Effects in Energy-Efficient Residential HVAC Technology Adoption. Environment and Behavior, 45(4), 476–503. doi:10.1177/0013916511421664If your neighborhood adopts greener, energy-efficient residential heating, ventilating, and air conditioning (HVAC) systems, will your proenvironmental behavior become contagious, spilling over into adjacent neighborhoods’ HVAC adoptions? Objective data on more than 300,000 detailed single-family house sale records in the Greater Chicago area from 1992 to 2004 are aggregated to census block-group neighborhoods to answer that question. Spatial lag regression models show that spatial dependence or “contagion” exists for neighborhood adoption of energy-efficient HVACs. Specifically, if 625 of 726 homes in a demonstration neighborhood upgraded to green HVAC, data of this study predict that at least 98 upgrades would occur in adjacent neighborhoods, more than doubling their baseline adoption rates. This spatial multiplier substantially magnifies the effects of factors affecting adoption rates. These results have important policy implications, especially in the context of new standards for neighborhood development, such as Leadership in Energy and Environmental Design (LEED) or Low-Impact Development standards

Economic, sociological, and neighbor dimensions of energy efficiency adoption behaviors: Evidence from the U.S. residential heating and air conditioning market

This study identifies factors that affect the adoption behavior for residential Heating, Ventilating, and Air Conditioning (HVAC) systems, including a spatial and temporal contagion effect, house characteristics, and other economic and contextual factors. The study draws on a dataset of house sale records in the greater Chicago area, spanning 1992–2004. First-differenced models and restricting the sample to new construction allow separate identification of adoption determinants for homeowners and for developers, respectively. We show that attributes of the building stock and demographics influence adoption decisions of both homeowners and developers. This includes a strong influence of square footage, a modest spatial clustering effect for existing homes, a consistent deterrent effect of higher property tax rates, and a positive influence of neighborhood education levels. Adoption decisions for existing homeowners appear to be driven by different factors than sellers of newly constructed homes. Adoption coincided with multi-story homes for developers, and neighbor adoption rates predicted adoption by existing homeowners but not developers. The results highlight the need for more research into the social context of energy efficiency investment

Mitigating the Performance Impact of Network Failures in Public Clouds

Some faults in data center networks require hours to days to repair because they may need reboots, re-imaging, or manual work by technicians. To reduce traffic impact, cloud providers \textit{mitigate} the effect of faults, for example, by steering traffic to alternate paths. The state-of-art in automatic network mitigations uses simple safety checks and proxy metrics to determine mitigations. SWARM, the approach described in this paper, can pick orders of magnitude better mitigations by estimating end-to-end connection-level performance (CLP) metrics. At its core is a scalable CLP estimator that quickly ranks mitigations with high fidelity and, on failures observed at a large cloud provider, outperforms the state-of-the-art by over 700$\times$ in some cases