CNN-Cert: An Efficient Framework for Certifying Robustness of Convolutional Neural Networks

Verifying robustness of neural network classifiers has attracted great interests and attention due to the success of deep neural networks and their unexpected vulnerability to adversarial perturbations. Although finding minimum adversarial distortion of neural networks (with ReLU activations) has been shown to be an NP-complete problem, obtaining a non-trivial lower bound of minimum distortion as a provable robustness guarantee is possible. However, most previous works only focused on simple fully-connected layers (multilayer perceptrons) and were limited to ReLU activations. This motivates us to propose a general and efficient framework, CNN-Cert, that is capable of certifying robustness on general convolutional neural networks. Our framework is general -- we can handle various architectures including convolutional layers, max-pooling layers, batch normalization layer, residual blocks, as well as general activation functions; our approach is efficient -- by exploiting the special structure of convolutional layers, we achieve up to 17 and 11 times of speed-up compared to the state-of-the-art certification algorithms (e.g. Fast-Lin, CROWN) and 366 times of speed-up compared to the dual-LP approach while our algorithm obtains similar or even better verification bounds. In addition, CNN-Cert generalizes state-of-the-art algorithms e.g. Fast-Lin and CROWN. We demonstrate by extensive experiments that our method outperforms state-of-the-art lower-bound-based certification algorithms in terms of both bound quality and speed.Comment: Accepted by AAAI 201

Efficient Neural Network Robustness Certification with General Activation Functions

Finding minimum distortion of adversarial examples and thus certifying robustness in neural network classifiers for given data points is known to be a challenging problem. Nevertheless, recently it has been shown to be possible to give a non-trivial certified lower bound of minimum adversarial distortion, and some recent progress has been made towards this direction by exploiting the piece-wise linear nature of ReLU activations. However, a generic robustness certification for general activation functions still remains largely unexplored. To address this issue, in this paper we introduce CROWN, a general framework to certify robustness of neural networks with general activation functions for given input data points. The novelty in our algorithm consists of bounding a given activation function with linear and quadratic functions, hence allowing it to tackle general activation functions including but not limited to four popular choices: ReLU, tanh, sigmoid and arctan. In addition, we facilitate the search for a tighter certified lower bound by adaptively selecting appropriate surrogates for each neuron activation. Experimental results show that CROWN on ReLU networks can notably improve the certified lower bounds compared to the current state-of-the-art algorithm Fast-Lin, while having comparable computational efficiency. Furthermore, CROWN also demonstrates its effectiveness and flexibility on networks with general activation functions, including tanh, sigmoid and arctan.Comment: Accepted by NIPS 2018. Huan Zhang and Tsui-Wei Weng contributed equall

Towards Fast Computation of Certified Robustness for ReLU Networks

Verifying the robustness property of a general Rectified Linear Unit (ReLU) network is an NP-complete problem [Katz, Barrett, Dill, Julian and Kochenderfer CAV17]. Although finding the exact minimum adversarial distortion is hard, giving a certified lower bound of the minimum distortion is possible. Current available methods of computing such a bound are either time-consuming or delivering low quality bounds that are too loose to be useful. In this paper, we exploit the special structure of ReLU networks and provide two computationally efficient algorithms Fast-Lin and Fast-Lip that are able to certify non-trivial lower bounds of minimum distortions, by bounding the ReLU units with appropriate linear functions Fast-Lin, or by bounding the local Lipschitz constant Fast-Lip. Experiments show that (1) our proposed methods deliver bounds close to (the gap is 2-3X) exact minimum distortion found by Reluplex in small MNIST networks while our algorithms are more than 10,000 times faster; (2) our methods deliver similar quality of bounds (the gap is within 35% and usually around 10%; sometimes our bounds are even better) for larger networks compared to the methods based on solving linear programming problems but our algorithms are 33-14,000 times faster; (3) our method is capable of solving large MNIST and CIFAR networks up to 7 layers with more than 10,000 neurons within tens of seconds on a single CPU core. In addition, we show that, in fact, there is no polynomial time algorithm that can approximately find the minimum $\ell_1$ adversarial distortion of a ReLU network with a $0.99\ln n$ approximation ratio unless $\mathsf{NP}$=$\mathsf{P}$, where $n$ is the number of neurons in the network.Comment: Tsui-Wei Weng and Huan Zhang contributed equall

Stochastic simulation and robust design optimization of integrated photonic filters

Manufacturing variations are becoming an unavoidable issue in modern fabrication processes; therefore, it is crucial to be able to include stochastic uncertainties in the design phase. In this paper, integrated photonic coupled ring resonator filters are considered as an example of significant interest. The sparsity structure in photonic circuits is exploited to construct a sparse combined generalized polynomial chaos model, which is then used to analyze related statistics and perform robust design optimization. Simulation results show that the optimized circuits are more robust to fabrication process variations and achieve a reduction of 11%–35% in the mean square errors of the 3 dB bandwidth compared to unoptimized nominal designs.MIT Skoltech InitiativeProgetto Roberto Rocca (Seed Funds)National Science Foundation (U.S.) (AIM Photonics Center. Contract 1227020-EEC)Semiconductor Research Corporatio

Life-space, frailty, and health-related quality of life

INTRODUCTION: Life-space and frailty are closely linked to health-related quality of life and understanding their inter-relationship could indicate potential intervention targets for improving quality of life. We set out to examine the relationship between frailty and life-space and their relative impact on quality of life measures. METHODS: Using cross-sectional data from a population-representative cohort of people aged ≥ 70 years, we assessed quality of life with the EuroQol Health Index tool (5-levels) (EQ-5D-5L). We also undertook a life-space assessment and derived a frailty index. Linear regression models estimated EQ-5D-5L scores (dependent variable) using life-space assessment, frailty index and interactions between them. All models were adjusted by age, sex, lifestyle, and social care factors. RESULTS: A higher EQ-5D Index was associated with higher life-space (0.02 per life-space assessment score, 95%CI: 0.01 to 0.03, p < 0.01) and decreasing frailty (-0.1 per SD, 95%CI: -0.1 to -0.1, p < 0.01). There was evidence of an interaction between life-space and frailty, where the steepest gradient for life-space and EQ-5D was in those with the highest frailty (interaction term = 0.02 per SD of frailty, 95%CI: 0.01 to 0.03, p < 0.01). CONCLUSION: Individuals with the highest frailty were twice as likely to have higher quality of life in association with a larger life-space. Interventions designed to improve quality of life in frail older people could focus on increasing a person's life-space

Persistent delirium in older hospital patients: an updated systematic review and meta-analysis

Introduction: Delirium is associated with future dementia progression. Yet whether this occurs subclinically over months and years, or persistent delirium merges into worsened dementia is not understood. Our objective was to estimate the prevalence of persistent delirium and understand variation in its duration. Methods: We adopted an identical search strategy to a previous systematic review, only including studies using a recognised diagnostic framework for ascertaining delirium at follow-up (persistent delirium). Studies included hospitalised older patients outside critical and palliative care settings. We searched MEDLINE, EMBASE, PsycINFO and the Cochrane Database of Systematic Reviews on 11th January 2022. We applied risk of bias assessments based on Standards of Reporting of Neurological Disorders criteria and assessed strength of recommendations using the grading of recommendation, assessment, development and evaluation (GRADE) approach. Estimates were pooled across studies using random-effects meta-analysis, and we estimated associations with follow-up duration using robust error meta-regression. Results: We identified 13 new cohorts, which we added to 10 from the previous systematic review (23 relevant studies, with 39 reports of persistent delirium at 7 time-points in 3186 individuals admitted to hospital care (mean age 82 years and 41% dementia prevalence). Studies were mainly at moderate risk of bias. Pooled delirium prevalence estimates at discharge were 36% (95% CI 22% to 51%, 13 studies). Robust error meta-regression did not show variation in prevalence of persistent delirium over time (-1.6% per month, 95% CI -4.8 to 1.6, p=0.08). Margins estimates for this model indicate a prevalence of persistent delirium of 16% (95% CI 6% to 25%) at 12 months. Conclusions: This systematic review emphasises the importance of delirium as a persistent and extensive problem (GRADE certainty = moderate), raising questions on chronic delirium as a clinical entity and how it might evolve into dementia. Addressing persistent delirium will require a whole-system, integrated approach to detect, follow-up and implement opportunities for recovery across all healthcare settings

Measuring Public Utilization Perception Potential of Unmanned Aircraft Systems

The integration of Unmanned Aircraft Systems (UAS) into the National Air Space (NAS) in recent times has been met by mixed public responses. The paper establishes four constructs each of which encapsulates multiple backgrounds and concerns of the stakeholders: functional knowledge, utilization trust, operational integration support, and safety risk-benefits. The paper hypothesizes that these constructs can serve as underlying components for a research instrument namely, the Public Utilization Perception Potential (PUPP) which can be used to assess the opinions of the public on UAS integration into NAS. Responses from the public on items in a beta-tested survey instrument were analyzed for construct validity and reliability using Principal Axis Factoring (PAF). Four factors that suggest constructs underlying PUPP instrument were derived. Using Structural Equation Model (SEM) approach, a hypothesized measurement model of PUPP was further validated and the final measurement model showed good fit of the observed data based on the RMSEA goodness-of-fit index (0.034). The paper further assessed the strength of relationships between the underlying constructs of PUPP. The results suggest that approximately 65% and 27% of all respondents had partial and no knowledge, respectively, about UAS integration into the NAS. There was a statistically significant difference in the mean scores on safety-risk benefits on UAS among gender. Males were found to be more likely to patronize unmanned commercial passenger services compared to females. The results indicated a statistically significant difference in UAS knowledge and perceptions across educational levels. It was rather counter-intuitive as respondents with lower educational levels were found to be more knowledgeable about UAS compared to those of higher levels based on the results of this study. Investments in information resources and training by industry, government and academia may be helpful to improve UAS knowledge and perceptions among the public if any commercial utilization as a transport mode will be feasible. Future studies will replicate the study in countries other than the United States