CNN-Cert: An Efficient Framework for Certifying Robustness of Convolutional Neural Networks

Verifying robustness of neural network classifiers has attracted great interests and attention due to the success of deep neural networks and their unexpected vulnerability to adversarial perturbations. Although finding minimum adversarial distortion of neural networks (with ReLU activations) has been shown to be an NP-complete problem, obtaining a non-trivial lower bound of minimum distortion as a provable robustness guarantee is possible. However, most previous works only focused on simple fully-connected layers (multilayer perceptrons) and were limited to ReLU activations. This motivates us to propose a general and efficient framework, CNN-Cert, that is capable of certifying robustness on general convolutional neural networks. Our framework is general -- we can handle various architectures including convolutional layers, max-pooling layers, batch normalization layer, residual blocks, as well as general activation functions; our approach is efficient -- by exploiting the special structure of convolutional layers, we achieve up to 17 and 11 times of speed-up compared to the state-of-the-art certification algorithms (e.g. Fast-Lin, CROWN) and 366 times of speed-up compared to the dual-LP approach while our algorithm obtains similar or even better verification bounds. In addition, CNN-Cert generalizes state-of-the-art algorithms e.g. Fast-Lin and CROWN. We demonstrate by extensive experiments that our method outperforms state-of-the-art lower-bound-based certification algorithms in terms of both bound quality and speed.Comment: Accepted by AAAI 201

Efficient Neural Network Robustness Certification with General Activation Functions

Finding minimum distortion of adversarial examples and thus certifying robustness in neural network classifiers for given data points is known to be a challenging problem. Nevertheless, recently it has been shown to be possible to give a non-trivial certified lower bound of minimum adversarial distortion, and some recent progress has been made towards this direction by exploiting the piece-wise linear nature of ReLU activations. However, a generic robustness certification for general activation functions still remains largely unexplored. To address this issue, in this paper we introduce CROWN, a general framework to certify robustness of neural networks with general activation functions for given input data points. The novelty in our algorithm consists of bounding a given activation function with linear and quadratic functions, hence allowing it to tackle general activation functions including but not limited to four popular choices: ReLU, tanh, sigmoid and arctan. In addition, we facilitate the search for a tighter certified lower bound by adaptively selecting appropriate surrogates for each neuron activation. Experimental results show that CROWN on ReLU networks can notably improve the certified lower bounds compared to the current state-of-the-art algorithm Fast-Lin, while having comparable computational efficiency. Furthermore, CROWN also demonstrates its effectiveness and flexibility on networks with general activation functions, including tanh, sigmoid and arctan.Comment: Accepted by NIPS 2018. Huan Zhang and Tsui-Wei Weng contributed equall

Glycemia and peak incremental indices of six popular fruits in Taiwan: healthy and Type 2 diabetes subjects compared

The aim of this study was to evaluate the glycemic index and peak incremental indices of six popular fruits in Taiwan, comparing healthy subjects (n = 20) and patients with Type 2 diabetes (n = 17). The six kinds of fruits tested were grapes, Asian pears, guavas, golden kiwifruit, lychees and bananas. Glycemic index values were tested according to the standard glycemic index testing protocol. The glycemic index and peak incremental indices were calculated according to published formulas. In Type 2 diabetes subjects, the glycemic index values of grapes, Asian pears, guavas, golden kiwifruit, lychees and bananas were 49.0 ± 4.5, 25.9 ± 2.9, 32.8 ± 5.2, 47.0 ± 6.5, 60.0 ± 8.0 and 41.3 ± 3.5. In healthy subjects, the glycemic index values were 49.1 ± 7.3, 18.0 ± 5.4, 31.1 ± 5.1, 47.3 ± 12.1, 47.9 ± 6.8 and 35.1 ± 5.6. There was no significant difference in glycemic index values between healthy and Type 2 diabetes subjects. There was also no significant difference in PII when comparing healthy subjects and subjects with Type 2 diabetes. In conclusion, glycemic index and peak incremental indices in healthy subjects can be approximately the same for Type 2 diabetes

Biomechanical comparison of lumbar spine instability between laminectomy and bilateral laminotomy for spinal stenosis syndrome – an experimental study in porcine model

<p>Abstract</p> <p>Background</p> <p>The association of lumbar spine instability between laminectomy and laminotomy has been clinically studied, but the corresponding <it>in vitro </it>biomechanical studies have not been reported. We investigated the hypothesis that the integrity of the posterior complex (spinous process-interspinous ligament-spinous process) plays an important role on the postoperative spinal stability in decompressive surgery.</p> <p>Methods</p> <p>Eight porcine lumbar spine specimens were studied. Each specimen was tested intact and after two decompression procedures. All posterior components were preserved in Group A (Intact). In Group B (Bilateral laminotomy), the inferior margin of L4 lamina and superior margin of L5 lamina were removed, but the L4–L5 supraspinous ligament was preserved. Fenestrations were made on both sides. In Group C (Laminectomy) the lamina and spinous processes of lower L4 and upper L5 were removed. Ligamentum flavum and supraspinous ligament of L4–L5 were removed. A hydraulic testing machine was used to generate an increasing moment up to 8400 N-mm in flexion and extension. Intervertebral displacement at decompressive level L4–L5 was measured by extensometer</p> <p>Results</p> <p>The results indicated that, under extension motion, intervertebral displacement between the specimen in intact form and at two different decompression levels did not significantly differ (<it>P </it>> 0.05). However, under flexion motion, intervertebral displacement of the laminectomy specimens at decompression level L4–L5 was statistically greater than in intact or bilateral laminotomy specimens (<it>P </it>= 0.0000963 and <it>P </it>= 0.000418, respectively). No difference was found between intact and bilateral laminotomy groups. (<it>P </it>> 0.05).</p> <p>Conclusion</p> <p>We concluded that a lumbar spine with posterior complex integrity is less likely to develop segment instability than a lumbar spine with a destroyed anchoring point for supraspinous ligament.</p

Web-based computer adaptive assessment of individual perceptions of job satisfaction for hospital workplace employees

<p>Abstract</p> <p>Background</p> <p>To develop a web-based computer adaptive testing (CAT) application for efficiently collecting data regarding workers' perceptions of job satisfaction, we examined whether a 37-item Job Content Questionnaire (JCQ-37) could evaluate the job satisfaction of individual employees as a single construct.</p> <p>Methods</p> <p>The JCQ-37 makes data collection via CAT on the internet easy, viable and fast. A Rasch rating scale model was applied to analyze data from 300 randomly selected hospital employees who participated in job-satisfaction surveys in 2008 and 2009 via non-adaptive and computer-adaptive testing, respectively.</p> <p>Results</p> <p>Of the 37 items on the questionnaire, 24 items fit the model fairly well. Person-separation reliability for the 2008 surveys was 0.88. Measures from both years and item-8 job satisfaction for groups were successfully evaluated through item-by-item analyses by using <it>t</it>-test. Workers aged 26 - 35 felt that job satisfaction was significantly worse in 2009 than in 2008.</p> <p>Conclusions</p> <p>A Web-CAT developed in the present paper was shown to be more efficient than traditional computer-based or pen-and-paper assessments at collecting data regarding workers' perceptions of job content.</p

Biomechanical comparison of pedicle screw fixation strength among three different screw trajectories using single vertebrae and one-level functional spinal unit

Three key factors are responsible for the biomechanical performance of pedicle screw fixation: screw mechanical characteristics, bone quality and insertion techniques. To the best of the authors’ knowledge, no study has directly compared the biomechanical performance among three trajectories, i.e., the traditional trajectory (TT), modified trajectory (MT) and cortical bone trajectory (CBT), in a porcine model. This study compared the pullout strength and insertion torque of three trajectory methods in single vertebrae, the pullout strength and fixation stiffness including flexion, extension, and lateral bending in a one-level instrumented functional spinal unit (FSU) that mimics the in vivo configuration were clarified. A total of 18 single vertebrae and 18 FSUs were randomly assigned into three screw insertion methods (n = 6 in each trajectory group). In the TT group, the screw converged from its entry point, passed completely inside the pedicle, was parallel to the superior endplate, was located in the superior third of the vertebral body and reached to at least the anterior third of the vertebral body. In the MT group, the convergent angle was similar to that of the TT method but directed caudally to the anterior inferior margin of the vertebral body. The results of insertion torque and pullout strength in single vertebrae were analyzed; in addition, the stiffness and pullout strength in the one-level FSU were also investigated. This study demonstrated that, in single vertebrae, the insertion torque was significantly higher in CBT groups than in TT and MT groups (p &lt; 0.05). The maximal pullout strength was significantly higher in MT groups than in TT and CBT groups (p &lt; 0.05). There was no significant difference in stiffness in the three motions among all groups. The maximal pullout strength in FSUs of MT and CBT groups were significantly higher than the TT groups (p &lt; 0.05). We concluded that either MT or CBT provides better biomechanical performance than TT in single vertebrae or FSUs. The lack of significance of stiffness in FSUs among three methods suggested that MT or CBT could be a reasonable alternative to TT if the traditional trajectory was not feasible