DeepBern-Nets: Taming the Complexity of Certifying Neural Networks using Bernstein Polynomial Activations and Precise Bound Propagation

Formal certification of Neural Networks (NNs) is crucial for ensuring their safety, fairness, and robustness. Unfortunately, on the one hand, sound and complete certification algorithms of ReLU-based NNs do not scale to large-scale NNs. On the other hand, incomplete certification algorithms are easier to compute, but they result in loose bounds that deteriorate with the depth of NN, which diminishes their effectiveness. In this paper, we ask the following question; can we replace the ReLU activation function with one that opens the door to incomplete certification algorithms that are easy to compute but can produce tight bounds on the NN's outputs? We introduce DeepBern-Nets, a class of NNs with activation functions based on Bernstein polynomials instead of the commonly used ReLU activation. Bernstein polynomials are smooth and differentiable functions with desirable properties such as the so-called range enclosure and subdivision properties. We design a novel algorithm, called Bern-IBP, to efficiently compute tight bounds on DeepBern-Nets outputs. Our approach leverages the properties of Bernstein polynomials to improve the tractability of neural network certification tasks while maintaining the accuracy of the trained networks. We conduct comprehensive experiments in adversarial robustness and reachability analysis settings to assess the effectiveness of the proposed Bernstein polynomial activation in enhancing the certification process. Our proposed framework achieves high certified accuracy for adversarially-trained NNs, which is often a challenging task for certifiers of ReLU-based NNs. Moreover, using Bern-IBP bounds for certified training results in NNs with state-of-the-art certified accuracy compared to ReLU networks. This work establishes Bernstein polynomial activation as a promising alternative for improving NN certification tasks across various applications

Formal Verication of Neural Networks

Neural networks(NNs) have been widely used over the past decade at the core of intelligentsystems from sensing modules to learning-based controllers. They've also been deployed in dierent safety-critical domains including healthcare and transportation. However, recent work has shown that NNs are fragile and can make dangerous mistakes that are either unintentional or adversarial. As a consequence, formal verication of NNs holds the promise of providing safety guarantees on the behaviour of such systems. We focus our work on ReLU networks as it is the most widely used activation function. Exact formal verication of ReLU NNs was proved to be NP-hard due to the combinatorial nature of the problem, therefore all of the current verication methods use some relaxation of the problem. We propose a novel framework for formal verication of ReLU neural networks that can ensure that they satisfy some polytopic specications on the input and the output of the network. Our approach uses a relaxed convex program to mitigate the combinatorial complexity of the problem together with some optimization heuristics to eciently verify the satisfaction of the specication on a given network. We have implemented our algorithm in a toolkit, PeregriNN. To test PeregriNN, we run it on two test benches in dierent domains. First, we achieve SOTA results on verifying the adversarial robustness of dierent networks on the MNIST dataset. Second, we verify the safety of a neural network controlled autonomous robot in a structured environment

Formal Verification of Neural Networks: Algorithms and Applications

Neural networks (NNs) have become the backbone of intelligent systems, with applications ranging from sensing modules to learning-based controllers. Their deployment in safety-critical domains such as healthcare and transportation underscores their significance. However, the fragility of NNs and their potential for dangerous mistakes, whether unintentional or adversarial, necessitates rigorous checks on their behavior. This thesis delves into the Formal Verification of NNs, a process that can provides formal guarantees on their behavior and ensures the reliability and robustness of these systems.The cornerstone of this work is a novel algorithm, PeregriNN, that has advanced the Formal Verification of NNs. This algorithm has been used to verify various NN properties such as Adversarial Robustness, safety of autonomous systems, and in demonstrating the fairness of NNs by innovatively formulating the fairness property into a Formal Verification problem. Further, this thesis explores a new type of activation function that simplifies the formal verification process, albeit at the cost of increased training time. This exploration, coupled with a collaborative effort that led to the proposal of a polynomial approximation method for ReLU NNs to formally verify them, provides valuable insights into the balance between verification ease and computational efficiency. These methods offer promising approaches to the formal verification of neural networks, further enhancing the robustness and reliability of these systems. The thesis also extends the application of these theories and algorithms to the safety of autonomous systems with neural network controllers. This practical application underscores the real-world implications of formal verification in ensuring the safety and reliability of autonomous systems. In summary, this thesis provides a comprehensive understanding of the formal verification of neural networks, underscoring the importance of algorithm development and its real-world applications. The findings from this study contribute significantly to this critical field of study, with implications for the fairness, safety, and robustness of NNs

CertiFair: A Framework for Certified Global Fairness of Neural Networks

We consider the problem of whether a Neural Network (NN) model satisfies global individual fairness. Individual Fairness (defined in (Dwork et al. 2012)) suggests that similar individuals with respect to a certain task are to be treated similarly by the decision model. In this work, we have two main objectives. The first is to construct a verifier which checks whether the fairness property holds for a given NN in a classification task or provides a counterexample if it is violated, i.e., the model is fair if all similar individuals are classified the same, and unfair if a pair of similar individuals are classified differently. To that end, we construct a sound and complete verifier that verifies global individual fairness properties of ReLU NN classifiers using distance-based similarity metrics. The second objective of this paper is to provide a method for training provably fair NN classifiers from unfair (biased) data. We propose a fairness loss that can be used during training to enforce fair outcomes for similar individuals. We then provide provable bounds on the fairness of the resulting NN. We run experiments on commonly used fairness datasets that are publicly available and we show that global individual fairness can be improved by 96 % without a significant drop in test accuracy

PEREGRiNN: Penalized-Relaxation Greedy Neural Network Verifier

Neural Networks (NNs) have increasingly apparent safety implications commensurate with their proliferation in real-world applications: both unanticipated as well as adversarial misclassifications can result in fatal outcomes. As a consequence, techniques of formal verification have been recognized as crucial to the design and deployment of safe NNs. In this paper, we introduce a new approach to formally verify the most commonly considered safety specifications for ReLU NNs -- i.e. polytopic specifications on the input and output of the network. Like some other approaches, ours uses a relaxed convex program to mitigate the combinatorial complexity of the problem. However, unique in our approach is the way we use a convex solver not only as a linear feasibility checker, but also as a means of penalizing the amount of relaxation allowed in solutions. In particular, we encode each ReLU by means of the usual linear constraints, and combine this with a convex objective function that penalizes the discrepancy between the output of each neuron and its relaxation. This convex function is further structured to force the largest relaxations to appear closest to the input layer; this provides the further benefit that the most problematic neurons are conditioned as early as possible, when conditioning layer by layer. This paradigm can be leveraged to create a verification algorithm that is not only faster in general than competing approaches, but is also able to verify considerably more safety properties; we evaluated PEREGRiNN on a standard MNIST robustness verification suite to substantiate these claims

Nanoparticles fabricated from the bioactive tilapia scale collagen for wound healing: Experimental approach.

The creation of innovative wound-healing nanomaterials based on natural compounds emerges as a top research goal. This research aimed to create a gel containing collagen nanoparticles and evaluate its therapeutic potential for skin lesions. Collagen nanoparticles were produced from fish scales using desolvation techniques. Using SDS PAGE electrophoresis, Fourier transform infrared spectroscopy (FTIR) as well as the structure of the isolated collagen and its similarities to collagen type 1 were identified. The surface morphology of the isolated collagen and its reformulation into nanoparticles were examined using transmission and scanning electron microscopy. A Zeta sizer was used to examine the size, zeta potential, and distribution of the synthesized collagen nanoparticles. The cytotoxicity of the nanomaterials was investigated and an experimental model was used to evaluate the wound healing capability. The overall collagen output from Tilapia fish scales was 42%. Electrophoretic patterns revealed that the isolated collagen included a unique protein with chain bands of 126-132 kDa and an elevated beta band of 255 kDa. When compared to the isolated collagen, the collagen nanoparticles' FTIR results revealed a significant drop in the amide II (42% decrease) and amide III (32% decrease) band intensities. According to SEM analysis, the generated collagen nanoparticles ranged in size from 100 to 350 nm, with an average diameter of 182 nm determined by the zeta sizer. The produced collagen nanoparticles were polydispersed in nature and had an equivalent average zeta potential of -17.7 mV. Cytotoxicity study showed that, when treating fibroblast cells with collagen nanoparticle concentrations, very mild morphological alterations were detected after human skin fibroblasts were treated with collagen nanoparticles 32 μg/ml for 24 hours, as higher concentrations of collagen nanoparticles caused cell detachment. Macroscopical and histological investigations proved that the fabricated fish scale collagen nanoparticles promoted the healing process in comparison to the saline group

Analysis and prediction of nutritional outcome of patients with pediatric inflammatory bowel disease from Bahrain

Abstract Background Inflammatory bowel disease (IBD) is a chronic gastrointestinal disease that causes anorexia, malabsorption, and increased energy requirements. Childhood IBD can significantly impact nutritional status and future health. Objective This study aimed to analyze the nutritional status of patients with pediatric IBD at presentation and during follow-up and to identify predictors of nutritional outcome. Methods This retrospective cohort study reviewed the medical records of children diagnosed with IBD in the Pediatric Department, Salmaniya Medical Complex, Bahrain, 1984 − 2023. Demographic data, clinical characteristics, and anthropometric data were collected. World Health Organization growth standards were used to interpret nutritional status. Results Of the 165 patients, 99 (60%) had anthropometric data at presentation, and 130 (78.8%) had follow-up data. Most patients were males (64.6%) and had Crohn’s disease (CD) (56.2%), while 43.8% had ulcerative colitis (UC). The median age at presentation was 10.9 years and the mean follow-up duration was 12.6 years. At presentation, 53.5% of the patients were malnourished, that decreased to 46.9% on follow-up. Thinness was reduced from 27.3% at presentation to 12.1% at follow-up (p = 0.003). There was an increased tendency to normal weight on follow-up (59.6%) compared to time of presentation (46.5%), p = 0.035. Overweightness showed a non-significant increase from 26.3% at presentation to 28.3% at follow-up (p = 0.791). Children with IBD were more likely to become obese when they grow up to adulthood (2.3% versus 20.5%, respectively, p < 0.001). Weight-for-age, and height-for-age at presentation were higher among CD compared to UC, but body mass index (BMI) at follow-up was higher among UC patients (p < 0.05). Thinness at follow up was associated with very early-onset disease (p = 0.02), lower weight and BMI at presentation (p < 0.001 each), younger age at follow-up (p = 0.002), pediatric age group (p = 0.023), lower hematocrit (p = 0.017), and higher C-reactive protein (p = 0.007). Overweight at follow up was associated with increased weight and BMI at presentation (p < 0.001 each), longer disease duration (p = 0.005), older age (p = 0.002), and azathioprine intake (p = 0.026). Considering follow-up duration, univariate analysis exhibited that Bahraini nationality, post-diagnosis disease duration, age at follow-up, occurrence of diarrhea, height, and BMI at presentation were factors that decreased liability to abnormal nutritional status, while CD, history of weight loss, perianal disease, and skin rash, and intake of prednisolone expressed increased liability of abnormal nutritional status (p < 0.05). Conclusion Pediatric IBD is associated with a high incidence of malnutrition. Thinness is more prominent at presentation, while overweight is higher on follow-up. Multiple risk factors aggravating abnormal nutritional status were highlighted. Accordingly, nutritional counseling should be prioritized in a multidisciplinary approach

Stoma-free survival after anastomotic leak following rectal cancer resection: worldwide cohort of 2470 patients

Background: The optimal treatment of anastomotic leak after rectal cancer resection is unclear. This worldwide cohort study aimed to provide an overview of four treatment strategies applied. Methods: Patients from 216 centres and 45 countries with anastomotic leak after rectal cancer resection between 2014 and 2018 were included. Treatment was categorized as salvage surgery, faecal diversion with passive or active (vacuum) drainage, and no primary/secondary faecal diversion. The primary outcome was 1-year stoma-free survival. In addition, passive and active drainage were compared using propensity score matching (2: 1). Results: Of 2470 evaluable patients, 388 (16.0 per cent) underwent salvage surgery, 1524 (62.0 per cent) passive drainage, 278 (11.0 per cent) active drainage, and 280 (11.0 per cent) had no faecal diversion. One-year stoma-free survival rates were 13.7, 48.3, 48.2, and 65.4 per cent respectively. Propensity score matching resulted in 556 patients with passive and 278 with active drainage. There was no statistically significant difference between these groups in 1-year stoma-free survival (OR 0.95, 95 per cent c.i. 0.66 to 1.33), with a risk difference of -1.1 (95 per cent c.i. -9.0 to 7.0) per cent. After active drainage, more patients required secondary salvage surgery (OR 2.32, 1.49 to 3.59), prolonged hospital admission (an additional 6 (95 per cent c.i. 2 to 10) days), and ICU admission (OR 1.41, 1.02 to 1.94). Mean duration of leak healing did not differ significantly (an additional 12 (-28 to 52) days). Conclusion: Primary salvage surgery or omission of faecal diversion likely correspond to the most severe and least severe leaks respectively. In patients with diverted leaks, stoma-free survival did not differ statistically between passive and active drainage, although the increased risk of secondary salvage surgery and ICU admission suggests residual confounding

Stoma-free Survival After Rectal Cancer Resection With Anastomotic Leakage: Development and Validation of a Prediction Model in a Large International Cohort.

Objective:To develop and validate a prediction model (STOMA score) for 1-year stoma-free survival in patients with rectal cancer (RC) with anastomotic leakage (AL).Background:AL after RC resection often results in a permanent stoma.Methods:This international retrospective cohort study (TENTACLE-Rectum) encompassed 216 participating centres and included patients who developed AL after RC surgery between 2014 and 2018. Clinically relevant predictors for 1-year stoma-free survival were included in uni and multivariable logistic regression models. The STOMA score was developed and internally validated in a cohort of patients operated between 2014 and 2017, with subsequent temporal validation in a 2018 cohort. The discriminative power and calibration of the models' performance were evaluated.Results:This study included 2499 patients with AL, 1954 in the development cohort and 545 in the validation cohort. Baseline characteristics were comparable. One-year stoma-free survival was 45.0% in the development cohort and 43.7% in the validation cohort. The following predictors were included in the STOMA score: sex, age, American Society of Anestesiologist classification, body mass index, clinical M-disease, neoadjuvant therapy, abdominal and transanal approach, primary defunctioning stoma, multivisceral resection, clinical setting in which AL was diagnosed, postoperative day of AL diagnosis, abdominal contamination, anastomotic defect circumference, bowel wall ischemia, anastomotic fistula, retraction, and reactivation leakage. The STOMA score showed good discrimination and calibration (c-index: 0.71, 95% CI: 0.66-0.76).Conclusions:The STOMA score consists of 18 clinically relevant factors and estimates the individual risk for 1-year stoma-free survival in patients with AL after RC surgery, which may improve patient counseling and give guidance when analyzing the efficacy of different treatment strategies in future studies