PIAT: Physics Informed Adversarial Training for Solving Partial Differential Equations

In this paper, we propose the physics informed adversarial training (PIAT) of neural networks for solving nonlinear differential equations (NDE). It is well-known that the standard training of neural networks results in non-smooth functions. Adversarial training (AT) is an established defense mechanism against adversarial attacks, which could also help in making the solution smooth. AT include augmenting the training mini-batch with a perturbation that makes the network output mismatch the desired output adversarially. Unlike formal AT, which relies only on the training data, here we encode the governing physical laws in the form of nonlinear differential equations using automatic differentiation in the adversarial network architecture. We compare PIAT with PINN to indicate the effectiveness of our method in solving NDEs for up to 10 dimensions. Moreover, we propose weight decay and Gaussian smoothing to demonstrate the PIAT advantages. The code repository is available at https://github.com/rohban-lab/PIAT

Unmasking the Chameleons: A Benchmark for Out-of-Distribution Detection in Medical Tabular Data

Despite their success, Machine Learning (ML) models do not generalize effectively to data not originating from the training distribution. To reliably employ ML models in real-world healthcare systems and avoid inaccurate predictions on out-of-distribution (OOD) data, it is crucial to detect OOD samples. Numerous OOD detection approaches have been suggested in other fields - especially in computer vision - but it remains unclear whether the challenge is resolved when dealing with medical tabular data. To answer this pressing need, we propose an extensive reproducible benchmark to compare different methods across a suite of tests including both near and far OODs. Our benchmark leverages the latest versions of eICU and MIMIC-IV, two public datasets encompassing tens of thousands of ICU patients in several hospitals. We consider a wide array of density-based methods and SOTA post-hoc detectors across diverse predictive architectures, including MLP, ResNet, and Transformer. Our findings show that i) the problem appears to be solved for far-OODs, but remains open for near-OODs; ii) post-hoc methods alone perform poorly, but improve substantially when coupled with distance-based mechanisms; iii) the transformer architecture is far less overconfident compared to MLP and ResNet

The impact of guided inquiry methods of teaching on the critical thinking of high school students

The objective pursued by the present study is to investigate the impact of guided inquiry and traditional methods of teaching on the critical thinking skills among second grade high school students. Given the purpose, a total of 190 second grade high school students were chosen through random, multi-step and cluster sampling methods in the form of 8 classes and placed into 8 experimental and control groups. A pre-test post-test design was administered to the control group. The demographic information was collected by a researcher –made questionnaire and the thinking skills information was determined by Watson - Glaser test. Two- factor covariance method was used for data analysis. Results showed that the guided inquiry method of teaching had significant impact (lower than 0.05) on the critical thinking skills of students in inference and conclusion subscales. The impact of gender factor on the students’ critical thinking was significant, in terms of conclusion and interpretation subscales as well. The impact of interaction between gender and teaching method was also significant in inference and interpretation subscales. Keywords: critical thinking, guided inquiry teaching method, traditional teaching method

Your Out-of-Distribution Detection Method is Not Robust!

Out-of-distribution (OOD) detection has recently gained substantial attention due to the importance of identifying out-of-domain samples in reliability and safety. Although OOD detection methods have advanced by a great deal, they are still susceptible to adversarial examples, which is a violation of their purpose. To mitigate this issue, several defenses have recently been proposed. Nevertheless, these efforts remained ineffective, as their evaluations are based on either small perturbation sizes, or weak attacks. In this work, we re-examine these defenses against an end-to-end PGD attack on in/out data with larger perturbation sizes, e.g. up to commonly used $\epsilon=8/255$ for the CIFAR-10 dataset. Surprisingly, almost all of these defenses perform worse than a random detection under the adversarial setting. Next, we aim to provide a robust OOD detection method. In an ideal defense, the training should expose the model to almost all possible adversarial perturbations, which can be achieved through adversarial training. That is, such training perturbations should based on both in- and out-of-distribution samples. Therefore, unlike OOD detection in the standard setting, access to OOD, as well as in-distribution, samples sounds necessary in the adversarial training setup. These tips lead us to adopt generative OOD detection methods, such as OpenGAN, as a baseline. We subsequently propose the Adversarially Trained Discriminator (ATD), which utilizes a pre-trained robust model to extract robust features, and a generator model to create OOD samples. Using ATD with CIFAR-10 and CIFAR-100 as the in-distribution data, we could significantly outperform all previous methods in the robust AUROC while maintaining high standard AUROC and classification accuracy. The code repository is available at https://github.com/rohban-lab/ATD .Comment: Accepted to NeurIPS 202

Blacksmith: Fast Adversarial Training of Vision Transformers via a Mixture of Single-step and Multi-step Methods

Despite the remarkable success achieved by deep learning algorithms in various domains, such as computer vision, they remain vulnerable to adversarial perturbations. Adversarial Training (AT) stands out as one of the most effective solutions to address this issue; however, single-step AT can lead to Catastrophic Overfitting (CO). This scenario occurs when the adversarially trained network suddenly loses robustness against multi-step attacks like Projected Gradient Descent (PGD). Although several approaches have been proposed to address this problem in Convolutional Neural Networks (CNNs), we found out that they do not perform well when applied to Vision Transformers (ViTs). In this paper, we propose Blacksmith, a novel training strategy to overcome the CO problem, specifically in ViTs. Our approach utilizes either of PGD-2 or Fast Gradient Sign Method (FGSM) randomly in a mini-batch during the adversarial training of the neural network. This will increase the diversity of our training attacks, which could potentially mitigate the CO issue. To manage the increased training time resulting from this combination, we craft the PGD-2 attack based on only the first half of the layers, while FGSM is applied end-to-end. Through our experiments, we demonstrate that our novel method effectively prevents CO, achieves PGD-2 level performance, and outperforms other existing techniques including N-FGSM, which is the state-of-the-art method in fast training for CNNs

The development of critical thinking skills in physics and sociology curricula

The present study aims to compare the impact of guided inquiry and traditional teaching methods on critical thinking skills of second-grade high school students in physics and sociology courses. Given the purpose, a total of 190 second grade high school students were chosen through random, multi-step and cluster sampling methods in the form of 8 classes and placed into 8 experimental and control groups in physics and sociology courses. A pre-test post-test design was administered to the control group. In order to collect information about participants, two tools were employed. The demographic information was collected by a researcher-made questionnaire and the thinking skills information was determined by Watson - Glaser test. Two- factor covariance method was utilized for data analysis. Results showed that the impact of guided inquiry teaching method on the critical thinking skills of students in inference and conclusion subscales, and the effect of subject in conclusion and interpretation subscales was significan

Dynamic wetting properties of PDMS pseudo-brushes: Four-phase contact point dynamics case

We investigate the wetting properties of PDMS (Polydimethylsiloxane) pseudo-brush anchored on glass substrates. These PDMS pseudo-brushes exhibit a significantly lower contact angle hysteresis compared to hydrophobic silanized substrates. The effect of different molar masses of the used PDMS on the wetting properties seems negligible. The surface roughness and thickness of the PDMS pseudo-brush are measured by atomic force microscopy and x-ray reflectivity. The outcome shows that these surfaces are extremely smooth (topologically and chemically), which explains the reduction in contact angle hysteresis. These special features make this kind of surfaces very useful for wetting experiments. Here, the dynamics of the four-phase contact point are studied on these surfaces. The four-phase contact point dynamics on PDMS pseudo-brushes deviate substantially from its dynamics on other substrates. These changes depend only a little on the molar mass of the used PDMS. In general, PDMS pseudo-brushes increase the traveling speed of four-phase contact point on the surface and change the associated power law of position vs time

Dynamic wetting properties of PDMS pseudo-brushes: Four-phase contact point dynamics case

We investigate the wetting properties of PDMS (Polydimethylsiloxane) pseudo-brush anchored on glass substrates. These PDMS pseudobrushes exhibit a significantly lower contact angle hysteresis compared to hydrophobic silanized substrates. The effect of different molar masses of the used PDMS on the wetting properties seems negligible. The surface roughness and thickness of the PDMS pseudo-brush are measured by atomic force microscopy and x-ray reflectivity. The outcome shows that these surfaces are extremely smooth (topologically and chemically), which explains the reduction in contact angle hysteresis. These special features make this kind of surfaces very useful for wetting experiments. Here, the dynamics of the four-phase contact point are studied on these surfaces. The four-phase contact point dynamics on PDMS pseudo-brushes deviate substantially from its dynamics on other substrates. These changes depend only a little on the molar mass of the used PDMS. In general, PDMS pseudo-brushes increase the traveling speed of four-phase contact point on the surface and change the associated power law of position vs time