Gradient Surgery for One-shot Unlearning on Generative Model

Recent regulation on right-to-be-forgotten emerges tons of interest in unlearning pre-trained machine learning models. While approximating a straightforward yet expensive approach of retrain-from-scratch, recent machine unlearning methods unlearn a sample by updating weights to remove its influence on the weight parameters. In this paper, we introduce a simple yet effective approach to remove a data influence on the deep generative model. Inspired by works in multi-task learning, we propose to manipulate gradients to regularize the interplay of influence among samples by projecting gradients onto the normal plane of the gradients to be retained. Our work is agnostic to statistics of the removal samples, outperforming existing baselines while providing theoretical analysis for the first time in unlearning a generative model.Comment: ICML 2023 Workshop on Generative AI & La

Geometrically Aligned Transfer Encoder for Inductive Transfer in Regression Tasks

Transfer learning is a crucial technique for handling a small amount of data that is potentially related to other abundant data. However, most of the existing methods are focused on classification tasks using images and language datasets. Therefore, in order to expand the transfer learning scheme to regression tasks, we propose a novel transfer technique based on differential geometry, namely the Geometrically Aligned Transfer Encoder (GATE). In this method, we interpret the latent vectors from the model to exist on a Riemannian curved manifold. We find a proper diffeomorphism between pairs of tasks to ensure that every arbitrary point maps to a locally flat coordinate in the overlapping region, allowing the transfer of knowledge from the source to the target data. This also serves as an effective regularizer for the model to behave in extrapolation regions. In this article, we demonstrate that GATE outperforms conventional methods and exhibits stable behavior in both the latent space and extrapolation regions for various molecular graph datasets.Comment: 12+11 pages, 6+1 figures, 0+7 table

Can We Utilize Pre-trained Language Models within Causal Discovery Algorithms?

Scaling laws have allowed Pre-trained Language Models (PLMs) into the field of causal reasoning. Causal reasoning of PLM relies solely on text-based descriptions, in contrast to causal discovery which aims to determine the causal relationships between variables utilizing data. Recently, there has been current research regarding a method that mimics causal discovery by aggregating the outcomes of repetitive causal reasoning, achieved through specifically designed prompts. It highlights the usefulness of PLMs in discovering cause and effect, which is often limited by a lack of data, especially when dealing with multiple variables. Conversely, the characteristics of PLMs which are that PLMs do not analyze data and they are highly dependent on prompt design leads to a crucial limitation for directly using PLMs in causal discovery. Accordingly, PLM-based causal reasoning deeply depends on the prompt design and carries out the risk of overconfidence and false predictions in determining causal relationships. In this paper, we empirically demonstrate the aforementioned limitations of PLM-based causal reasoning through experiments on physics-inspired synthetic data. Then, we propose a new framework that integrates prior knowledge obtained from PLM with a causal discovery algorithm. This is accomplished by initializing an adjacency matrix for causal discovery and incorporating regularization using prior knowledge. Our proposed framework not only demonstrates improved performance through the integration of PLM and causal discovery but also suggests how to leverage PLM-extracted prior knowledge with existing causal discovery algorithms

Deep neural networks show an equivalent and often superior performance to dermatologists in onychomycosis diagnosis: Automatic construction of onychomycosis datasets by region-based convolutional deep neural network.

Although there have been reports of the successful diagnosis of skin disorders using deep learning, unrealistically large clinical image datasets are required for artificial intelligence (AI) training. We created datasets of standardized nail images using a region-based convolutional neural network (R-CNN) trained to distinguish the nail from the background. We used R-CNN to generate training datasets of 49,567 images, which we then used to fine-tune the ResNet-152 and VGG-19 models. The validation datasets comprised 100 and 194 images from Inje University (B1 and B2 datasets, respectively), 125 images from Hallym University (C dataset), and 939 images from Seoul National University (D dataset). The AI (ensemble model; ResNet-152 + VGG-19 + feedforward neural networks) results showed test sensitivity/specificity/ area under the curve values of (96.0 / 94.7 / 0.98), (82.7 / 96.7 / 0.95), (92.3 / 79.3 / 0.93), (87.7 / 69.3 / 0.82) for the B1, B2, C, and D datasets. With a combination of the B1 and C datasets, the AI Youden index was significantly (p = 0.01) higher than that of 42 dermatologists doing the same assessment manually. For B1+C and B2+ D dataset combinations, almost none of the dermatologists performed as well as the AI. By training with a dataset comprising 49,567 images, we achieved a diagnostic accuracy for onychomycosis using deep learning that was superior to that of most of the dermatologists who participated in this study

Electron-Transfer Kinetics through Interfaces between Electron-Transport and Ion-Transport Layers in Solid-State Dye-Sensitized Solar Cells Utilizing Solid Polymer Electrolyte

The origin of the differences between the performance parameters found for dye-sensitized solar cells (DSCs) using liquid and poly(ethylene oxide)-based solid polymer electrolytes has been investigated. Limitations associated with poor polymer electrolyte penetration and ionic diffusion have been analyzed together with other effects such as the dye regeneration rate, the conduction band edge shift, and the electron recombination kinetics occurring in the solid polymer electrolyte. We have found that dye regeneration was faster for sensitized TiO2 films fully wetted with polymer electrolyte than that with liquid cells. This new result was attributed to a 0.2 eV decrease in the dye highest occupied molecular orbital energy yielding to an increase in the driving force for dye regeneration. These understandings may contribute to a further increase in the energy-conversion efficiency of DSCs employing solid polymer electrolyte.This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea Center for Artificial Photosynthesis (KCAP) (No. 2009-0093883)

Effect of Washing Condition on the Fracture Strength, and the Degree of Conversion of 3D Printing Resin

This study compared the surface roughness, contact angle, surface energy, residual monomers, degree of conversion, and flexural strength of 3D-printed dental resin under various washing conditions. The specimens were printed with a digital light processing (DLP) printer and were divided into four groups: the group dipped in IPA for 5 s (IPA-D), the group washed in IPA for 1 min (IPA-1), the group washed in IPA for 10 min (IPA-10), and the group washed with TPM for 10 min (TPM-10). Following, the groups were redivided into two groups: a cured group and an uncured group. All experimental data were statistically analyzed using one-way analysis of variance and Tukey’s test. In all groups, the surface roughness showed a value of 1.2–1.8 μm, with no significant difference (p > 0.05). Contact angle showed a significant difference between the three groups using IPA and the TPM group, whereby the TPM-washed specimen showed a low contact angle (p p p > 0.05). The washing time and washing solution type of the 3D printing material had no significant effect on surface roughness and flexural strength

Automated establishment of the A1 dataset using R-CNN and two CNNs.

<p>1) CNN hand & foot image selector which automatically detects and selects hand and foot images from the entire dataset. 2) R-CNN nail part extractor which identifies and crops part of the nail from individual clinical images by differentiating the nail plate from the background. 3) CNN fine image selector which rules out photographs with inadequate composition or focus.</p

The change in the AUC of onychomycosis classification (ensemble model; test = B1 + C dataset) and the adequate value of the fine image selector.

<p>Fine image selector is a CNN model (ResNet-152) trained to analyze photos, categorize into adequate, inadequate, or wrong image, and produce an adequate value as an output. As the amount of added brightness and noise increased, the AUC (Test = B1 and C dataset), which represents the classification accuracy, and the adequate value of the fine image selector decreased.</p

Summary of image characteristics and available demographic information.

<p>Summary of image characteristics and available demographic information.</p