Fast Yet Effective Machine Unlearning

Unlearning the data observed during the training of a machine learning (ML) model is an important task that can play a pivotal role in fortifying the privacy and security of ML-based applications. This paper raises the following questions: (i) can we unlearn a single or multiple classes of data from an ML model without looking at the full training data even once? (ii) can we make the process of unlearning fast and scalable to large datasets, and generalize it to different deep networks? We introduce a novel machine unlearning framework with error-maximizing noise generation and impair-repair based weight manipulation that offers an efficient solution to the above questions. An error-maximizing noise matrix is learned for the class to be unlearned using the original model. The noise matrix is used to manipulate the model weights to unlearn the targeted class of data. We introduce impair and repair steps for a controlled manipulation of the network weights. In the impair step, the noise matrix along with a very high learning rate is used to induce sharp unlearning in the model. Thereafter, the repair step is used to regain the overall performance. With very few update steps, we show excellent unlearning while substantially retaining the overall model accuracy. Unlearning multiple classes requires a similar number of update steps as for the single class, making our approach scalable to large problems. Our method is quite efficient in comparison to the existing methods, works for multi-class unlearning, doesn't put any constraints on the original optimization mechanism or network design, and works well in both small and large-scale vision tasks. This work is an important step towards fast and easy implementation of unlearning in deep networks. We will make the source code publicly available

TabSynDex: A Universal Metric for Robust Evaluation of Synthetic Tabular Data

Synthetic tabular data generation becomes crucial when real data is limited, expensive to collect, or simply cannot be used due to privacy concerns. However, producing good quality synthetic data is challenging. Several probabilistic, statistical, and generative adversarial networks (GANs) based approaches have been presented for synthetic tabular data generation. Once generated, evaluating the quality of the synthetic data is quite challenging. Some of the traditional metrics have been used in the literature but there is lack of a common, robust, and single metric. This makes it difficult to properly compare the effectiveness of different synthetic tabular data generation methods. In this paper we propose a new universal metric, TabSynDex, for robust evaluation of synthetic data. TabSynDex assesses the similarity of synthetic data with real data through different component scores which evaluate the characteristics that are desirable for "high quality" synthetic data. Being a single score metric, TabSynDex can also be used to observe and evaluate the training of neural network based approaches. This would help in obtaining insights that was not possible earlier. Further, we present several baseline models for comparative analysis of the proposed evaluation metric with existing generative models

Optimization of DNA delivery by three classes of hybrid nanoparticle/DNA complexes

Plasmid DNA encoding a luciferase reporter gene was complexed with each of six different hybrid nanoparticles (NPs) synthesized from mixtures of poly (D, L-lactide-co-glycolide acid) (PLGA 50:50) and the cationic lipids DOTAP (1, 2-Dioleoyl-3-Trimethyammonium-Propane) or DC-Chol {3β-[N-(N', N'-Dimethylaminoethane)-carbamyl] Cholesterol}. Particles were 100-400 nm in diameter and the resulting complexes had DNA adsorbed on the surface (out), encapsulated (in), or DNA adsorbed and encapsulated (both). A luciferase reporter assay was used to quantify DNA expression in 293 cells for the uptake of six different NP/DNA complexes. Optimal DNA delivery occurred for 105 cells over a range of 500 ng - 10 μg of NPs containing 20-30 μg DNA per 1 mg of NPs. Uptake of DNA from NP/DNA complexes was found to be 500-600 times as efficient as unbound DNA. Regression analysis was performed and lines were drawn for DNA uptake over a four week interval. NP/DNA complexes with adsorbed NPs (out) showed a large initial uptake followed by a steep slope of DNA decline and large angle of declination; lines from uptake of adsorbed and encapsulated NPs (both) also exhibited a large initial uptake but was followed by a gradual slope of DNA decline and small angle of declination, indicating longer times of luciferase expression in 293 cells. NPs with encapsulated DNA only (in), gave an intermediate activity. The latter two effects were best seen with DOTAP-NPs while the former was best seen with DC-Chol-NPs. These results provide optimal conditions for using different hybrid NP/DNA complexes in vitro and in the future, will be tested in vivo

A Novel Peptide Derived from Human Apolipoprotein E Is an Inhibitor of Tumor Growth and Ocular Angiogenesis

Angiogenesis is a hallmark of tumor development and metastasis and now a validated target for cancer treatment. We previously reported that a novel dimer peptide (apoEdp) derived from the receptor binding region of human apolipoprotein E (apoE) inhibits virus-induced angiogenesis. However, its role in tumor anti-angiogenesis is unknown. This study demonstrates that apoEdp has anti-angiogenic property in vivo through reduction of tumor growth in a mouse model and ocular angiogenesis in a rabbit eye model. Our in vitro studies show that apoEdp inhibits human umbilical vein endothelial cell proliferation, migration, invasion and capillary tube formation. We document that apoEdp inhibits vascular endothelial growth factor-induced Flk-1 activation as well as downstream signaling pathways that involve c-Src, Akt, eNOS, FAK, and ERK1/2. These in vitro data suggest potential sites of the apoE dipeptide inhibition that could occur in vivo

Molecular modelling of MHC class I carbohydrates

96-103In this article we present the results of molecular modelling of four complex carbohydrates which have been found in the MHC class I proteins. Though these proteins show diversity in their sequences, the glycosylation sites are highly conserved indicating a possible structural/functional role of the glycan chain. Similar glycan chains have been found linked with other proteins of completely different function, such as IgG, and erythropoeitin. Thus, the molecular modelling of these carbohydrates will not only provide structural/dynamic information of these complex molecules but will also provide conformational information which can be utilised to build the glycoprotein model s. The results presented here indicate that although several linkages show less conformational flexibility, terminal linkages can be quite flexible

Degenerate intermolecular and intramolecular proton-transfer reactions: electronic structure of the transition states

Density functional theory (DFT) calculations are performed on a series of double and single proton-transfer reactions to study the variation in polarizations in complexes during the dynamics of proton transfer from one isoenergetic, hydrogen-bonded ground-state structure to the other. The isotropic average polarizability (αav) shows an interesting single-humped profile with a maxima coinciding with the transition state of the reaction. Similar profiles are also computed at Nd:YaG frequencies. The origin of the maximal polarizability at the transition state is traced to maximal charge separation and large D (donor)-A (acceptor) distances. Maximal polarizability for the transition state suggests an interesting, novel, and less memory extensive computational tool to locate the transition state for hydrogen-transfer reactions in hydrogen-bonded complexes

Structural Variation in Homopolymers Bearing Zwitterionic and Ionic Liquid Pendants for Achieving Tunable Multi-Stimuli Responsiveness and Hierarchical Nanoaggregates

A series of monomers comprising units bearing both imidazolium bromide ionic liquid (IL) and zwitterionic imidazolium alkyl carboxylate moieties with different alkyl spacer groups are designed and synthesized. RAFT polymerization of these monomers produces a new class of ionic homopolymers, named poly­(zwitterionic ionic liquid)­s (PZILs), which behave like poly­(ionic liquid)­s as well as poly­(zwitterion)­s depending on the solution pH. Such PZILs exhibit an isoelectric point (pI) at pH 5.7, where they exist in their zwitterionic form, making them dual responsive to both pH and temperature. Above pH 5, the aqueous transparent solution of PZIL transforms into turbid suspension due to the formation of insoluble hierarchical nanoaggregates (NAs) of various morphologies such as small spheres, large spheres, flower-like, dendrite-shaped, and dendritic fibril-like depending upon the solution pH and PZILs’ structures. The dissolution of aggregates upon heating and reaggregation upon cooling suggests existence of reversible upper critical solution temperature (UCST)-type phase transition above pH 5. Below pH 5, owing to the presence of cationic IL moieties, aqueous PZIL solution exhibits transparent-to-turbid transition due to the formation of anion-induced NAs of various dendritic morphologies upon addition of various chaotropic anions within the Hofmeister series. Upon heating, this colloidal turbid suspension becomes transparent, showing a distinct UCST-type phase transition, and the process is reversible. It is easily possible to fine-tune the cloud point and morphologies of the NAs by changing various parameters such as molecular weight, concentrations, structure of PZILs, nature and concentration of anions, and solution pH

Understanding Peierls distortion in one-dimensional infinite V-chain and V-Bz multi-decker complex

We perform first-principles calculations based on density-functional theory to study the stable structures of one-dimensional (1D) linear infinite vanadium (V) chain. The calculation shows that it prefers to dimerize according to the Peierls theorem. However, in 1D infinite neutral V-benzene (V-Bz) multi-decker complex, the dimerization almost disappears because of the screening effect of the intervening benzene rings. Additionally, we study the effect of electronic correlations on dimerization in 1D chains. Our numerical analysis reveals that, although the strong electron-electron interaction suppresses the dimerization, strong electron-phonon coupling overwhelms it to gain stability through dimerization in such systems

Zero-Shot Machine Unlearning

Modern privacy regulations grant citizens the right to be forgotten by products, services and companies. In case of machine learning (ML) applications, this necessitates deletion of data not only from storage archives but also from ML models. Due to an increasing need for regulatory compliance required for ML applications, machine unlearning is becoming an emerging research problem. The right to be forgotten requests come in the form of removal of a certain set or class of data from the already trained ML model. Practical considerations preclude retraining of the model from scratch minus the deleted data. The few existing studies use either the whole training data, or a subset of training data, or some metadata stored during training to update the model weights for unlearning. However, strict regulatory compliance requires time-bound deletion of data. Thus, in many cases, no data related to the training process or training samples may be accessible even for the unlearning purpose. We therefore ask the question: is it possible to achieve unlearning with zero training samples? In this paper, we introduce the novel problem of zero-shot machine unlearning that caters for the extreme but practical scenario where zero original data samples are available for use. We then propose two novel solutions for zero-shot machine unlearning based on (a) error minimizing-maximizing noise and (b) gated knowledge transfer. These methods remove the information of the forget data from the model while maintaining the model efficacy on the retain data. The zero-shot approach offers good protection against the model inversion attacks and membership inference attacks. We introduce a new evaluation metric, Anamnesis Index (AIN) to effectively measure the quality of the unlearning method. The experiments show promising results for unlearning in deep learning models on benchmark vision data-sets