A Comprehensive Survey on Knowledge Distillation of Diffusion Models

Diffusion Models (DMs), also referred to as score-based diffusion models, utilize neural networks to specify score functions. Unlike most other probabilistic models, DMs directly model the score functions, which makes them more flexible to parametrize and potentially highly expressive for probabilistic modeling. DMs can learn fine-grained knowledge, i.e., marginal score functions, of the underlying distribution. Therefore, a crucial research direction is to explore how to distill the knowledge of DMs and fully utilize their potential. Our objective is to provide a comprehensible overview of the modern approaches for distilling DMs, starting with an introduction to DMs and a discussion of the challenges involved in distilling them into neural vector fields. We also provide an overview of the existing works on distilling DMs into both stochastic and deterministic implicit generators. Finally, we review the accelerated diffusion sampling algorithms as a training-free method for distillation. Our tutorial is intended for individuals with a basic understanding of generative models who wish to apply DM's distillation or embark on a research project in this field

Enhancing Adversarial Robustness via Score-Based Optimization

Adversarial attacks have the potential to mislead deep neural network classifiers by introducing slight perturbations. Developing algorithms that can mitigate the effects of these attacks is crucial for ensuring the safe use of artificial intelligence. Recent studies have suggested that score-based diffusion models are effective in adversarial defenses. However, existing diffusion-based defenses rely on the sequential simulation of the reversed stochastic differential equations of diffusion models, which are computationally inefficient and yield suboptimal results. In this paper, we introduce a novel adversarial defense scheme named ScoreOpt, which optimizes adversarial samples at test-time, towards original clean data in the direction guided by score-based priors. We conduct comprehensive experiments on multiple datasets, including CIFAR10, CIFAR100 and ImageNet. Our experimental results demonstrate that our approach outperforms existing adversarial defenses in terms of both robustness performance and inference speed

Purify++: Improving Diffusion-Purification with Advanced Diffusion Models and Control of Randomness

Adversarial attacks can mislead neural network classifiers. The defense against adversarial attacks is important for AI safety. Adversarial purification is a family of approaches that defend adversarial attacks with suitable pre-processing. Diffusion models have been shown to be effective for adversarial purification. Despite their success, many aspects of diffusion purification still remain unexplored. In this paper, we investigate and improve upon three limiting designs of diffusion purification: the use of an improved diffusion model, advanced numerical simulation techniques, and optimal control of randomness. Based on our findings, we propose Purify++, a new diffusion purification algorithm that is now the state-of-the-art purification method against several adversarial attacks. Our work presents a systematic exploration of the limits of diffusion purification methods

Mining the Relationship between Emoji Usage Patterns and Personality

Emojis have been widely used in textual communications as a new way to convey nonverbal cues. An interesting observation is the various emoji usage patterns among different users. In this paper, we investigate the correlation between user personality traits and their emoji usage patterns, particularly on overall amounts and specific preferences. To achieve this goal, we build a large Twitter dataset which includes 352,245 users and over 1.13 billion tweets associated with calculated personality traits and emoji usage patterns. Our correlation and emoji prediction results provide insights into the power of diverse personalities that lead to varies emoji usage patterns as well as its potential in emoji recommendation tasks.Comment: To appear at The International AAAI Conference on Web and Social Media (ICWSM) 201

Enhanced superconductivity at the interface of W/Sr2_{2}RuO4_{4} point contact

Differential resistance measurements are conducted for point contacts (PCs) between tungsten tip approaching along the $c$ axis direction and the $ab$ plane of Sr$_{2}$RuO$_{4}$ single crystal. Three key features are found. Firstly, within 0.2 mV there is a dome like conductance enhancement due to Andreev reflection at the normal-superconducting interface. By pushing the W tip further, the conductance enhancement increases from 3\% to more than 20\%, much larger than that was previously reported, probably due to the pressure exerted by the tip. Secondly, there are also superconducting like features at bias higher than 0.2 mV which persists up to 6.2 K, resembling the enhanced superconductivity under uniaxial pressure for bulk Sr$_{2}$RuO$_{4}$ crystals but more pronounced here. Third, the logarithmic background can be fitted with the Altshuler-Aronov theory of tunneling into quasi two dimensional electron system, consistent with the highly anisotropic electronic system in Sr$_{2}$RuO$_{4}$.Comment: prb style, 9 pages, 8 fig

Remote Medication Status Prediction for Individuals with Parkinson's Disease using Time-series Data from Smartphones

Medication for neurological diseases such as the Parkinson's disease usually happens remotely away from hospitals. Such out-of-lab environments pose challenges in collecting timely and accurate health status data. Individual differences in behavioral signals collected from wearable sensors also lead to difficulties in adopting current general machine learning analysis pipelines. To address these challenges, we present a method for predicting the medication status of Parkinson's disease patients using the public mPower dataset, which contains 62,182 remote multi-modal test records collected on smartphones from 487 patients. The proposed method shows promising results in predicting three medication statuses objectively: Before Medication (AUC=0.95), After Medication (AUC=0.958), and Another Time (AUC=0.976) by examining patient-wise historical records with the attention weights learned through a Transformer model. Our method provides an innovative way for personalized remote health sensing in a timely and objective fashion which could benefit a broad range of similar applications.Comment: Accepted to ICDH-2023. Camera ready with supplementary materia