DiffKendall: A Novel Approach for Few-Shot Learning with Differentiable Kendall's Rank Correlation

Few-shot learning aims to adapt models trained on the base dataset to novel tasks where the categories are not seen by the model before. This often leads to a relatively uniform distribution of feature values across channels on novel classes, posing challenges in determining channel importance for novel tasks. Standard few-shot learning methods employ geometric similarity metrics such as cosine similarity and negative Euclidean distance to gauge the semantic relatedness between two features. However, features with high geometric similarities may carry distinct semantics, especially in the context of few-shot learning. In this paper, we demonstrate that the importance ranking of feature channels is a more reliable indicator for few-shot learning than geometric similarity metrics. We observe that replacing the geometric similarity metric with Kendall's rank correlation only during inference is able to improve the performance of few-shot learning across a wide range of datasets with different domains. Furthermore, we propose a carefully designed differentiable loss for meta-training to address the non-differentiability issue of Kendall's rank correlation. Extensive experiments demonstrate that the proposed rank-correlation-based approach substantially enhances few-shot learning performance

Meta-Transformer: A Unified Framework for Multimodal Learning

Multimodal learning aims to build models that can process and relate information from multiple modalities. Despite years of development in this field, it still remains challenging to design a unified network for processing various modalities ($\textit{e.g.}$ natural language, 2D images, 3D point clouds, audio, video, time series, tabular data) due to the inherent gaps among them. In this work, we propose a framework, named Meta-Transformer, that leverages a $\textbf{frozen}$ encoder to perform multimodal perception without any paired multimodal training data. In Meta-Transformer, the raw input data from various modalities are mapped into a shared token space, allowing a subsequent encoder with frozen parameters to extract high-level semantic features of the input data. Composed of three main components: a unified data tokenizer, a modality-shared encoder, and task-specific heads for downstream tasks, Meta-Transformer is the first framework to perform unified learning across 12 modalities with unpaired data. Experiments on different benchmarks reveal that Meta-Transformer can handle a wide range of tasks including fundamental perception (text, image, point cloud, audio, video), practical application (X-Ray, infrared, hyperspectral, and IMU), and data mining (graph, tabular, and time-series). Meta-Transformer indicates a promising future for developing unified multimodal intelligence with transformers. Code will be available at https://github.com/invictus717/MetaTransformerComment: Project website: https://kxgong.github.io/meta_transformer

A Brief Online Mindfulness-Based Group Intervention for Psychological Distress Among Chinese Residents During COVID-19: a Pilot Randomized Controlled Trial

Objectives The coronavirus (COVID-19) global pandemic has increased psychological distress among the general population. The objective of this study is to evaluate a mindfulness-based intervention for psychological distress among Chinese residents during COVID-19. Methods This study used a switching replications design to test the feasibility and efficacy of a brief online mindfulness-based intervention for Chinese residents’ psychological distress. Fifty-one residents in the Hubei province were randomly allocated to two groups (experimental group and waitlist control group) with three waves of measurement at time 1, time 2, and time 3 for changes in mindfulness and psychological distress. Results In addition to significant within-group improvements over time for both groups, OLS linear regression with full information likelihood estimation revealed statistically significant between-group treatment effects across outcome domains, including mindfulness awareness, b = 2.84, p < 0.001, g = 6.92, psychological distress, b = −21.33, p < 0.001, g = 6.62, somatic symptoms, b = −6.22, p < 0.001, g = 4.42, depressive symptoms, b = −7.16, p < 0.001, g = 5.07, and anxiety symptoms, b = −8.09, p < 0.001, g = 6.84. Conclusions Results suggest that a brief online mindfulness-based intervention can be a feasible and promising intervention for improving mindfulness and decreasing psychological distress among Chinese residents staying at home during the COVID-19 outbreak. The study used a small convenience sample which led to a concern of external generalizability and with limited evaluation of long-term change.Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/167607/1/Zhang_2021_Article_ABriefOnlineMindfulness-BasedG.pdfDescription of Zhang_2021_Article_ABriefOnlineMindfulness-BasedG.pdf : Main articleSEL

DREAM+: Efficient Dataset Distillation by Bidirectional Representative Matching

Dataset distillation plays a crucial role in creating compact datasets with similar training performance compared with original large-scale ones. This is essential for addressing the challenges of data storage and training costs. Prevalent methods facilitate knowledge transfer by matching the gradients, embedding distributions, or training trajectories of synthetic images with those of the sampled original images. Although there are various matching objectives, currently the strategy for selecting original images is limited to naive random sampling. We argue that random sampling overlooks the evenness of the selected sample distribution, which may result in noisy or biased matching targets. Besides, the sample diversity is also not constrained by random sampling. Additionally, current methods predominantly focus on single-dimensional matching, where information is not fully utilized. To address these challenges, we propose a novel matching strategy called Dataset Distillation by Bidirectional REpresentAtive Matching (DREAM+), which selects representative original images for bidirectional matching. DREAM+ is applicable to a variety of mainstream dataset distillation frameworks and significantly reduces the number of distillation iterations by more than 15 times without affecting performance. Given sufficient training time, DREAM+ can further improve the performance and achieve state-of-the-art results. We have released the code at github.com/NUS-HPC-AI-Lab/DREAM+.Comment: This is an extension of the ICCV conference versio

Age moderates the association between psychological distress and engagement in mindfulness among cancer patients and survivors: A population-based study

Purpose We aim to evaluate the relationship between psychological distress and engagement in mindfulness among a national representative sample of cancer survivors. Sample and design Using the 2017 National Health Interview Survey, our final analytical sample included 3068 participants who reported having been diagnosed with cancer. Methods and analysis We used logistic regression analysis to assess the association and to test age as a moderator. We also conducted follow-up analysis using Fisher’s r-to-z transformation. All analyses were adjusted for complex sample weights. Findings Cancer survivors who had subclinical and clinical psychological distress were more likely to engage in mindfulness, OR = 1.59, 95% CI [1.24, 2.02] and OR = 1.45, 95% CI [1.02, 2.05], respectively. Age significantly moderated such association with the relationship much stronger among those who are younger (younger than 65 years old) than those who are older (65 years or older), b = 0.97, 95% CI [0.95, 0.99]. Conclusions The relationship between psychological distress and engagement in mindfulness differs significantly by a survivor’s age. Psychosocial oncological providers need to account for a survivor’s age when delivering mindfulness based care to address psychological distress.Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/167611/1/Zhang et al., 2020 Age moderator.pdfSEL