Few-shot Learning with Multi-scale Self-supervision

Learning concepts from the limited number of datapoints is a challenging task usually addressed by the so-called one- or few-shot learning. Recently, an application of second-order pooling in few-shot learning demonstrated its superior performance due to the aggregation step handling varying image resolutions without the need of modifying CNNs to fit to specific image sizes, yet capturing highly descriptive co-occurrences. However, using a single resolution per image (even if the resolution varies across a dataset) is suboptimal as the importance of image contents varies across the coarse-to-fine levels depending on the object and its class label e. g., generic objects and scenes rely on their global appearance while fine-grained objects rely more on their localized texture patterns. Multi-scale representations are popular in image deblurring, super-resolution and image recognition but they have not been investigated in few-shot learning due to its relational nature complicating the use of standard techniques. In this paper, we propose a novel multi-scale relation network based on the properties of second-order pooling to estimate image relations in few-shot setting. To optimize the model, we leverage a scale selector to re-weight scale-wise representations based on their second-order features. Furthermore, we propose to a apply self-supervised scale prediction. Specifically, we leverage an extra discriminator to predict the scale labels and the scale discrepancy between pairs of images. Our model achieves state-of-the-art results on standard few-shot learning datasets

Dual Adversarial Alignment for Realistic Support-Query Shift Few-shot Learning

Support-query shift few-shot learning aims to classify unseen examples (query set) to labeled data (support set) based on the learned embedding in a low-dimensional space under a distribution shift between the support set and the query set. However, in real-world scenarios the shifts are usually unknown and varied, making it difficult to estimate in advance. Therefore, in this paper, we propose a novel but more difficult challenge, RSQS, focusing on Realistic Support-Query Shift few-shot learning. The key feature of RSQS is that the individual samples in a meta-task are subjected to multiple distribution shifts in each meta-task. In addition, we propose a unified adversarial feature alignment method called DUal adversarial ALignment framework (DuaL) to relieve RSQS from two aspects, i.e., inter-domain bias and intra-domain variance. On the one hand, for the inter-domain bias, we corrupt the original data in advance and use the synthesized perturbed inputs to train the repairer network by minimizing distance in the feature level. On the other hand, for intra-domain variance, we proposed a generator network to synthesize hard, i.e., less similar, examples from the support set in a self-supervised manner and introduce regularized optimal transportation to derive a smooth optimal transportation plan. Lastly, a benchmark of RSQS is built with several state-of-the-art baselines among three datasets (CIFAR100, mini-ImageNet, and Tiered-Imagenet). Experiment results show that DuaL significantly outperforms the state-of-the-art methods in our benchmark.Comment: Best student paper in PAKDD 202

Understanding the temporal dynamics of visual hallucinations in Parkinson's Disease with dementia

PhD ThesisBackground Integrative models of visual hallucinations (VH) posit that the symptom requires disruptions to both bottom-up and top-down visual processing. Although many lines of evidence point to a mixture of aberrant processing and disconnection between key nodes in the visual system, in particular the dorsal and ventral attention networks, there have been no attempts to understand the dynamic behaviour of these systems in Parkinson’s disease with dementia (PDD) with VH. Aims The primary aim of this thesis was to explore the correlates of synaptic communication in the visual system and how spatio-temporal dynamics of the early visual system are altered in relation to the severity of VH. The secondary aim was to help understand the balance between the contributions of bottom-up and top-down processing for the experience of VH in PDD. Methods An assortment of investigative approaches, including resting state electroencephalography (EEG), visual evoked potentials (VEPs), and concurrent EEG and transcranial magnetic stimulation (TMS) were applied in a group of PDD patients with a range of VH severities (n = 26) and contrasted with a group of age matched healthy controls (n = 17). Results Latency of the N1 component was similar between groups, suggesting intact transfer between the retina and the cortex. However, PDD patients had an inherent reduction in the amplitude of the VEP components and displayed a pattern of declining P1 latencies in association with more frequent and severe VH. Evoked potentials arising from TMS of the striate cortex were similar in amplitude and latency for each of the components between PDD and controls. However, inter-component activity at several stages was altered in the PDD group, whilst the frequency and severity of VH was positively associated with the amplitudes of several components in the occipital and parietal regions. Finally, attentional modulation as measured by the alpha-band reactivity was also compromised in PDD patients. iv Conclusions These data provide neurophysiological evidence that both early bottom-up and top-down dysfunctions of the visual system occur in PDD patients who hallucinate, thus supporting integrative models of VH.National Institute for Health Research (NIHR) Biomedical Research Unit (BRU)

Task-Adaptive Negative Class Envision for Few-Shot Open-Set Recognition

Recent works seek to endow recognition systems with the ability to handle the open world. Few shot learning aims for fast learning of new classes from limited examples, while open-set recognition considers unknown negative class from the open world. In this paper, we study the problem of few-shot open-set recognition (FSOR), which learns a recognition system robust to queries from new sources with few examples and from unknown open sources. To achieve that, we mimic human capability of envisioning new concepts from prior knowledge, and propose a novel task-adaptive negative class envision method (TANE) to model the open world. Essentially we use an external memory to estimate a negative class representation. Moreover, we introduce a novel conjugate episode training strategy that strengthens the learning process. Extensive experiments on four public benchmarks show that our approach significantly improves the state-of-the-art performance on few-shot open-set recognition. Besides, we extend our method to generalized few-shot open-set recognition (GFSOR), where we also achieve performance gains on MiniImageNet