Prototypical Kernel Learning and Open-set Foreground Perception for Generalized Few-shot Semantic Segmentation

Generalized Few-shot Semantic Segmentation (GFSS) extends Few-shot Semantic Segmentation (FSS) to simultaneously segment unseen classes and seen classes during evaluation. Previous works leverage additional branch or prototypical aggregation to eliminate the constrained setting of FSS. However, representation division and embedding prejudice, which heavily results in poor performance of GFSS, have not been synthetical considered. We address the aforementioned problems by jointing the prototypical kernel learning and open-set foreground perception. Specifically, a group of learnable kernels is proposed to perform segmentation with each kernel in charge of a stuff class. Then, we explore to merge the prototypical learning to the update of base-class kernels, which is consistent with the prototype knowledge aggregation of few-shot novel classes. In addition, a foreground contextual perception module cooperating with conditional bias based inference is adopted to perform class-agnostic as well as open-set foreground detection, thus to mitigate the embedding prejudice and prevent novel targets from being misclassified as background. Moreover, we also adjust our method to the Class Incremental Few-shot Semantic Segmentation (CIFSS) which takes the knowledge of novel classes in a incremental stream. Extensive experiments on PASCAL-5i and COCO-20i datasets demonstrate that our method performs better than previous state-of-the-art.Comment: Accepted by ICCV202

MCP: Self-supervised Pre-training for Personalized Chatbots with Multi-level Contrastive Sampling

Personalized chatbots focus on endowing the chatbots with a consistent personality to behave like real users and further act as personal assistants. Previous studies have explored generating implicit user profiles from the user's dialogue history for building personalized chatbots. However, these studies only use the response generation loss to train the entire model, thus it is prone to suffer from the problem of data sparsity. Besides, they overemphasize the final generated response's quality while ignoring the correlations and fusions between the user's dialogue history, leading to rough data representations and performance degradation. To tackle these problems, we propose a self-supervised learning framework MCP for capturing better representations from users' dialogue history for personalized chatbots. Specifically, we apply contrastive sampling methods to leverage the supervised signals hidden in user dialog history, and generate the pre-training samples for enhancing the model. We design three pre-training tasks based on three types of contrastive pairs from user dialogue history, namely response pairs, sequence augmentation pairs, and user pairs. We pre-train the utterance encoder and the history encoder towards the contrastive objectives and use these pre-trained encoders for generating user profiles while personalized response generation. Experimental results on two real-world datasets show a significant improvement in our proposed model MCP compared with the existing methods

SeDR: Segment Representation Learning for Long Documents Dense Retrieval

Recently, Dense Retrieval (DR) has become a promising solution to document retrieval, where document representations are used to perform effective and efficient semantic search. However, DR remains challenging on long documents, due to the quadratic complexity of its Transformer-based encoder and the finite capacity of a low-dimension embedding. Current DR models use suboptimal strategies such as truncating or splitting-and-pooling to long documents leading to poor utilization of whole document information. In this work, to tackle this problem, we propose Segment representation learning for long documents Dense Retrieval (SeDR). In SeDR, Segment-Interaction Transformer is proposed to encode long documents into document-aware and segment-sensitive representations, while it holds the complexity of splitting-and-pooling and outperforms other segment-interaction patterns on DR. Since GPU memory requirements for long document encoding causes insufficient negatives for DR training, Late-Cache Negative is further proposed to provide additional cache negatives for optimizing representation learning. Experiments on MS MARCO and TREC-DL datasets show that SeDR achieves superior performance among DR models, and confirm the effectiveness of SeDR on long document retrieval

Quantum Multicritical Behavior for Coupled Optical Cavities with Driven Laser Fields

Quantum phase transitions with multicritical points are fascinating phenomena occurring in interacting quantum many-body systems. However, multicritical points predicted by theory have been rarely verified experimentally; finding multicritical points with specific behaviors and realizing their control remains a challenging topic. Here, we propose a system that a quantized light field interacts with a two-level atomic ensemble coupled by microwave fields in optical cavities, which is described by a generalized Dicke model. Multicritical points for the superradiant quantum phase transition are shown to occur. We determine the number and position of these critical points and demonstrate that they can be effectively manipulated through the tuning of system parameters. Particularly, we find that the quantum critical points can evolve into a Lifshitz point if the Rabi frequency of the light field is modulated periodically in time. Remarkably, the texture of atomic pseudo-spins can be used to characterize the quantum critical behaviors of the system. The magnetic orders of the three phases around the Lifshitz point, represented by the atomic pseudo-spins, are similar to those of an axial next-nearest-neighboring Ising model. The results reported here are beneficial for unveiling intriguing physics of quantum phase transitions and pave the way towards to find novel quantum multicritical phenomena based on the generalized Dicke model

Superconducting fluctuations and charge-4ee plaquette state at strong coupling

Recent experiments indicate that superconducting fluctuations also play an important role in overdoped cuprates. Here we apply the static auxiliary field Monte Carlo approach to study phase correlations of the pairing fields in a microscopic model with spin-singlet pairing interaction. We find that the short- and long-range phase correlations are well captured by the phase mutual information, which allows us to construct a theoretical phase diagram containing the uniform $d$-wave superconducting region, the phase fluctuating region, the local pairing region, and the disordered region. We show that the gradual development of phase coherence has a number of consequences on spectroscopic measurements, such as the development of the Fermi arc and the anisotropy in the angle-resolved spectra, scattering rate, entropy, specific heat, and quasiparticle dispersion, in good agreement with experimental observations. For strong coupling, our Monte Carlo simulation reveals an unexpected charge-4$e$ plaquette state with $d$-wave bonds, which competes with the uniform $d$-wave superconductivity and exhibits a U-shaped density of states

Highly pathogenic avian influenza A virus H5N1 NS1 protein induces caspase-dependent apoptosis in human alveolar basal epithelial cells

<p>Abstract</p> <p>Background</p> <p>It is widely considered that the multifunctional NS1 protein of influenza A viruses contributes significantly disease pathogenesis by modulating a number of virus and host-cell processes, but it is highly controversial whether this non-structural protein is a proapoptotic or antiapoptotic factor in infected cells.</p> <p>Results</p> <p>NS1 protein of influenza A/chicken/Jilin/2003 virus, a highly pathogenic H5N1 strain, could induce apoptosis in the carcinomic human alveolar basal epithelial cells (A549) by electron microscopic and flow cytometric analyses. NS1 protein-triggered apoptosis in A549 cells is via caspase-dependent pathway.</p> <p>Conclusions</p> <p>Influenza A virus NS1 protein serves as a strong inducer of apoptosis in infected human respiratory epithelial cells and plays a critical role in disease pathogenesis.</p

Heteroatom-doped core/shell carbonaceous framework materials : synthesis, characterization and electrochemical properties

Organic-inorganic hybrid core@shell nanospherical particles with 200 nm to 600 nm in diameter were formed between cyclomatrix poly(organophosphazenes) (POP) and zeolitic imidazolate framework-8 (ZIF-8) in a methanol solution at room temperature. This facile synthesis route produced core@shell spheres with controlled structure and properties, such as mono-dispersed particles, 50 nm to 100 nm shell thickness, surface area of 1557 m2 g-1 and homogenously doped Zn and heteroatoms (N, S, P, O, Cl). The POP/ZIF-8 core@shell structures were subsequently converted into porous carbonaceous materials, and investigated as anode materials in a lithium-ion coin cell battery. It showed a stable discharge capacity of 538 mA h g-1 over 250 cycles, high rate capability (0.1 C to 1 C) and excellent capacity retention, which are promising for rapid charge-discharge applications. Higher ZIF-8 loading ratio in the core@shell structure increased the capacity of the electrode material and stablised the lithiated active materials. The facile synthesis method and the carbonaceous framework materials are applicable for high performance energy storage materials in electrochemical power devices

Retentive Network: A Successor to Transformer for Large Language Models

In this work, we propose Retentive Network (RetNet) as a foundation architecture for large language models, simultaneously achieving training parallelism, low-cost inference, and good performance. We theoretically derive the connection between recurrence and attention. Then we propose the retention mechanism for sequence modeling, which supports three computation paradigms, i.e., parallel, recurrent, and chunkwise recurrent. Specifically, the parallel representation allows for training parallelism. The recurrent representation enables low-cost $O(1)$ inference, which improves decoding throughput, latency, and GPU memory without sacrificing performance. The chunkwise recurrent representation facilitates efficient long-sequence modeling with linear complexity, where each chunk is encoded parallelly while recurrently summarizing the chunks. Experimental results on language modeling show that RetNet achieves favorable scaling results, parallel training, low-cost deployment, and efficient inference. The intriguing properties make RetNet a strong successor to Transformer for large language models. Code will be available at https://aka.ms/retnet