ICPC: Instance-Conditioned Prompting with Contrastive Learning for Semantic Segmentation

Modern supervised semantic segmentation methods are usually finetuned based on the supervised or self-supervised models pre-trained on ImageNet. Recent work shows that transferring the knowledge from CLIP to semantic segmentation via prompt learning can achieve promising performance. The performance boost comes from the feature enhancement with multimodal alignment, i.e., the dot product between vision and text embeddings. However, how to improve the multimodal alignment for better transfer performance in dense tasks remains underexplored. In this work, we focus on improving the quality of vision-text alignment from two aspects of prompting design and loss function, and present an instance-conditioned prompting with contrastive learning (ICPC) framework. First, compared with the static prompt designs, we reveal that dynamic prompting conditioned on image content can more efficiently utilize the text encoder for complex dense tasks. Second, we propose an align-guided contrastive loss to refine the alignment of vision and text embeddings. We further propose lightweight multi-scale alignment for better performance. Extensive experiments on three large-scale datasets (ADE20K, COCO-Stuff10k, and ADE20K-Full) demonstrate that ICPC brings consistent improvements across diverse backbones. Taking ResNet-50 as an example, ICPC outperforms the state-of-the-art counterpart by 1.71%, 1.05%, and 1.41% mIoU on the three datasets, respectively

Paper capillary force driven hollow channel as a platform for multiphase flows bioassays

AbstractThis paper develops a simple, inexpensive, and portable diagnostic assays that may be useful in remote settings, and in particular, in less industrialized countries where simple assays are becoming increasingly important for detecting disease and monitoring health. In this assays, the paper capillary force is first used to transport complex fluids such as whole blood or colloidal suspensions that contain particulates in a new type channel - paper capillary driven hollow channel, which offset the disadvantages of current paper microfluidic technologies. To demonstrate the various applications of the paper capillary force driven hollow channel, several devices are design and made to complete the purpose of exhibiting laminar flow in a T-junction microchannel, sheath a core stream in a three-inlet channel and transportation whole blood

Do the government subsidies inhibit the entity over-financialization? Fresh evidence from China

In order to verify effect of the industrial policies on solving the problem of market failure, we collect the data from China A-share listed companies among 2008-2019, and analyze the effect of government subsidies on the entity over-financialization. The results show that government subsidies significantly inhibit the entity over-financialization. Because the government subsidies could increase the performance of enterprise’s main business and level of the enterprise’s profitability. Subsequently, the enterprise’s arbitrage from cross-industries and the managers’ composition could be decreased. Consequently, government subsidies could reduce the entity over-financialization by the reduce of enterprise’s arbitrage from multi-industries and increase of the managers’ composition which is related to the enterprise’s performance. The results also indicate that the entity financialization is mainly motivated by enterprise arbitrage rather than ‘preventive reserve’ in China. Moreover, the inhibitory effect of government subsidies on the entity over-financialization is only significant in the enterprises with non-state-owned, high-tech, and higher level of demand of innovation. Thus, the government should accurately implement subsidy policies for the enterprises and increase the supports for enterprises with high-tech and higher level of demand of innovation, which could promote economy high-quality development

RegionBLIP: A Unified Multi-modal Pre-training Framework for Holistic and Regional Comprehension

In this work, we investigate extending the comprehension of Multi-modal Large Language Models (MLLMs) to regional objects. To this end, we propose to extract features corresponding to regional objects as soft prompts for LLM, which provides a straightforward and scalable approach and eliminates the need for LLM fine-tuning. To effectively extract regional features from regular image features and irregular point cloud features, we present a novel and unified position-assisted feature extraction module. Furthermore, training an MLLM from scratch is highly time-consuming. Thus, we propose incrementally extending existing pre-trained MLLMs to comprehend more modalities and the regional objects of those modalities. Specifically, we freeze the Q-Former from BLIP-2, an impressive MLLM, and optimize the modality-specific Lora parameters in Q-Former and LLM for each newly introduced modality. The freezing of the Q-Former eliminates the need for extensive pre-training on massive image-text data. The freezed Q-Former pre-trained from massive image-text data is also beneficial for the pre-training on image-region-text data. We name our framework RegionBLIP. We pre-train RegionBLIP on image-region-text, point-cloud-text, and point-cloud-region-text data. Experimental results verify that \Ours{} can preserve the image comprehension capability of BILP-2 and further gain a comprehension of the newly introduced point cloud modality and regional objects. The Data, Code, and Pre-trained models will be available at https://github.com/mightyzau/RegionBLIP