Mitigating Both Covariate and Conditional Shift for Domain Generalization

Domain generalization (DG) aims to learn a model on several source domains, hoping that the model can generalize well to unseen target domains. The distribution shift between domains contains the covariate shift and conditional shift, both of which the model must be able to handle for better generalizability. In this paper, a novel DG method is proposed to deal with the distribution shift via Visual Alignment and Uncertainty-guided belief Ensemble (VAUE). Specifically, for the covariate shift, a visual alignment module is designed to align the distribution of image style to a common empirical Gaussian distribution so that the covariate shift can be eliminated in the visual space. For the conditional shift, we adopt an uncertainty-guided belief ensemble strategy based on the subjective logic and Dempster-Shafer theory. The conditional distribution given a test sample is estimated by the dynamic combination of that of source domains. Comprehensive experiments are conducted to demonstrate the superior performance of the proposed method on four widely used datasets, i.e., Office-Home, VLCS, TerraIncognita, and PACS

Simultaneous saccharification and cofermentation of lignocellulosic residues from commercial furfural production and corn kernels using different nutrient media

<p>Abstract</p> <p>Background</p> <p>As the supply of starch grain and sugar cane, currently the main feedstocks for bioethanol production, become limited, lignocelluloses will be sought as alternative materials for bioethanol production. Production of cellulosic ethanol is still cost-inefficient because of the low final ethanol concentration and the addition of nutrients. We report the use of simultaneous saccharification and cofermentation (SSCF) of lignocellulosic residues from commercial furfural production (furfural residue, FR) and corn kernels to compare different nutritional media. The final ethanol concentration, yield, number of live yeast cells, and yeast-cell death ratio were investigated to evaluate the effectiveness of integrating cellulosic and starch ethanol.</p> <p>Results</p> <p>Both the ethanol yield and number of live yeast cells increased with increasing corn-kernel concentration, whereas the yeast-cell death ratio decreased in SSCF of FR and corn kernels. An ethanol concentration of 73.1 g/L at 120 h, which corresponded to a 101.1% ethanol yield based on FR cellulose and corn starch, was obtained in SSCF of 7.5% FR and 14.5% corn kernels with mineral-salt medium. SSCF could simultaneously convert cellulose into ethanol from both corn kernels and FR, and SSCF ethanol yield was similar between the organic and mineral-salt media.</p> <p>Conclusions</p> <p>Starch ethanol promotes cellulosic ethanol by providing important nutrients for fermentative organisms, and in turn cellulosic ethanol promotes starch ethanol by providing cellulosic enzymes that convert the cellulosic polysaccharides in starch materials into additional ethanol. It is feasible to produce ethanol in SSCF of FR and corn kernels with mineral-salt medium. It would be cost-efficient to produce ethanol in SSCF of high concentrations of water-insoluble solids of lignocellulosic materials and corn kernels. Compared with prehydrolysis and fed-batch strategy using lignocellulosic materials, addition of starch hydrolysates to cellulosic ethanol production is a more suitable method to improve the final ethanol concentration.</p

Constrained Maximum Cross-Domain Likelihood for Domain Generalization

As a recent noticeable topic, domain generalization aims to learn a generalizable model on multiple source domains, which is expected to perform well on unseen test domains. Great efforts have been made to learn domain-invariant features by aligning distributions across domains. However, existing works are often designed based on some relaxed conditions which are generally hard to satisfy and fail to realize the desired joint distribution alignment. In this paper, we propose a novel domain generalization method, which originates from an intuitive idea that a domain-invariant classifier can be learned by minimizing the KL-divergence between posterior distributions from different domains. To enhance the generalizability of the learned classifier, we formalize the optimization objective as an expectation computed on the ground-truth marginal distribution. Nevertheless, it also presents two obvious deficiencies, one of which is the side-effect of entropy increase in KL-divergence and the other is the unavailability of ground-truth marginal distributions. For the former, we introduce a term named maximum in-domain likelihood to maintain the discrimination of the learned domain-invariant representation space. For the latter, we approximate the ground-truth marginal distribution with source domains under a reasonable convex hull assumption. Finally, a Constrained Maximum Cross-domain Likelihood (CMCL) optimization problem is deduced, by solving which the joint distributions are naturally aligned. An alternating optimization strategy is carefully designed to approximately solve this optimization problem. Extensive experiments on four standard benchmark datasets, i.e., Digits-DG, PACS, Office-Home and miniDomainNet, highlight the superior performance of our method

Low-rank Attention Side-Tuning for Parameter-Efficient Fine-Tuning

In finetuning a large pretrained model to downstream tasks, parameter-efficient fine-tuning (PEFT) methods can effectively finetune pretrained models with few trainable parameters, but suffer from high GPU memory consumption and slow training speed. Because learnable parameters from these methods are entangled with the pretrained model, gradients related to the frozen pretrained model's parameters have to be computed and stored during finetuning. We propose Low-rank Attention Side-Tuning (LAST), which disentangles the trainable module from the pretrained model by freezing not only parameters but also outputs of the pretrained network. LAST trains a side-network composed of only low-rank self-attention modules. By viewing the pretrained model as a frozen feature extractor, the side-network takes intermediate output from the pretrained model and focus on learning task-specific knowledge. We also show that LAST can be highly parallel across multiple optimization objectives, making it very efficient in downstream task adaptation, for example, in finding optimal hyperparameters. LAST outperforms previous state-of-the-art methods on VTAB-1K and other visual adaptation tasks with roughly only 30\% of GPU memory footprint and 60\% of training time compared to existing PEFT methods, but achieves significantly higher accuracy

A Novel Algorithm for Detecting Protein Complexes with the Breadth First Search

Most biological processes are carried out by protein complexes. A substantial number of false positives of the protein-protein interaction (PPI) data can compromise the utility of the datasets for complexes reconstruction. In order to reduce the impact of such discrepancies, a number of data integration and affinity scoring schemes have been devised. The methods encode the reliabilities (confidence) of physical interactions between pairs of proteins. The challenge now is to identify novel and meaningful protein complexes fromthe weighted PPI network. To address this problem, a novel protein complex mining algorithm ClusterBFS (Cluster with Breadth-First Search) is proposed. Based on the weighted density, ClusterBFS detects protein complexes of the weighted network by the breadth first search algorithm, which originates from a given seed protein used as starting-point. The experimental results show that ClusterBFS performs significantly better than the other computational approaches in terms of the identification of protein complexes

Inter-Instance Similarity Modeling for Contrastive Learning

The existing contrastive learning methods widely adopt one-hot instance discrimination as pretext task for self-supervised learning, which inevitably neglects rich inter-instance similarities among natural images, then leading to potential representation degeneration. In this paper, we propose a novel image mix method, PatchMix, for contrastive learning in Vision Transformer (ViT), to model inter-instance similarities among images. Following the nature of ViT, we randomly mix multiple images from mini-batch in patch level to construct mixed image patch sequences for ViT. Compared to the existing sample mix methods, our PatchMix can flexibly and efficiently mix more than two images and simulate more complicated similarity relations among natural images. In this manner, our contrastive framework can significantly reduce the gap between contrastive objective and ground truth in reality. Experimental results demonstrate that our proposed method significantly outperforms the previous state-of-the-art on both ImageNet-1K and CIFAR datasets, e.g., 3.0% linear accuracy improvement on ImageNet-1K and 8.7% kNN accuracy improvement on CIFAR100. Moreover, our method achieves the leading transfer performance on downstream tasks, object detection and instance segmentation on COCO dataset. The code is available at https://github.com/visresearch/patchmi