Building a Winning Team: Selecting Source Model Ensembles using a Submodular Transferability Estimation Approach

Estimating the transferability of publicly available pretrained models to a target task has assumed an important place for transfer learning tasks in recent years. Existing efforts propose metrics that allow a user to choose one model from a pool of pre-trained models without having to fine-tune each model individually and identify one explicitly. With the growth in the number of available pre-trained models and the popularity of model ensembles, it also becomes essential to study the transferability of multiple-source models for a given target task. The few existing efforts study transferability in such multi-source ensemble settings using just the outputs of the classification layer and neglect possible domain or task mismatch. Moreover, they overlook the most important factor while selecting the source models, viz., the cohesiveness factor between them, which can impact the performance and confidence in the prediction of the ensemble. To address these gaps, we propose a novel Optimal tranSport-based suBmOdular tRaNsferability metric (OSBORN) to estimate the transferability of an ensemble of models to a downstream task. OSBORN collectively accounts for image domain difference, task difference, and cohesiveness of models in the ensemble to provide reliable estimates of transferability. We gauge the performance of OSBORN on both image classification and semantic segmentation tasks. Our setup includes 28 source datasets, 11 target datasets, 5 model architectures, and 2 pre-training methods. We benchmark our method against current state-of-the-art metrics MS-LEEP and E-LEEP, and outperform them consistently using the proposed approach.Comment: To appear at ICCV 202

MADG: Margin-based Adversarial Learning for Domain Generalization

Domain Generalization (DG) techniques have emerged as a popular approach to address the challenges of domain shift in Deep Learning (DL), with the goal of generalizing well to the target domain unseen during the training. In recent years, numerous methods have been proposed to address the DG setting, among which one popular approach is the adversarial learning-based methodology. The main idea behind adversarial DG methods is to learn domain-invariant features by minimizing a discrepancy metric. However, most adversarial DG methods use 0-1 loss based $\mathcal{H}\Delta\mathcal{H}$ divergence metric. In contrast, the margin loss-based discrepancy metric has the following advantages: more informative, tighter, practical, and efficiently optimizable. To mitigate this gap, this work proposes a novel adversarial learning DG algorithm, MADG, motivated by a margin loss-based discrepancy metric. The proposed MADG model learns domain-invariant features across all source domains and uses adversarial training to generalize well to the unseen target domain. We also provide a theoretical analysis of the proposed MADG model based on the unseen target error bound. Specifically, we construct the link between the source and unseen domains in the real-valued hypothesis space and derive the generalization bound using margin loss and Rademacher complexity. We extensively experiment with the MADG model on popular real-world DG datasets, VLCS, PACS, OfficeHome, DomainNet, and TerraIncognita. We evaluate the proposed algorithm on DomainBed's benchmark and observe consistent performance across all the datasets

Downregulation of RWA genes in hybrid aspen affects xylan acetylation and wood saccharification

High acetylation of angiosperm wood hinders its conversion to sugars by glycoside hydrolases, subsequent ethanol fermentation and (hence) its use for biofuel production. We studied the REDUCED WALL ACETYLATION (RWA) gene family of the hardwood model Populus to evaluate its potential for improving saccharification. The family has two clades, AB and CD, containing two genes each. All four genes are expressed in developing wood but only RWA-A and -B are activated by master switches of the secondary cell wall PtNST1 and PtMYB21. Histochemical analysis of promoter:: GUS lines in hybrid aspen (Populus tremula x tremuloides) showed activation of RWA-A and -B promoters in the secondary wall formation zone, while RWA-C and -D promoter activity was diffuse. Ectopic downregulation of either clade reduced wood xylan and xyloglucan acetylation. Suppressing both clades simultaneously using the wood-specific promoter reduced wood acetylation by 25% and decreased acetylation at position 2 of Xylp in the dimethyl sulfoxide-extracted xylan. This did not affect plant growth but decreased xylose and increased glucose contents in the noncellulosic monosaccharide fraction, and increased glucose and xylose yields of wood enzymatic hydrolysis without pretreatment. Both RWA clades regulate wood xylan acetylation in aspen and are promising targets to improve wood saccharification.Peer reviewe