Design from Policies: Conservative Test-Time Adaptation for Offline Policy Optimization

In this work, we decouple the iterative bi-level offline RL (value estimation and policy extraction) from the offline training phase, forming a non-iterative bi-level paradigm and avoiding the iterative error propagation over two levels. Specifically, this non-iterative paradigm allows us to conduct inner-level optimization (value estimation) in training, while performing outer-level optimization (policy extraction) in testing. Naturally, such a paradigm raises three core questions that are not fully answered by prior non-iterative offline RL counterparts like reward-conditioned policy: (q1) What information should we transfer from the inner-level to the outer-level? (q2) What should we pay attention to when exploiting the transferred information for safe/confident outer-level optimization? (q3) What are the benefits of concurrently conducting outer-level optimization during testing? Motivated by model-based optimization (MBO), we propose DROP (design from policies), which fully answers the above questions. Specifically, in the inner-level, DROP decomposes offline data into multiple subsets, and learns an MBO score model (a1). To keep safe exploitation to the score model in the outer-level, we explicitly learn a behavior embedding and introduce a conservative regularization (a2). During testing, we show that DROP permits deployment adaptation, enabling an adaptive inference across states (a3). Empirically, we evaluate DROP on various tasks, showing that DROP gains comparable or better performance compared to prior methods.Comment: NeurIPS 202

Targeted Poverty Alleviation and Households’ Livelihood Strategy in a Relation-Based Society: Evidence from Northeast China

Abstract: Although China is experiencing a transition from a relation-based society to a rule-based society, relationships among acquaintances still play an important role in resource allocation, such as the allocation of policy resources. This is particularly true in rural China, where targeted poverty alleviation is prevalent and a relation-based social structure still dominates. However, it is still unknown how relationships affect the livelihood strategy of households in rural China and how poverty alleviation policies plays a role between them. Therefore, this paper embeds poverty alleviation into the relation-based society and explores how households respond to the policy in this specific context. Using grounded theory research method and the sustainable livelihoods approach (SLA) framework, this paper contains in-depth interviews and field observations from three povertystricken villages in Northeast China. The results show that relationships have a significant impact on the households’ livelihood strategy. In other words, the households’ livelihood strategy is embedded in the relation-based society. The types of relationships induce households to choose maintained or developmental type livelihood strategies, while relationships influence how the poverty alleviation policies affect the livelihood strategy. This study is not only an extension of the SLA in the research context, but also provides a significant perspective for enriching the long-term mechanism of targeted poverty alleviation by building a theoretical model of the relationships between a relation-based society, targeted poverty alleviation and the livelihood strategies of households.publishedVersio

Griffon v2: Advancing Multimodal Perception with High-Resolution Scaling and Visual-Language Co-Referring

Large Vision Language Models have achieved fine-grained object perception, but the limitation of image resolution remains a significant obstacle to surpass the performance of task-specific experts in complex and dense scenarios. Such limitation further restricts the model's potential to achieve nuanced visual and language referring in domains such as GUI Agents, Counting and \etc. To address this issue, we introduce a unified high-resolution generalist model, Griffon v2, enabling flexible object referring with visual and textual prompts. To efficiently scaling up image resolution, we design a simple and lightweight down-sampling projector to overcome the input tokens constraint in Large Language Models. This design inherently preserves the complete contexts and fine details, and significantly improves multimodal perception ability especially for small objects. Building upon this, we further equip the model with visual-language co-referring capabilities through a plug-and-play visual tokenizer. It enables user-friendly interaction with flexible target images, free-form texts and even coordinates. Experiments demonstrate that Griffon v2 can localize any objects of interest with visual and textual referring, achieve state-of-the-art performance on REC, phrase grounding, and REG tasks, and outperform expert models in object detection and object counting. Data, codes and models will be released at https://github.com/jefferyZhan/Griffon.Comment: Tech report working in progress. Codes, models and datasets will be released at https://github.com/jefferyZhan/Griffo

VIRT: Improving Representation-based Models for Text Matching through Virtual Interaction

With the booming of pre-trained transformers, remarkable progress has been made on textual pair modeling to support relevant natural language applications. Two lines of approaches are developed for text matching: interaction-based models performing full interactions over the textual pair, and representation-based models encoding the pair independently with siamese encoders. The former achieves compelling performance due to its deep interaction modeling ability, yet with a sacrifice in inference latency. The latter is efficient and widely adopted for practical use, however, suffers from severe performance degradation due to the lack of interactions. Though some prior works attempt to integrate interactive knowledge into representation-based models, considering the computational cost, they only perform late interaction or knowledge transferring at the top layers. Interactive information in the lower layers is still missing, which limits the performance of representation-based solutions. To remedy this, we propose a novel \textit{Virtual} InteRacTion mechanism, termed as VIRT, to enable full and deep interaction modeling in representation-based models without \textit{actual} inference computations. Concretely, VIRT asks representation-based encoders to conduct virtual interactions to mimic the behaviors as interaction-based models do. In addition, the knowledge distilled from interaction-based encoders is taken as supervised signals to promise the effectiveness of virtual interactions. Since virtual interactions only happen at the training stage, VIRT would not increase the inference cost. Furthermore, we design a VIRT-adapted late interaction strategy to fully utilize the learned virtual interactive knowledge

RSG: Fast Learning Adaptive Skills for Quadruped Robots by Skill Graph

Developing robotic intelligent systems that can adapt quickly to unseen wild situations is one of the critical challenges in pursuing autonomous robotics. Although some impressive progress has been made in walking stability and skill learning in the field of legged robots, their ability to fast adaptation is still inferior to that of animals in nature. Animals are born with massive skills needed to survive, and can quickly acquire new ones, by composing fundamental skills with limited experience. Inspired by this, we propose a novel framework, named Robot Skill Graph (RSG) for organizing massive fundamental skills of robots and dexterously reusing them for fast adaptation. Bearing a structure similar to the Knowledge Graph (KG), RSG is composed of massive dynamic behavioral skills instead of static knowledge in KG and enables discovering implicit relations that exist in be-tween of learning context and acquired skills of robots, serving as a starting point for understanding subtle patterns existing in robots' skill learning. Extensive experimental results demonstrate that RSG can provide rational skill inference upon new tasks and environments and enable quadruped robots to adapt to new scenarios and learn new skills rapidly

A β-glucosidase hyper-production Trichoderma reesei mutant reveals a potential role of cel3D in cellulase production

Abstract Background The conversion of cellulose by cellulase to fermentable sugars for biomass-based products such as cellulosic biofuels, biobased fine chemicals and medicines is an environment-friendly and sustainable process, making wastes profitable and bringing economic benefits. Trichoderma reesei is the well-known major workhorse for cellulase production in industry, but the low β-glucosidase activity in T. reesei cellulase leads to inefficiency in biomass degradation and limits its industrial application. Thus, there are ongoing interests in research to develop methods to overcome this insufficiency. Moreover, although β-glucosidases have been demonstrated to influence cellulase production and participate in the regulation of cellulase production, the underlying mechanism remains unclear. Results The T. reesei recombinant strain TRB1 was constructed from T. reesei RUT-C30 by the T-DNA-based mutagenesis. Compared to RUT-C30, TRB1 displays a significant enhancement of extracellular β-glucosidase (BGL1) activity with 17-fold increase, a moderate increase of both the endoglucanase (EG) activity and the exoglucanase (CBH) activity, a minor improvement of the total filter paper activity, and a faster cellulase induction. This superiority of TRB1 over RUT-C30 is independent on carbon sources and improves the saccharification ability of TRB1 cellulase on pretreated corn stover. Furthermore, TRB1 shows better resistance to carbon catabolite repression than RUT-C30. Secretome characterization of TRB1 shows that the amount of CBH, EG and BGL in the supernatant of T. reesei TRB1 was indeed increased along with the enhanced activities of these three enzymes. Surprisingly, qRT-PCR and gene cloning showed that in TRB1 β-glucosidase cel3D was mutated through the random insertion by AMT and was not expressed. Conclusions The T. reesei recombinant strain TRB1 constructed in this study is more desirable for industrial application than the parental strain RUT-C30, showing extracellular β-glucosidase hyper production, high cellulase production within a shorter time and a better resistance to carbon catabolite repression. Disruption of β-glucosidase cel3D in TRB1 was identified, which might contribute to the superiority of TRB1 over RUT-C30 and might play a role in the cellulase production. These results laid a foundation for future investigations to further improve cellulase enzymatic efficiency and reduce cost for T. reesei cellulase production.http://deepblue.lib.umich.edu/bitstream/2027.42/134636/1/12934_2016_Article_550.pd

The Impact of Variational Primary Collaterals on Cerebral Autoregulation

The influence of the anterior and posterior communicating artery (ACoA and PCoA) on dynamic cerebral autoregulation (dCA) is largely unknown. In this study, we aimed to test whether substantial differences in collateral anatomy were associated with differences in dCA in two common types of stenosis according to digital subtraction angiography (DSA): either isolated basal artery and/or bilateral vertebral arteries severe stenosis/occlusion (group 1; group 1A: with bilateral PCoAs; and group 1B: without bilateral PCoAs), or isolated unilateral internal carotid artery severe stenosis/occlusion (group 2; group 2A: without ACoA and with PCoA; group 2B: with ACoA and without PCoAs; and group 2C: without both ACoA and PCoA). The dCA was calculated by transfer function analysis (a mathematical model), and was evaluated in middle cerebral artery (MCA) and/or posterior cerebral artery (PCA). Of a total of 231 non-acute phase ischemic stroke patients who received both dCA assessment and DSA in our lab between 2014 and 2017, 51 patients met inclusion criteria based on the presence or absence of ACoA or PCoA, including 21 patients in the group 1, and 30 patients in the group 2. There were no significant differences in gender, age, and mean blood pressure between group 1A and group 1B, and among group 2A, group 2B, and group 2C. In group 1, the PCA phase difference values (autoregulatory parameter) were significantly higher in the subgroup with patent PCoAs, compared to those without. In group 2, the MCA phase difference values were higher in the subgroup with patent ACoA, compared to those without. This pilot study found that the cross-flow of the ACoA/PCoA to the affected area compensates for compromised dCA in the affected area, which suggests an important role of the ACoA/PCoA in stabilizing cerebral blood flow