Dynamic MDETR: A Dynamic Multimodal Transformer Decoder for Visual Grounding

Multimodal transformer exhibits high capacity and flexibility to align image and text for visual grounding. However, the existing encoder-only grounding framework (e.g., TransVG) suffers from heavy computation due to the self-attention operation with quadratic time complexity. To address this issue, we present a new multimodal transformer architecture, coined as Dynamic Mutilmodal DETR (Dynamic MDETR), by decoupling the whole grounding process into encoding and decoding phases. The key observation is that there exists high spatial redundancy in images. Thus, we devise a new dynamic multimodal transformer decoder by exploiting this sparsity prior to speed up the visual grounding process. Specifically, our dynamic decoder is composed of a 2D adaptive sampling module and a text guided decoding module. The sampling module aims to select these informative patches by predicting the offsets with respect to a reference point, while the decoding module works for extracting the grounded object information by performing cross attention between image features and text features. These two modules are stacked alternatively to gradually bridge the modality gap and iteratively refine the reference point of grounded object, eventually realizing the objective of visual grounding. Extensive experiments on five benchmarks demonstrate that our proposed Dynamic MDETR achieves competitive trade-offs between computation and accuracy. Notably, using only 9% feature points in the decoder, we can reduce ~44% GFLOPs of the multimodal transformer, but still get higher accuracy than the encoder-only counterpart. In addition, to verify its generalization ability and scale up our Dynamic MDETR, we build the first one-stage CLIP empowered visual grounding framework, and achieve the state-of-the-art performance on these benchmarks.Comment: Accepted by IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) in October 202

Accuracies of field CO2–H2O data from open-path eddy-covariance flux systems: assessment based on atmospheric physics and biological environment

Ecosystem CO2–H2O data measured by infrared gas analyzers in open-path eddy-covariance (OPEC) systems have numerous applications, such as estimations of CO2 and H2O fluxes in the atmospheric boundary layer. To assess the applicability of the data for these estimations, data uncertainties from analyzer measurements are needed. The uncertainties are sourced from the analyzers in zero drift, gain drift, cross-sensitivity, and precision variability. These four uncertainty sources are individually specified for analyzer performance, but so far no methodology exists yet to combine these individual sources into a composite uncertainty for the specification of an overall accuracy, which is ultimately needed. Using the methodology for closed-path eddy-covariance systems, this overall accuracy for OPEC systems is determined from all individual uncertainties via an accuracy model and further formulated into CO2 and H2O accuracy equations. Based on atmospheric physics and the biological environment, for EC150 infrared CO2–H2O analyzers, these equations are used to evaluate CO2 accuracy (±1.22 mgCO2 m−3, relatively ±0.19 %) and H2O accuracy (±0.10 gH2O m−3, relatively ±0.18 % in saturated air at 35 ∘C and 101.325 kPa). Both accuracies are applied to conceptual models addressing their roles in uncertainty analyses for CO2 and H2O fluxes. For the high-frequency air temperature derived from H2O density along with sonic temperature and atmospheric pressure, the role of H2O accuracy in its uncertainty is similarly addressed. Among the four uncertainty sources, cross-sensitivity and precision variability are minor, although unavoidable, uncertainties, whereas zero drift and gain drift are major uncertainties but are minimizable via corresponding zero and span procedures during field maintenance. The accuracy equations provide rationales to assess and guide the procedures. For the atmospheric background CO2 concentration, CO2 zero and CO2 span procedures can narrow the CO2 accuracy range by 40 %, from ±1.22 to ±0.72 mgCO2 m−3. In hot and humid weather, H2O gain drift potentially adds more to the H2O measurement uncertainty, which requires more attention. If H2O zero and H2O span procedures can be performed practically from 5 to 35 ∘C, the H2O accuracy can be improved by at least 30 %: from ±0.10 to ±0.07 gH2O m−3. Under freezing conditions, the H2O span procedure is impractical but can be neglected because of its trivial contributions to the overall uncertainty. However, the zero procedure for H2O, along with CO2, is imperative as an operational and efficient option under these conditions to minimize H2O measurement uncertainty.</p

Periplasmic biomineralization for semi-artificial photosynthesis

Semiconductor-based biointerfaces are typically established either on the surface of the plasma membrane or within the cytoplasm. In Gram-negative bacteria, the periplasmic space, characterized by its confinement and the presence of numerous enzymes and peptidoglycans, offers additional opportunities for biomineralization, allowing for nongenetic modulation interfaces. We demonstrate semiconductor nanocluster precipitation containing single- and multiple-metal elements within the periplasm, as observed through various electron- and x-ray-based imaging techniques. The periplasmic semiconductors are metastable and display defect-dominant fluorescent properties. Unexpectedly, the defect-rich (i.e., the low-grade) semiconductor nanoclusters produced in situ can still increase adenosine triphosphate levels and malate production when coupled with photosensitization. We expand the sustainability levels of the biohybrid system to include reducing heavy metals at the primary level, building living bioreactors at the secondary level, and creating semi-artificial photosynthesis at the tertiary level. The biomineralization-enabled periplasmic biohybrids have the potential to serve as defect-tolerant platforms for diverse sustainable applications

USP29-mediated HIF1α stabilization is associated with Sorafenib resistance of hepatocellular carcinoma cells by upregulating glycolysis

Understanding the mechanisms underlying evasive resistance in cancer is an unmet medical need to improve the efficacy of current therapies. In hepatocellular carcinoma (HCC), aberrant expression of hypoxia-inducible factor 1 α (HIF1α) and increased aerobic glycolysis metabolism are drivers of resistance to therapy with the multi-kinase inhibitor Sorafenib. However, it has remained unknown how HIF1α is activated and how its activity and the subsequent induction of aerobic glycolysis promote Sorafenib resistance in HCC. Here, we report the ubiquitin-specific peptidase USP29 as a new regulator of HIF1α and of aerobic glycolysis during the development of Sorafenib resistance in HCC. In particular, we identified USP29 as a critical deubiquitylase (DUB) of HIF1α, which directly deubiquitylates and stabilizes HIF1α and, thus, promotes its transcriptional activity. Among the transcriptional targets of HIF1α is the gene encoding hexokinase 2 (HK2), a key enzyme of the glycolytic pathway. The absence of USP29, and thus of HIF1α transcriptional activity, reduces the levels of aerobic glycolysis and restores sensitivity to Sorafenib in Sorafenib-resistant HCC cells in vitro and in xenograft transplantation mouse models in vivo. Notably, the absence of USP29 and high HK2 expression levels correlate with the response of HCC patients to Sorafenib therapy. Together, the data demonstrate that, as a DUB of HIF1α, USP29 promotes Sorafenib resistance in HCC cells, in parts by upregulating glycolysis, thereby opening new avenues for therapeutically targeting Sorafenib-resistant HCC in patients