LLMs Meet VLMs: Boost Open Vocabulary Object Detection with Fine-grained Descriptors

Inspired by the outstanding zero-shot capability of vision language models (VLMs) in image classification tasks, open-vocabulary object detection has attracted increasing interest by distilling the broad VLM knowledge into detector training. However, most existing open-vocabulary detectors learn by aligning region embeddings with categorical labels (e.g., bicycle) only, disregarding the capability of VLMs on aligning visual embeddings with fine-grained text description of object parts (e.g., pedals and bells). This paper presents DVDet, a Descriptor-Enhanced Open Vocabulary Detector that introduces conditional context prompts and hierarchical textual descriptors that enable precise region-text alignment as well as open-vocabulary detection training in general. Specifically, the conditional context prompt transforms regional embeddings into image-like representations that can be directly integrated into general open vocabulary detection training. In addition, we introduce large language models as an interactive and implicit knowledge repository which enables iterative mining and refining visually oriented textual descriptors for precise region-text alignment. Extensive experiments over multiple large-scale benchmarks show that DVDet outperforms the state-of-the-art consistently by large margins

Domain Generalization via Balancing Training Difficulty and Model Capability

Domain generalization (DG) aims to learn domain-generalizable models from one or multiple source domains that can perform well in unseen target domains. Despite its recent progress, most existing work suffers from the misalignment between the difficulty level of training samples and the capability of contemporarily trained models, leading to over-fitting or under-fitting in the trained generalization model. We design MoDify, a Momentum Difficulty framework that tackles the misalignment by balancing the seesaw between the model's capability and the samples' difficulties along the training process. MoDify consists of two novel designs that collaborate to fight against the misalignment while learning domain-generalizable models. The first is MoDify-based Data Augmentation which exploits an RGB Shuffle technique to generate difficulty-aware training samples on the fly. The second is MoDify-based Network Optimization which dynamically schedules the training samples for balanced and smooth learning with appropriate difficulty. Without bells and whistles, a simple implementation of MoDify achieves superior performance across multiple benchmarks. In addition, MoDify can complement existing methods as a plug-in, and it is generic and can work for different visual recognition tasks.Comment: 11 pages, 6 figures, Accepted by ICCV 202

Halo: Estimation and Reduction of Hallucinations in Open-Source Weak Large Language Models

Large Language Models (LLMs) have revolutionized Natural Language Processing (NLP). Although convenient for research and practical applications, open-source LLMs with fewer parameters often suffer from severe hallucinations compared to their larger counterparts. This paper focuses on measuring and reducing hallucinations in BLOOM 7B, a representative of such weaker open-source LLMs that are publicly available for research and commercial applications. We introduce HaloCheck, a lightweight BlackBox knowledge-free framework designed to quantify the severity of hallucinations in LLMs. Additionally, we explore techniques like knowledge injection and teacher-student approaches to alleviate hallucinations in low-parameter LLMs. Our experiments effectively demonstrate the reduction of hallucinations in challenging domains for these LLMs

Rician Noise Removal via a Learned Dictionary

This paper proposes a new effective model for denoising images with Rician noise. The sparse representations of images have been shown to be efficient approaches for image processing. Inspired by this, we learn a dictionary from the noisy image and then combine the MAP model with it for Rician noise removal. For solving the proposed model, the primal-dual algorithm is applied and its convergence is studied. The computational results show that the proposed method is promising in restoring images with Rician noise

Enhanced Interfacial Dzyaloshinskii-Moriya Interaction in annealed Pt/Co/MgO structures

The interfacial Dzyaloshinskii-Moriya interaction (iDMI) is attracting great interests for spintronics. An iDMI constant larger than 3 mJ/m^2 is expected to minimize the size of skyrmions and to optimize the DW dynamics. In this study, we experimentally demonstrate an enhanced iDMI in Pt/Co/X/MgO ultra-thin film structures with perpendicular magnetization. The iDMI constants were measured using a field-driven creep regime domain expansion method. The enhancement of iDMI with an atomically thin insertion of Ta and Mg is comprehensively understood with the help of ab-initio calculations. Thermal annealing has been used to crystallize the MgO thin layer for improving tunneling magneto-resistance (TMR), but interestingly it also provides a further increase of the iDMI constant. An increase of the iDMI constant up to 3.3 mJ/m^2 is shown, which could be promising for the scaling down of skyrmion electronics

Detection of TMPRSS2 : ERG fusion gene in circulating prostate cancer cells

Creative Commons Attribution-NonCommercial-Share Alike 3.0 license (CC BY-NC SA)Aim: To investigate the existence of TMPRSS2:ERG fusion gene in circulating tumor cells (CTC) from prostate cancer patients and its potential in monitoring tumor metastasis. Methods: We analyzed the frequency of TMPRSS2: ERG and TMPRSS2:ETV1 transcripts in 27 prostate cancer biopsies from prostatectomies, and TMPRSS2:ERG transcripts in CTC isolated from 15 patients with advanced androgen independent disease using reverse transcription polymerase chain reaction (RT-PCR). Fluorescence in situ hybridization (FISH) was applied to analyze the genomic truncation of ERG, which is the result of TMPRSS2:ERG fusion in 10 of the 15 CTC samples. Results: TMPRSS2: ERG transcripts were found in 44% of our samples, but we did not detect expression of TMPRSS2:ETV1. Using FISH analysis we detected chromosomal rearrangements affecting the ERG gene in 6 of 10 CTC samples, including 1 case with associated TMPRSS2:ERG fusion at the primary site. However, TMPRSS2:ERG transcripts were not detected in any of the 15 CTC samples, including the 10 cases analyzed by FISH. Conclusion: Although further study is required to address the association between TMPRSS2:ERG fusion and prostate cancer metastasis, detection of genomic truncation of the ERG gene by FISH analysis could be useful for monitoring the appearance of CTC and the potential for prostate cancer metastasis.Peer reviewedFinal Published versio

Increased Variation in Body Weight and Food Intake Is Related to Increased Dietary Fat but Not Increased Carbohydrate or Protein in Mice

Funding This study was funded by the National Key R&D Program of China (2019YFA0801900) to JS and the Postdoctoral Innovation Fund (2021) to YW. The original diet exposure experiment was funded by the Chinese Academy of Sciences Strategic Program (XDB13030100). JS was also supported during this work by a PIFI professorial fellowship from CAS and a Wolfson merit award from the UK Royal Society. CORRECTION article Front. Nutr., 21 October 2022 Sec. Nutrition and Metabolism https://doi.org/10.3389/fnut.2022.1049766 Corrigendum: Increased variation in body weight and food intake is related to increased dietary fat but not increased carbohydrate or protein in micePeer reviewedPublisher PD

Review of Associations between Built Environment Characteristics and Severe Acute Respiratory Syndrome Coronavirus 2 Infection Risk.

The coronavirus disease 2019 pandemic has stimulated intensive research interest in its transmission pathways and infection factors, e.g., socioeconomic and demographic characteristics, climatology, baseline health conditions or pre-existing diseases, and government policies. Meanwhile, some empirical studies suggested that built environment attributes may be associated with the transmission mechanism and infection risk of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). However, no review has been conducted to explore the effect of built environment characteristics on the infection risk. This research gap prevents government officials and urban planners from creating effective urban design guidelines to contain SARS-CoV-2 infections and face future pandemic challenges. This review summarizes evidence from 25 empirical studies and provides an overview of the effect of built environment on SARS-CoV-2 infection risk. Virus infection risk was positively associated with the density of commercial facilities, roads, and schools and with public transit accessibility, whereas it was negatively associated with the availability of green spaces. This review recommends several directions for future studies, namely using longitudinal research design and individual-level data, considering multilevel factors and extending to diversified geographic areas