Chatting Makes Perfect -- Chat-based Image Retrieval

Chats emerge as an effective user-friendly approach for information retrieval, and are successfully employed in many domains, such as customer service, healthcare, and finance. However, existing image retrieval approaches typically address the case of a single query-to-image round, and the use of chats for image retrieval has been mostly overlooked. In this work, we introduce ChatIR: a chat-based image retrieval system that engages in a conversation with the user to elicit information, in addition to an initial query, in order to clarify the user's search intent. Motivated by the capabilities of today's foundation models, we leverage Large Language Models to generate follow-up questions to an initial image description. These questions form a dialog with the user in order to retrieve the desired image from a large corpus. In this study, we explore the capabilities of such a system tested on a large dataset and reveal that engaging in a dialog yields significant gains in image retrieval. We start by building an evaluation pipeline from an existing manually generated dataset and explore different modules and training strategies for ChatIR. Our comparison includes strong baselines derived from related applications trained with Reinforcement Learning. Our system is capable of retrieving the target image from a pool of 50K images with over 78% success rate after 5 dialogue rounds, compared to 75% when questions are asked by humans, and 64% for a single shot text-to-image retrieval. Extensive evaluations reveal the strong capabilities and examine the limitations of CharIR under different settings

MELON: NeRF with Unposed Images Using Equivalence Class Estimation

Neural radiance fields enable novel-view synthesis and scene reconstruction with photorealistic quality from a few images, but require known and accurate camera poses. Conventional pose estimation algorithms fail on smooth or self-similar scenes, while methods performing inverse rendering from unposed views require a rough initialization of the camera orientations. The main difficulty of pose estimation lies in real-life objects being almost invariant under certain transformations, making the photometric distance between rendered views non-convex with respect to the camera parameters. Using an equivalence relation that matches the distribution of local minima in camera space, we reduce this space to its quotient set, in which pose estimation becomes a more convex problem. Using a neural-network to regularize pose estimation, we demonstrate that our method - MELON - can reconstruct a neural radiance field from unposed images with state-of-the-art accuracy while requiring ten times fewer views than adversarial approaches

Profiling molecular and behavioral circadian rhythms in the non-symbiotic sea anemone Nematostella vectensis

© The Author(s), 2015. This article is distributed under the terms of the Creative Commons Attribution License. The definitive version was published in Scientific Reports 5 (2015): 11418, doi:10.1038/srep11418.Endogenous circadian clocks are poorly understood within early-diverging animal lineages. We have characterized circadian behavioral patterns and identified potential components of the circadian clock in the starlet sea anemone, Nematostella vectensis: a model cnidarian which lacks algal symbionts. Using automatic video tracking we showed that Nematostella exhibits rhythmic circadian locomotor activity, which is persistent in constant dark, shifted or disrupted by external dark/light cues and maintained the same rate at two different temperatures. This activity was inhibited by a casein kinase 1δ/ε inhibitor, suggesting a role for CK1 homologue(s) in Nematostella clock. Using high-throughput sequencing we profiled Nematostella transcriptomes over 48 hours under a light-dark cycle. We identified 180 Nematostella diurnally-oscillated transcripts and compared them with previously established databases of adult and larvae of the symbiotic coral Acropora millepora, revealing both shared homologues and unique rhythmic genes. Taken together, this study further establishes Nematostella as a non-symbiotic model organism to study circadian rhythms and increases our understanding about the fundamental elements of circadian regulation and their evolution within the Metazoa.This work was supported by the Israel-US Binational Science Foundation to OL and AMT (Award 2011187). Additional support was provided by the WHOI Early Career Scientist Award to AMT

The VIL gene CRAWLING ELEPHANT controls maturation and differentiation in tomato via polycomb silencing

VERNALIZATION INSENSITIVE 3-LIKE (VIL) proteins are PHD-finger proteins that recruit the repressor complex Polycomb Repressive Complex 2 (PRC2) to the promoters of target genes. Most known VIL targets are flowering repressor genes. Here, we show that the tomato VIL gene CRAWLING ELEPHANT (CREL) promotes differentiation throughout plant development by facilitating the trimethylation of Histone H3 on lysine 27 (H3K27me3). We identified the crel mutant in a screen for suppressors of the simple-leaf phenotype of entire (e), a mutant in the AUX/IAA gene ENTIRE/SlIAA9, involved in compound-leaf development in tomato. crel mutants have increased leaf complexity, and suppress the ectopic blade growth of e mutants. In addition, crel mutants are late flowering, and have delayed and aber rant stem, root and flower development. Consistent with a role for CREL in recruiting PRC2, crel mutants show drastically reduced H3K27me3 enrichment at approximately half of the 14,789 sites enriched in wild-type plants, along with upregulation of many underlying genes. Interestingly, this reduction in H3K27me3 across the genome in crel is also associated with gains in H3K27me3 at a smaller number of sites that normally have modest levels of the mark in wild-type plants, suggesting that PRC2 activity is no longer limiting in the absence of CREL. Our results uncover a wide role for CREL in plant and organ differentiation in tomato and suggest that CREL is required for targeting PRC2 activity to, and thus silencing, a spe cific subset of polycomb targets

Electronic Structure Calculation by First Principles for Strongly Correlated Electron Systems

Recent trends of ab initio studies and progress in methodologies for electronic structure calculations of strongly correlated electron systems are discussed. The interest for developing efficient methods is motivated by recent discoveries and characterizations of strongly correlated electron materials and by requirements for understanding mechanisms of intriguing phenomena beyond a single-particle picture. A three-stage scheme is developed as renormalized multi-scale solvers (RMS) utilizing the hierarchical electronic structure in the energy space. It provides us with an ab initio downfolding of the global band structure into low-energy effective models followed by low-energy solvers for the models. The RMS method is illustrated with examples of several materials. In particular, we overview cases such as dynamics of semiconductors, transition metals and its compounds including iron-based superconductors and perovskite oxides, as well as organic conductors of kappa-ET type.Comment: 44 pages including 38 figures, to appear in J. Phys. Soc. Jpn. as an invited review pape

Altered somatic hypermutation patterns in COVID-19 patients classifies disease severity

IntroductionThe success of the human body in fighting SARS-CoV2 infection relies on lymphocytes and their antigen receptors. Identifying and characterizing clinically relevant receptors is of utmost importance.MethodsWe report here the application of a machine learning approach, utilizing B cell receptor repertoire sequencing data from severely and mildly infected individuals with SARS-CoV2 compared with uninfected controls.ResultsIn contrast to previous studies, our approach successfully stratifies non-infected from infected individuals, as well as disease level of severity. The features that drive this classification are based on somatic hypermutation patterns, and point to alterations in the somatic hypermutation process in COVID-19 patients.DiscussionThese features may be used to build and adapt therapeutic strategies to COVID-19, in particular to quantitatively assess potential diagnostic and therapeutic antibodies. These results constitute a proof of concept for future epidemiological challenges

The neuropeptide NMU amplifies ILC2-driven allergic lung inflammation

Type 2 innate lymphoid cells (ILC2s) both contribute to mucosal homeostasis and initiate pathologic inflammation in allergic asthma. However, the signals that direct ILC2s to promote homeostasis versus inflammation are unclear. To identify such molecular cues, we profiled mouse lung-resident ILCs using single-cell RNA sequencing at steady state and after in vivo stimulation with the alarmin cytokines IL-25 and IL-33. ILC2s were transcriptionally heterogeneous after activation, with subpopulations distinguished by expression of proliferative, homeostatic and effector genes. The neuropeptide receptor Nmur1 was preferentially expressed by ILC2s at steady state and after IL-25 stimulation. Neuromedin U (NMU), the ligand of NMUR1, activated ILC2s in vitro, and in vivo co-administration of NMU with IL-25 strongly amplified allergic inflammation. Loss of NMU-NMUR1 signalling reduced ILC2 frequency and effector function, and altered transcriptional programs following allergen challenge in vivo. Thus, NMUR1 signalling promotes inflammatory ILC2 responses, highlighting the importance of neuro-immune crosstalk in allergic inflammation at mucosal surfaces

COVID-19 symptoms at hospital admission vary with age and sex: results from the ISARIC prospective multinational observational study

Background: The ISARIC prospective multinational observational study is the largest cohort of hospitalized patients with COVID-19. We present relationships of age, sex, and nationality to presenting symptoms. Methods: International, prospective observational study of 60 109 hospitalized symptomatic patients with laboratory-confirmed COVID-19 recruited from 43 countries between 30 January and 3 August 2020. Logistic regression was performed to evaluate relationships of age and sex to published COVID-19 case definitions and the most commonly reported symptoms. Results: ‘Typical’ symptoms of fever (69%), cough (68%) and shortness of breath (66%) were the most commonly reported. 92% of patients experienced at least one of these. Prevalence of typical symptoms was greatest in 30- to 60-year-olds (respectively 80, 79, 69%; at least one 95%). They were reported less frequently in children (≤ 18 years: 69, 48, 23; 85%), older adults (≥ 70 years: 61, 62, 65; 90%), and women (66, 66, 64; 90%; vs. men 71, 70, 67; 93%, each P < 0.001). The most common atypical presentations under 60 years of age were nausea and vomiting and abdominal pain, and over 60 years was confusion. Regression models showed significant differences in symptoms with sex, age and country. Interpretation: This international collaboration has allowed us to report reliable symptom data from the largest cohort of patients admitted to hospital with COVID-19. Adults over 60 and children admitted to hospital with COVID-19 are less likely to present with typical symptoms. Nausea and vomiting are common atypical presentations under 30 years. Confusion is a frequent atypical presentation of COVID-19 in adults over 60 years. Women are less likely to experience typical symptoms than men

Fast neurotransmission related genes are expressed in non nervous endoderm in the sea anemone Nematostella vectensis.

Cnidarian nervous systems utilize chemical transmission to transfer signals through synapses and neurons. To date, ample evidence has been accumulated for the participation of neuropeptides, primarily RFamides, in neurotransmission. Yet, it is still not clear if this is the case for the classical fast neurotransmitters such as GABA, Glutamate, Acetylcholine and Monoamines. A large repertoire of cnidarian Fast Neurotransmitter related Genes (FNGs) has been recently identified in the genome of the sea anemone, Nematostella vectensis. In order to test whether FNGs are localized in cnidarian neurons, we characterized the expression patterns of eight Nematostella genes that are closely or distantly related to human central and peripheral nervous systems genes, in adult Nematostella and compared them to the RFamide localization. Our results show common expression patterns for all tested genes, in a single endodermal cell layer. These expressions did not correspond with the RFamide expressing nerve cell network. Following these results we suggest that the tested Nematostella genes may not be directly involved in vertebrate-like fast neurotransmission