Exploring Question Decomposition for Zero-Shot VQA

Visual question answering (VQA) has traditionally been treated as a single-step task where each question receives the same amount of effort, unlike natural human question-answering strategies. We explore a question decomposition strategy for VQA to overcome this limitation. We probe the ability of recently developed large vision-language models to use human-written decompositions and produce their own decompositions of visual questions, finding they are capable of learning both tasks from demonstrations alone. However, we show that naive application of model-written decompositions can hurt performance. We introduce a model-driven selective decomposition approach for second-guessing predictions and correcting errors, and validate its effectiveness on eight VQA tasks across three domains, showing consistent improvements in accuracy, including improvements of >20% on medical VQA datasets and boosting the zero-shot performance of BLIP-2 above chance on a VQA reformulation of the challenging Winoground task. Project Site: https://zaidkhan.me/decomposition-0shot-vqa/Comment: NeurIPS 2023 Camera Read

Single-Stream Multi-Level Alignment for Vision-Language Pretraining

Recent progress in large-scale vision-language pre-training has shown the importance of aligning the visual and text modalities for downstream vision-language tasks. Many methods use a dual-stream architecture that fuses visual tokens and language tokens after representation learning, which aligns only at a global level and cannot extract finer-scale semantics. In contrast, we propose a single stream model that aligns the modalities at multiple levels: i) instance level, ii) fine-grained patch level, iii) conceptual semantic level. We achieve this using two novel tasks: symmetric cross-modality reconstruction and a pseudo-labeled key word prediction. In the former part, we mask the input tokens from one of the modalities and use the cross-modal information to reconstruct the masked token, thus improving fine-grained alignment between the two modalities. In the latter part, we parse the caption to select a few key words and feed it together with the momentum encoder pseudo signal to self-supervise the visual encoder, enforcing it to learn rich semantic concepts that are essential for grounding a textual token to an image region. We demonstrate top performance on a set of Vision-Language downstream tasks such as zero-shot/fine-tuned image/text retrieval, referring expression, and VQA. We also demonstrate how the proposed models can align the modalities at multiple levels.Comment: 22 pages, 7 figure

Oral immunization of haemaggulutinin H5 expressed in plant endoplasmic reticulum with adjuvant saponin protects mice against highly pathogenic avian influenza A virus infection

Pandemics in poultry caused by the highly pathogenic avian influenza (HPAI) A virus occur too frequently globally, and there is growing concern about the HPAI A virus due to the possibility of a pandemic among humans. Thus, it is important to develop a vaccine against HPAI suitable for both humans and animals. Various approaches are underway to develop such vaccines. In particular, an edible vaccine would be a convenient way to vaccinate poultry because of the behaviour of the animals. However, an edible vaccine is still not available. In this study, we developed a strategy of effective vaccination of mice by the oral administration of transgenic Arabidopsis plants (HA-TG) expressing haemagglutinin (HA) in the endoplasmic reticulum (ER). Expression of HA in the ER resulted in its high-level accumulation, N-glycosylation, protection from proteolytic degradation and long-term stability. Oral administration of HA-TG with saponin elicited high levels of HA-specific systemic IgG and mucosal IgA responses in mice, which resulted in protection against a lethal influenza virus infection with attenuated inflammatory symptoms. Based on these results, we propose that oral administration of freeze-dried leaf powders from transgenic plants expressing HA in the ER together with saponin is an attractive strategy for vaccination against influenza A virus.X111411Ysciescopu

Small intestinal eosinophils regulate Th17 cells by producing IL-1 receptor antagonist

Eosinophils play proinflammatory roles in helminth infections and allergic diseases. Under steady-state conditions, eosinophils are abundantly found in the small intestinal lamina propria, but their physiological function is largely unexplored. In this study, we found that small intestinal eosinophils down-regulate Th17 cells. Th17 cells in the small intestine were markedly increased in the Delta dblGATA-1 mice lacking eosinophils, and an inverse correlation was observed between the number of eosinophils and that of Th17 cells in the small intestine of wild-type mice. In addition, small intestinal eosinophils suppressed the in vitro differentiation of Th17 cells, as well as IL-17 production by small intestinal CD4(+) T cells. Unlike other small intestinal immune cells or circulating eosinophils, we found that small intestinal eosinophils have a unique ability to constitutively secrete high levels of IL-1 receptor antagonist (IL-1Ra), a natural inhibitor of IL-1 beta. Moreover, small intestinal eosinophils isolated from IL-1Ra-deficient mice failed to suppress Th17 cells. Collectively, our results demonstrate that small intestinal eosinophils play a pivotal role in the maintenance of intestinal homeostasis by regulating Th17 cells via production of IL-1Ra.open111815sciescopu

Venom gland transcriptomes of two elapid snakes (Bungarus multicinctus and Naja atra) and evolution of toxin genes

<p>Abstract</p> <p>Background</p> <p>Kraits (genus <it>Bungarus</it>) and cobras (genus <it>Naja</it>) are two representative toxic genera of elapids in the old world. Although they are closely related genera and both of their venoms are very toxic, the compositions of their venoms are very different. To unveil their detailed venoms and their evolutionary patterns, we constructed venom gland cDNA libraries and genomic bacterial artificial chromosome (BAC) libraries for <it>Bungarus multicinctus </it>and <it>Naja atra</it>, respectively. We sequenced about 1500 cDNA clones for each of the venom cDNA libraries and screened BAC libraries of the two snakes by blot analysis using four kinds of toxin probes; <it>i.e</it>., three-finger toxin (3FTx), phospholipase A2 (PLA2), kunitz-type protease inhibitor (Kunitz), and natriuretic peptide (NP).</p> <p>Results</p> <p>In total, 1092 valid expressed sequences tags (ESTs) for <it>B. multicinctus </it>and 1166 ESTs for <it>N. atra </it>were generated. About 70% of these ESTs can be annotated as snake toxin transcripts. 3FTx (64.5%) and <it>β </it>bungarotoxin (25.1%) comprise the main toxin classes in <it>B. multicinctus</it>, while 3FTx (95.8%) is the dominant toxin in <it>N. atra</it>. We also observed several less abundant venom families in <it>B. multicinctus </it>and <it>N. atra</it>, such as PLA2, C-type lectins, and Kunitz. Peculiarly a cluster of NP precursors with tandem NPs was detected in <it>B. multicinctus</it>. A total of 71 positive toxin BAC clones in <it>B. multicinctus </it>and <it>N. atra </it>were identified using four kinds of toxin probes (3FTx, PLA2, Kunitz, and NP), among which 39 3FTx-postive BACs were sequenced to reveal gene structures of 3FTx toxin genes.</p> <p>Conclusions</p> <p>Based on the toxin ESTs and 3FTx gene sequences, the major components of <it>B. multicinctus </it>venom transcriptome are neurotoxins, including long chain alpha neurotoxins (<it>α</it>-ntx) and the recently originated <it>β </it>bungarotoxin, whereas the <it>N. atra </it>venom transcriptome mainly contains 3FTxs with cytotoxicity and neurotoxicity (short chain <it>α</it>-ntx). The data also revealed that tandem duplications contributed the most to the expansion of toxin multigene families. Analysis of nonsynonymous to synonymous nucleotide substitution rate ratios (<it>dN</it>/<it>dS</it>) indicates that not only multigene toxin families but also other less abundant toxins might have been under rapid diversifying evolution.</p

Evaluation of CXCL9 and CXCL10 as circulating biomarkers of human cardiac allograft rejection

BACKGROUND: Cardiac allograft rejection remains a significant clinical problem in the early phase after heart transplantation and requires frequent surveillance with endomyocardial biopsy. However, this is an invasive procedure, which is unpleasant for the patient and carries a certain risk. Therefore, a sensitive non-invasive biomarker of acute rejection would be desirable. METHODS: Endomyocardial tissue samples and serum were obtained in connection with clinical biopsies from twenty consecutive heart transplant patients followed for six months. A rejection episode was observed in 14 patients (11 men and 3 women) and biopsies obtained before, during and after the episode were identified. Endomyocardial RNA, from three patients, matching these three points in time were analysed with DNA microarray. Genes showing up-regulation during rejection followed by normalization after the rejection episode were evaluated further with real-time RT-PCR. Finally, ELISA was performed to investigate whether change in gene-regulation during graft rejection was reflected in altered concentrations of the encoded protein in serum. RESULTS: Three potential cardiac allograft rejection biomarker genes, chemokine (C-X-C motif) ligand 9 (CXCL9), chemokine (C-X-C motif) ligand 10 (CXCL10) and Natriuretic peptide precursor A (NPPA), from the DNA microarray analysis were selected for further evaluation. CXCL9 was significantly upregulated during rejection (p < 0.05) and CXCL10 displayed a similar pattern without reaching statistical significance. Serum levels of CXCL9 and CXCL10 were measured by ELISA in samples from 10 patients before, during and after cardiac rejection. There were no changes in CXCL9 and CXCL10 serum concentrations during cardiac rejection. Both chemokines displayed large individual variations in the selected samples, but the serum levels between the two chemokines correlated (p < 0.001). CONCLUSION: We conclude, that despite a distinct up-regulation of CXCL9 mRNA in human hearts during cardiac allograft rejection, this was not reflected in the serum levels of the encoded protein. Thus, in contrast to previous suggestions, serum CXCL9 does not appear to be a promising serum biomarker for cardiac allograft rejection

Observation of an Exotic S=+1S=+1 Baryon in Exclusive Photoproduction from the Deuteron

In an exclusive measurement of the reaction $\gamma d \to K^+ K^- p n$, a narrow peak that can be attributed to an exotic baryon with strangeness $S=+1$ is seen in the $K^+n$ invariant mass spectrum. The peak is at $1.542\pm 0.005$ GeV/c$^2$ with a measured width of 0.021 GeV/c$^2$ FWHM, which is largely determined by experimental mass resolution. The statistical significance of the peak is $5.2 \pm 0.6 \sigma$. The mass and width of the observed peak are consistent with recent reports of a narrow $S=+1$ baryon by other experimental groups.Comment: 5 pages, 5 figure

Measurement of Beam-Spin Asymmetries for Deep Inelastic π+\pi^+ Electroproduction

We report the first evidence for a non-zero beam-spin azimuthal asymmetry in the electroproduction of positive pions in the deep-inelastic region. Data have been obtained using a polarized electron beam of 4.3 GeV with the CLAS detector at the Thomas Jefferson National Accelerator Facility (JLab). The amplitude of the $\sin\phi$ modulation increases with the momentum of the pion relative to the virtual photon, $z$, with an average amplitude of $0.038 \pm 0.005 \pm 0.003$ for $0.5 < z < 0.8$ range.Comment: 5 pages, RevTEX4, 3 figures, 2 table

Measurement of the Polarized Structure Function σLT′\sigma_{LT^\prime} for p(e⃗,e′p)πop(\vec{e},e'p)\pi^o in the Δ(1232)\Delta(1232) Resonance Region

The polarized longitudinal-transverse structure function $\sigma_{LT^\prime}$ has been measured in the $\Delta(1232)$ resonance region at $Q^2=0.40$ and 0.65 GeV$^2$. Data for the $p(\vec e,e'p)\pi^o$ reaction were taken at Jefferson Lab with the CEBAF Large Acceptance Spectrometer (CLAS) using longitudinally polarized electrons at an energy of 1.515 GeV. For the first time a complete angular distribution was measured, permitting the separation of different non-resonant amplitudes using a partial wave analysis. Comparison with previous beam asymmetry measurements at MAMI indicate a deviation from the predicted $Q^2$ dependence of $\sigma_{LT^{\prime}}$ using recent phenomenological models.Comment: 5 pages, LaTex, 4 eps figures: to be published in PRC/Rapid Communications. Version 2 has revised Q^2 analysi

Two-Nucleon Momentum Distributions Measured in 3He(e,e'pp)n

We have measured the 3He(e,e'pp)n reaction at 2.2 GeV over a wide kinematic range. The kinetic energy distribution for `fast' nucleons (p > 250 MeV/c) peaks where two nucleons each have 20% or less, and the third nucleon has most of the transferred energy. These fast pp and pn pairs are back-to-back with little momentum along the three-momentum transfer, indicating that they are spectators. Experimental and theoretical evidence indicates that we have measured distorted two-nucleon momentum distributions by striking the third nucleon and detecting the spectator correlated pair.Comment: 6 pages, 5 figures, submitted to PR