280 research outputs found
Text-Only Training for Image Captioning using Noise-Injected CLIP
We consider the task of image-captioning using only the CLIP model and
additional text data at training time, and no additional captioned images. Our
approach relies on the fact that CLIP is trained to make visual and textual
embeddings similar. Therefore, we only need to learn how to translate CLIP
textual embeddings back into text, and we can learn how to do this by learning
a decoder for the frozen CLIP text encoder using only text. We argue that this
intuition is "almost correct" because of a gap between the embedding spaces,
and propose to rectify this via noise injection during training. We demonstrate
the effectiveness of our approach by showing SOTA zero-shot image captioning
across four benchmarks, including style transfer. Code, data, and models are
available on GitHub.Comment: Will be presented at EMNLP 2022. GitHub:
https://github.com/DavidHuji/CapDe
Prompt-to-Prompt Image Editing with Cross Attention Control
Recent large-scale text-driven synthesis models have attracted much attention
thanks to their remarkable capabilities of generating highly diverse images
that follow given text prompts. Such text-based synthesis methods are
particularly appealing to humans who are used to verbally describe their
intent. Therefore, it is only natural to extend the text-driven image synthesis
to text-driven image editing. Editing is challenging for these generative
models, since an innate property of an editing technique is to preserve most of
the original image, while in the text-based models, even a small modification
of the text prompt often leads to a completely different outcome.
State-of-the-art methods mitigate this by requiring the users to provide a
spatial mask to localize the edit, hence, ignoring the original structure and
content within the masked region. In this paper, we pursue an intuitive
prompt-to-prompt editing framework, where the edits are controlled by text
only. To this end, we analyze a text-conditioned model in depth and observe
that the cross-attention layers are the key to controlling the relation between
the spatial layout of the image to each word in the prompt. With this
observation, we present several applications which monitor the image synthesis
by editing the textual prompt only. This includes localized editing by
replacing a word, global editing by adding a specification, and even delicately
controlling the extent to which a word is reflected in the image. We present
our results over diverse images and prompts, demonstrating high-quality
synthesis and fidelity to the edited prompts
Signature-Tagged Mutagenesis in a Chicken Infection Model Leads to the Identification of a Novel Avian Pathogenic Escherichia coli Fimbrial Adhesin
The extraintestinal pathogen, avian pathogenic E. coli (APEC), known to cause systemic infections in chickens, is responsible for large economic losses in the poultry industry worldwide. In order to identify genes involved in the early essential stages of pathogenesis, namely adhesion and colonization, Signature-tagged mutagenesis (STM) was applied to a previously established lung colonization model of infection by generating and screening a total of 1,800 mutants of an APEC strain IMT5155 (O2:K1:H5; Sequence type complex 95). The study led to the identification of new genes of interest, including two adhesins, one of which coded for a novel APEC fimbrial adhesin (Yqi) not described for its role in APEC pathogenesis to date. Its gene product has been temporarily designated ExPEC Adhesin I (EA/I) until the adhesin-specific receptor is identified. Deletion of the ExPEC adhesin I gene resulted in reduced colonization ability by APEC strain IMT5155 both in vitro and in vivo. Furthermore, complementation of the adhesin gene restored its ability to colonize epithelial cells in vitro. The ExPEC adhesin I protein was successfully expressed in vitro. Electron microscopy of an afimbriate strain E. coli AAEC189 over-expressed with the putative EA/I gene cluster revealed short fimbrial-like appendages protruding out of the bacterial outer membrane. We observed that this adhesin coding gene yqi is prevalent among extraintestinal pathogenic E. coli (ExPEC) isolates, including APEC (54.4%), uropathogenic E. coli (UPEC) (65.9%) and newborn meningitic E. coli (NMEC) (60.0%), and absent in all of the 153 intestinal pathogenic E. coli strains tested, thereby validating the designation of the adhesin as ExPEC Adhesin I. In addition, prevalence of EA/I was most frequently associated with the B2 group of the EcoR classification and ST95 complex of the multi locus sequence typing (MLST) scheme, with evidence of a positive selection within this highly pathogenic complex. This is the first report of the newly identified and functionally characterized ExPEC adhesin I and its significant role during APEC infection in chickens
Changing Hydrozoan Bauplans by Silencing Hox-Like Genes
Regulatory genes of the Antp class have been a major factor for the invention and radiation of animal bauplans. One of the most diverse animal phyla are the Cnidaria, which are close to the root of metazoan life and which often appear in two distinct generations and a remarkable variety of body forms. Hox-like genes have been known to be involved in axial patterning in the Cnidaria and have been suspected to play roles in the genetic control of many of the observed bauplan changes. Unfortunately RNAi mediated gene silencing studies have not been satisfactory for marine invertebrate organisms thus far. No direct evidence supporting Hox-like gene induced bauplan changes in cnidarians have been documented as of yet. Herein, we report a protocol for RNAi transfection of marine invertebrates and demonstrate that knock downs of Hox-like genes in Cnidaria create substantial bauplan alterations, including the formation of multiple oral poles (“heads”) by Cnox-2 and Cnox-3 inhibition, deformation of the main body axis by Cnox-5 inhibition and duplication of tentacles by Cnox-1 inhibition. All phenotypes observed in the course of the RNAi studies were identical to those obtained by morpholino antisense oligo experiments and are reminiscent of macroevolutionary bauplan changes. The reported protocol will allow routine RNAi studies in marine invertebrates to be established
Modulation of COUP-TF Expression in a Cnidarian by Ectopic Wnt Signalling and Allorecognition
COUP transcription factors are required for the regulation of gene expression underlying development, differentiation, and homeostasis. They have an evolutionarily conserved function, being a known marker for neurogenesis from cnidarians to vertebrates. A homologue of this gene was shown previously to be a neuronal and nematocyte differentiation marker in Hydra. However, COUP-TFs had not previously been studied in a colonial cnidarian.We cloned a COUP-TF homologue from the colonial marine cnidarian Hydractinia echinata. Expression of the gene was analysed during normal development, allorecognition events and ectopic Wnt activation, using in situ hybridisation and quantitative PCR. During normal Hydractinia development, the gene was first expressed in post-gastrula stages. It was undetectable in larvae, and its mRNA was present again in putative differentiating neurons and nematocytes in post-metamorphic stages. Global activation of canonical Wnt signalling in adult animals resulted in the upregulation of COUP-TF. We also monitored a strong COUP-TF upregulation in stolons undergoing allogeneic interactions. COUP-TF mRNA was most concentrated in the tissues that contacted allogeneic, non-self tissues, and decreased in a gradient away from the contact area. Interestingly, the gene was transiently upregulated during initial contact of self stolons, but dissipated rapidly following self recognition, while in non-self contacts high expression levels were maintained.We conclude that COUP-TF is likely involved in neuronal/nematocyte differentiation in a variety of contexts. This has now been shown to include allorecognition, where COUP-TF is thought to have been co-opted to mediate allorejection by recruiting stinging cells that are the effectors of cytotoxic rejection of allogeneic tissue. Our findings that Wnt activation upregulates COUP-TF expression suggests that Wnts' role in neuronal differentiation could be mediated through COUP-TF
Indirect effects of species interactions on habitat provisioning
Species that shelter in a biogenic habitat can influence their refugia and, in turn, play an essential role in shaping local patterns of biodiversity. Here we explore a positive feedback loop between the provisioning rate of habitat-forming branching corals and their associated fishes and show how interactions between two groups of fish—the planktivorous damselfish and predatory hawkfish—altered the feedback. A field experiment confirmed that skeletal growth of branching coral (genus Pocillopora) increased substantially with increasing numbers (biomass) of resident fishes, likely because they greatly increased the interstitial concentrations of nutrients. Because there is a positive relationship between colony size and number (biomass) of associated fishes (primarily damselfishes in the Family Pomacentridae), a structure–function feedback loop exists in which increasing numbers of damselfish enhance coral growth and larger corals host greater abundances (and species richness) of fish. However, interactions between damselfishes and arc-eye hawkfish, Paracirrhites arcatus, a largely solitary resident, can disrupt this positive feedback loop. Field surveys revealed a marked pattern of fish occupancy related to coral size: Pocillopora colonies of sufficient size to host fish (>40 cm circumference) had either groups of damselfish or an arc-eye hawkfish; only larger colonies (>75 cm) were occupied by both the damselfish and hawkfish. Subsequent short- and long-term experiments revealed that on intermediate-sized Pocillopora colonies, arc-eye hawkfish prevented the establishment of damselfish by suppressing their recruitment. The demographic consequences to the host coral were substantial; in a 1-year-long experiment, intermediate-size Pocillopora occupied by hawkfish grew at half the rate of corals that hosted groups of damselfish. These findings indicate that: (1) species which occupy a biogenic habitat can enhance the provisioning rate of their habitat; (2) such positive feedbacks between community structure and ecosystem function can be disrupted by a strong interactor; (3) even substantial consequences on ecosystem processes that arise can be difficult to discern
A Multi-Lab Test of the Facial Feedback Hypothesis by the Many Smiles Collaboration
Following theories of emotional embodiment, the facial feedback hypothesis suggests that individuals’ subjective experiences of emotion are influenced by their facial expressions. However, evidence for this hypothesis has been mixed. We thus formed a global adversarial collaboration and carried out a preregistered, multicentre study designed to specify and test the conditions that should most reliably produce facial feedback effects. Data from n = 3,878 participants spanning 19 countries indicated that a facial mimicry and voluntary facial action task could both amplify and initiate feelings of happiness. However, evidence of facial feedback effects was less conclusive when facial feedback was manipulated unobtrusively via a pen-in-mouth task
A multi-lab test of the facial feedback hypothesis by the Many Smiles Collaboration
Following theories of emotional embodiment, the facial feedback hypothesis suggests that individuals' subjective experiences of emotion are influenced by their facial expressions. However, evidence for this hypothesis has been mixed. We thus formed a global adversarial collaboration and carried out a preregistered, multicentre study designed to specify and test the conditions that should most reliably produce facial feedback effects. Data from n = 3,878 participants spanning 19 countries indicated that a facial mimicry and voluntary facial action task could both amplify and initiate feelings of happiness. However, evidence of facial feedback effects was less conclusive when facial feedback was manipulated unobtrusively via a pen-in-mouth task
- …