Retrieval-Augmented Multimodal Language Modeling

Recent multimodal models such as DALL-E and CM3 have achieved remarkable progress in text-to-image and image-to-text generation. However, these models store all learned knowledge (e.g., the appearance of the Eiffel Tower) in the model parameters, requiring increasingly larger models and training data to capture more knowledge. To integrate knowledge in a more scalable and modular way, we propose a retrieval-augmented multimodal model, which enables a base multimodal model (generator) to refer to relevant knowledge fetched by a retriever from external memory (e.g., multimodal documents on the web). Specifically, we implement a retriever using the pretrained CLIP model and a generator using the CM3 Transformer architecture, and train this model using the LAION dataset. Our resulting model, named Retrieval-Augmented CM3 (RA-CM3), is the first multimodal model that can retrieve and generate mixtures of text and images. We show that RA-CM3 significantly outperforms baseline multimodal models such as DALL-E and CM3 on both image and caption generation tasks (12 FID and 17 CIDEr improvements on MS-COCO), while requiring much less compute for training (<30% of DALL-E). Moreover, we show that RA-CM3 exhibits novel capabilities such as knowledge-intensive image generation and multimodal in-context learning

Discovery of a peculiar Cepheid-like star towards the northern edge of the Small Magellanic Cloud

For seven years, the EROS-2 project obtained a mass of photometric data on variable stars. We present a peculiar Cepheid-like star, in the direction of the Small Magellanic Cloud, which demonstrates unusual photometric behaviour over a short time interval. We report on data of the photometry acquired by the MARLY telescope and spectroscopy from the EFOSC instrument for this star, called EROS2 J005135-714459(sm0060n13842), which resembles the unusual Cepheid HR 7308. The light curve of our target is analysed using the Analysis of Variance method to determine a pulsational period of 5.5675 days. A fit of time-dependent Fourier coefficients is performed and a search for proper motion is conducted. The light curve exhibits a previously unobserved and spectacular change in both mean magnitude and amplitude, which has no clear theoretical explanation. Our analysis of the spectrum implies a radial velocity of 104 km s$^{-1}$ and a metallicity of -0.4$\pm$0.2 dex. In the direction of right ascension, we measure a proper motion of 17.4$\pm$6.0 mas yr$^{-1}$ using EROS astrometry, which is compatible with data from the NOMAD catalogue. The nature of EROS2 J005135-714459(sm0060n13842) remains unclear. For this star, we may have detected a non-zero proper motion for this star, which would imply that it is a foreground object. Its radial velocity, pulsational characteristics, and photometric data, however, suggest that it is instead a Cepheid-like object located in the SMC. In such a case, it would present a challenge to conventional Cepheid models.Comment: Correction of typos in the abstrac

Blue Hill Ship Building

https://digitalmaine.com/blue_hill_documents/1180/thumbnail.jp

Basic and reductive sulfone-directed ring-opening reactions of difluorinated oxa[2.2.1]bicycloheptanes

Phenylsulfenyl chloride reacts with racemic endo Diels-Alder adduct 4 (DEC ) CONEt2) to afford lactone 8, which can be reduced and protected in a series of high-yielding steps. Key sulfone 10 can be ring opened under strong base conditions to afford vinyl sulfone 11. Attempted desulfonation resulted in the formation of a monofluoroalkene, but a direct desulfonation/eliminative ring opening with strain relief delivered highly functionalized monocyclic species 16

Performance Measurement Versus City Service Satisfaction: Intra-City Variations in Quality?

This article illustrates the usefulness of citizen satisfaction surveys to account for variation in citizen satisfaction with public-service quality within a city's boundaries. Copyright (c) 2005 by the Southwestern Social Science Association.