CHAMPAGNE: Learning Real-world Conversation from Large-Scale Web Videos

Visual information is central to conversation: body gestures and physical behaviour, for example, contribute to meaning that transcends words alone. To date, however, most neural conversational models are limited to just text. We introduce CHAMPAGNE, a generative model of conversations that can account for visual contexts. To train CHAMPAGNE, we collect and release YTD-18M, a large-scale corpus of 18M video-based dialogues. YTD-18M is constructed from web videos: crucial to our data collection pipeline is a pretrained language model that converts error-prone automatic transcripts to a cleaner dialogue format while maintaining meaning. Human evaluation reveals that YTD-18M is more sensible and specific than prior resources (MMDialog, 1M dialogues), while maintaining visual-groundedness. Experiments demonstrate that 1) CHAMPAGNE learns to conduct conversation from YTD-18M; and 2) when fine-tuned, it achieves state-of-the-art results on four vision-language tasks focused on real-world conversations. We release data, models, and code.Comment: ICCV 2023, Project page: https://seungjuhan.me/champagn

Reading Books is Great, But Not if You Are Driving! Visually Grounded Reasoning about Defeasible Commonsense Norms

Commonsense norms are defeasible by context: reading books is usually great, but not when driving a car. While contexts can be explicitly described in language, in embodied scenarios, contexts are often provided visually. This type of visually grounded reasoning about defeasible commonsense norms is generally easy for humans, but (as we show) poses a challenge for machines, as it necessitates both visual understanding and reasoning about commonsense norms. We construct a new multimodal benchmark for studying visual-grounded commonsense norms: NORMLENS. NORMLENS consists of 10K human judgments accompanied by free-form explanations covering 2K multimodal situations, and serves as a probe to address two questions: (1) to what extent can models align with average human judgment? and (2) how well can models explain their predicted judgments? We find that state-of-the-art model judgments and explanations are not well-aligned with human annotation. Additionally, we present a new approach to better align models with humans by distilling social commonsense knowledge from large language models. The data and code are released at https://seungjuhan.me/normlens.Comment: Published as a conference paper at EMNLP 2023 (long

Structure of Poly(γ\gamma-Benzyl-L-Glutamate) Monolayers at the Gas–Water Interface: A Brewster Angle Microscopy and X-Ray Scattering Study

This paper reports Brewster angle microscopy (BAM), x-ray specular reflectivity (XR), grazing incidence diffraction (GID) and off-specular diffuse scattering (XOSDS) measurements of Langmuir monolayers formed on water by both mono- and polydisperse samples of $\alpha$-helical poly($\gamma$-benzyl L-glutamate) (PBLG) as a function of area/monomer A. The microscopic behavior does not exhibit any discernible effects due to differing dispersity. At low surface densities (A > $\sim$ 21  Å$^2$/monomer, surface pressure $\Pi$ = 0), BAM images reveal partial surface coverage by solidlike monolayer islands. GID measurements show an interhelix peak corresponding to a local parallel alignment of rodlike PBLG molecules, indicating their tendency to aggregate laterally without external pressure. Compression to A < 21  Å$^2$/monomer first leads to full and uniform surface coverage by the monolayer, followed by a steep rise in $\Pi$ that is accompanied by a decrease in the interhelix distance. Further compression results in a plateau of constant $\Pi$ in the $\Pi$-A isotherm ( $\sim$ 11.5 < A < $\sim$ 18.5  Å$^2$/monomer, $\Pi$ $\sim$ 9 dyn/cm), which has previously been attributed to a first-order monolayer–bilayer transition. The interfacial electron density profiles determined by the XR measurements on both sides of the coexistence plateau provide direct evidence for this transition. On the basis of x-ray scattering results, the film on the high-density side of the plateau is shown to consist of a newly formed incomplete and incommensurate second layer that sits on top of and has lower average density than a homogeneous first layer. GID measurements indicate that the second layer can be characterized by larger interhelix d-spacing than the first layer, while XOSDS measurements on the bilayer suggest that the second layer is microscopically inhomogeneous. For both mono- and bilayers, the analysis of observed GID peak widths indicates that the extent of lateral positional correlations between parallel PBLG rods ranges from a few to no more than $\sim$15 interhelix distances, implying short-range order.Engineering and Applied Science

Internal Segregation and Side Chain Ordering in Hairy-Rod Polypeptide Monolayers at the Gas/Water Interface: An X-Ray Scattering Study

We report studies of the structure and packing of Langmuir monolayers (LMs) of polypeptide poly($\gamma$-4-(n-hexadecyloxy)benzyl $\alpha$,L-glutamate) (C16–O–PBLG) on the surface of water. The molecule is a “hairy rod” and consists of side attachments of hexadecyloxy chains (–O–C16) to the rigid rod-like core made up of $\alpha$-helical poly($\gamma$-benzyl L-glutamate) (PBLG). Measurements include surface pressure ($\Pi$) versus area/monomer (A) isotherms, x-ray specular reflectivity (XR), and grazing incidence diffraction (GID). In contrast to the LM of bare PBLG on water, which undergoes a monolayer/bilayer transition with increasing $\Pi$, monolayers of C16–O–PBLG remain stable up to the highest densities. On the basis of XR and GID results, the structure of the C16–O–PBLG monolayer is characterized by the following main features. First, hydrophobicity causes the –O–C16 chains to segregate towards the film/gas interface and away from water and the PBLG cores, which sit parallel to and near the water/film interface. Since the attachment position of some of the side chains is at the core/water interface, the segregation forces these chains into the space between neighboring core rods. Compression associated with increasing $\Pi$ thickens the film but the internally segregated structure is maintained for all $\Pi$ (i.e., >$\sim$30 dyne/cm). Second, the C16–O–PBLG rods form domains in which the rods are aligned parallel to each other and to the interface. The correlation length for the interhelix positional order of the rods is short and typically comparable to or less than the length of the rods. With increasing $\Pi$ the spacing d between nearest-neighbor rods decreases linearly with A at high $\Pi$, indicating a direct correspondence between the macroscopic compressibility and the microscopic interhelix compressibility. Third, as $\Pi$ increases past $\sim$5 dyne/cm, the local packing of tethered –O–C16 chains displays the same herringbone (HB) order that is common for high-density bulk and monolayer phases of alkyl chains. Various features of the observed GID peaks also imply that the HB order of –O–C16 chains is oriented with respect to the helical axes of aligned PBLG cores. We propose that the HB order is established initially by one-dimensionally confined chains between aligned rods at low Π and grows laterally with compression.Engineering and Applied Science

Internal segregation and side chain ordering in hairy-rod polypeptide monolayers at the gas/water interface: An x-ray scattering study

We report studies of the structure and packing of Langmuirmonolayers (LMs) of polypeptide poly(γ-4-(n-hexadecyloxy)benzyl α,L-glutamate) (C16–O–PBLG) on the surface of water. The molecule is a “hairy rod” and consists of side attachments of hexadecyloxy chains (–O–C16) to the rigid rod-like core made up of α-helical poly(γ-benzyl L-glutamate) (PBLG). Measurements include surface pressure (Π) versus area/monomer (A) isotherms, x-ray specular reflectivity (XR), and grazing incidence diffraction(GID). In contrast to the LM of bare PBLG on water, which undergoes a monolayer/bilayer transition with increasing Π, monolayers of C16–O–PBLG remain stable up to the highest densities. On the basis of XR and GID results, the structure of the C16–O–PBLG monolayer is characterized by the following main features. First, hydrophobicity causes the –O–C16 chains to segregate towards the film/gas interface and away from water and the PBLG cores, which sit parallel to and near the water/film interface. Since the attachment position of some of the side chains is at the core/water interface, the segregation forces these chains into the space between neighboring core rods. Compression associated with increasing Π thickens the film but the internally segregated structure is maintained for all Π (i.e., >∼30 dyne/cm). Second, the C16–O–PBLG rods form domains in which the rods are aligned parallel to each other and to the interface. The correlation length for the interhelix positional order of the rods is short and typically comparable to or less than the length of the rods. With increasing Π the spacing d between nearest-neighbor rods decreases linearly with A at high Π, indicating a direct correspondence between the macroscopic compressibility and the microscopic interhelix compressibility. Third, as Π increases past ∼5 dyne/cm, the local packing of tethered –O–C16 chains displays the same herringbone (HB) order that is common for high-density bulk and monolayer phases of alkyl chains. Various features of the observed GID peaks also imply that the HB order of –O–C16 chains is oriented with respect to the helical axes of aligned PBLG cores. We propose that the HB order is established initially by one-dimensionally confined chains between aligned rods at low Π and grows laterally with compression

A Comparative Study of Co-residence for the Elderly and their Adult Children between Urban and Rural Area: Empirical Evidences from Korea and the US

Population aging is expected to have a major impact on many aspects of social and economic life in the twenty-first century. The present study intends to investigate whether there are differences to take co-residence for the elderly with their children between Korea and the US. According to a theoretical argument from the mainstream about the living arrangements for the elderly, the theory argues that the living arrangements of the aged have resulted primarily from an increase in the resources of the aged, which has enabled increasing numbers of the elderly to afford independent living. The opposite argues that the decline of the multi-generational family occurred mainly because of increasing opportunities for the young and declining parental control over their children. Adopting the census data of the 1990s and the 2000s from the both countries, the present study found that both theories can be applied to the living arrangements for the elderly in Korea and the U.S. The present study concludes by suggesting some policy implications and future studies

Smectic ordering of rod-like polymers owing to monodispersity of chain length: Synthesis and characterization of benzyl and (4-hexadecyloxy)benzyl esters of monodisperse derivatives of poly(alpha,L-glutamate)

Two types of monodisperse rod-like polymers, poly($\gamma$-benzyl $\alpha$,L-glutamate) (PBLG) and poly($\gamma$-4-(hexadecyloxy)benzyl $\alpha$,L-glutamate) with backbone sequences of have been prepared by chemical modification of monodisperse poly($\alpha$,L-glutamic acid) (PLGA) that was synthesized by recombinant DNA biosynthesis techniques. The monodisperse PLGA () was produced in Escherichia coli as a fusion to mouse dihydrofolate reductase and purified to homogeneity by metal affinity chromatography, CNBr digestion and ion-exchange chromatography.\rm GluAsp(Glu\sb{17}Asp)\sb{x}GluGlu\quad x = 3, 4, 5, 6\eqno {\bf1} The monodisperse PBLG derived from showed smectic ordering both in solution and in films as characterized by polarized light optical microscopy and X-ray diffraction. The layer spacings of smectic order were nearly identical to the expected length of the rods, given the axial rise per residue of 1.5 A for the $\alpha$-helix. X-ray diffraction patterns of magnetically oriented films were consistent with the supramolecular structure in which helical rods are arranged in layers with their helical axes approximately perpendicular to the smectic layers. Transmission electron microscopy and electron diffraction on the PBLG films revealed a banded morphology with an approximately 120 nm period which provides strong evidence for helical rotation of the director field as in cholesteric order. Detailed examination of the relative orientation of the banding in the morphology image and the reflections in the electron diffraction pattern leads to the conclusion that the structure of the unoriented smectic PBLG is that of a smectic A*. The monodisperse poly($\gamma$-4-(hexadecyloxy)benzyl $\alpha$,L-glutamate)s exhibit strong melting transitions at around 40\sp\circC; however no ordered melt was observed due in part to the reduced aspect ratio brought about by the long alkyl side groups