Joint Video and Text Parsing for Understanding Events and Answering Queries

We propose a framework for parsing video and text jointly for understanding events and answering user queries. Our framework produces a parse graph that represents the compositional structures of spatial information (objects and scenes), temporal information (actions and events) and causal information (causalities between events and fluents) in the video and text. The knowledge representation of our framework is based on a spatial-temporal-causal And-Or graph (S/T/C-AOG), which jointly models possible hierarchical compositions of objects, scenes and events as well as their interactions and mutual contexts, and specifies the prior probabilistic distribution of the parse graphs. We present a probabilistic generative model for joint parsing that captures the relations between the input video/text, their corresponding parse graphs and the joint parse graph. Based on the probabilistic model, we propose a joint parsing system consisting of three modules: video parsing, text parsing and joint inference. Video parsing and text parsing produce two parse graphs from the input video and text respectively. The joint inference module produces a joint parse graph by performing matching, deduction and revision on the video and text parse graphs. The proposed framework has the following objectives: Firstly, we aim at deep semantic parsing of video and text that goes beyond the traditional bag-of-words approaches; Secondly, we perform parsing and reasoning across the spatial, temporal and causal dimensions based on the joint S/T/C-AOG representation; Thirdly, we show that deep joint parsing facilitates subsequent applications such as generating narrative text descriptions and answering queries in the forms of who, what, when, where and why. We empirically evaluated our system based on comparison against ground-truth as well as accuracy of query answering and obtained satisfactory results

Augmenting transformers with recursively composed multi-grained representations

We present ReCAT, a recursive composition augmented Transformer that is able to explicitly model hierarchical syntactic structures of raw texts without relying on gold trees during both learning and inference. Existing research along this line restricts data to follow a hierarchical tree structure and thus lacks inter-span communications. To overcome the problem, we propose a novel contextual inside-outside (CIO) layer that learns contextualized representations of spans through bottom-up and top-down passes, where a bottom-up pass forms representations of high-level spans by composing low-level spans, while a top-down pass combines information inside and outside a span. By stacking several CIO layers between the embedding layer and the attention layers in Transformer, the ReCAT model can perform both deep intra-span and deep inter-span interactions, and thus generate multi-grained representations fully contextualized with other spans. Moreover, the CIO layers can be jointly pre-trained with Transformers, making ReCAT enjoy scaling ability, strong performance, and interpretability at the same time. We conduct experiments on various sentence-level and span-level tasks. Evaluation results indicate that ReCAT can significantly outperform vanilla Transformer models on all span-level tasks and baselines that combine recursive networks with Transformers on natural language inference tasks. More interestingly, the hierarchical structures induced by ReCAT exhibit strong consistency with human-annotated syntactic trees, indicating good interpretability brought by the CIO layers.Comment: preprin

Enhancement of Hydroxyapatite Dissolution through Krypton Ion Irradiation

International audienceHydroxyapatite (HA) synthesized by a wet chemical route was subjected to heavy ion irradiation, using 4 MeV Krypton ions (Kr 17+) with ion fluence ranging from 1×10 13 to 1×10 15 ions/cm 2. Glancing incidence X-ray diffraction (GIXRD) results confirmed the phase purity of irradiated HA with a moderate contraction in lattice parameters, and further indicated irradiation-induced structural disorder, evident by a broadening of diffraction peaks. High-resolution transmission electron microscopy (HRTEM) observations indicated that the applied Kr irradiation induced significant damage in the hydroxyapatite lattice. Specifically, cavities were observed with their diameter and density varying with irradiation fluences while a radiation-induced crystalline-to-amorphous transition with increasing ion dose was identified. Raman and X-ray photoelectron spectroscopy (XPS) analysis further indicated the presence of irradiation-induced defects. Compositional analysis of pristine and irradiated materials following immersion in Tris (pH 7.4, 37℃) buffer showed that dissolution in vitro was enhanced by irradiation, reaching a peak for 0.1dpa. We examined the effects of irradiation on the early stages of the mouse osteoblast-like cells (MC3T3-E) response. A cell counting kit-8 assay (CCK-8 test) was carried out to investigate the cytotoxicity of samples, and viable cells can be observed on the irradiated materials.L'hydroxyapatite (HA) synthétisée par voie chimique a été soumise à une irradiation ionique lourde, en utilisant des ions Krypton 4 MeV (Kr 17+) avec une fluence ionique allant de 1 × 10 13 à 1 × 10 15 ions / cm 2. Incidence du regard X- Les résultats de la diffraction des rayons (GIXRD) ont confirmé la pureté de phase de l'AH irradié avec une contraction modérée des paramètres du réseau et ont en outre indiqué un trouble structurel induit par l'irradiation, évident par un élargissement des pics de diffraction. Des observations en microscopie électronique à transmission à haute résolution (HRTEM) ont indiqué que l'irradiation au Kr appliquée a induit des dommages importants dans le réseau d'hydroxyapatite. Plus précisément, des cavités ont été observées avec leur diamètre et leur densité variant avec les fluences d'irradiation tandis qu'une transition cristalline-amorphe induite par le rayonnement avec une dose ionique croissante a été identifiée. L'analyse par spectroscopie photoélectronique Raman et X (XPS) a en outre indiqué la présence de défauts induits par l'irradiation. L'analyse de la composition des matériaux vierges et irradiés après immersion dans du tampon Tris (pH 7,4, 37 ℃) a montré que la dissolution in vitro était améliorée par irradiation, atteignant un pic de 0,1 dpa. Nous avons examiné les effets de l'irradiation sur les premiers stades de la réponse des cellules de type ostéoblaste de souris (MC3T3-E). Un test de comptage de cellules kit 8 (test CCK-8) a été réalisé pour étudier la cytotoxicité des échantillons, et des cellules viables peuvent être observées sur les matériaux irradiés

Characteristics of Local Modulation Beam Propagating through Spatial Filter System

As local defects may significantly harm beam quality and affect safe operation, a systematic analysis of the ability of a spatial filter to alleviate these adverse effects is required. Thus, the evolutional characteristics of a beam modulated by a local defect propagating through a spatial filter system at an image reply plane and a downstream plane are analyzed in detail. Modulation stripes appear at the image reply plane; these are caused by the pinhole cutoff effect. The modulation degree increases with increasing defect size. The maximum intensification factor can reach 3.2 under certain conditions. Thus, the defect size should be restricted to a reasonable size for safe operation with a specified pinhole size. Moreover, a maximal value appears at the downstream plane, and the intensity enhances with increasing defect size. To ensure beam quality, the maximum allowable defect size and angle of the spatial filter should meet special constraints. The maximum allowable defect size is calculated based on practical configuration parameters

Aperçu nanostructural du comportement en dissolution de l'hydroxyapatite dopée au Sr

International audienceIn this study, high-resolution transmission electron microscopy (HRTEM) was employed to characterize the nanostructure of strontium-substituted hydroxyapatite (Sr-HA) and its evolution following in vitro immersion in physiological solutions. HRTEM images showed that the substitution of Sr induced local distortions in the hydroxyapatite (HA) lattice: minor levels of edge dislocations were detected at low doping contents of Sr ions (1 at %); when the Sr content exceeded 10 at%, the density of grain boundaries increased notably and triple junctions were clearly observed. The dissolution of undoped HA was initiated at crystallite surfaces, whereas the dissolution of Sr-HA started around grain boundaries. Acicular nanocrystal reprecipitation was observed on grain surfaces immersed in simulated body fluid (SBF), while not in dilute hydrochloric acid (HCl). These findings suggest appropriate levels of Sr incorporation can introduce imperfections in the crystal structure of apatite and thus enhance its dissolution rate towards enhanced physicochemical performance in biomedical applicationshttps://doi.org/10.1016/j.jeurceramsoc.2018.07.056Dans cette étude, la microscopie électronique à transmission à haute résolution (HRTEM) a été utilisée pour caractériser la nanostructure de l'hydroxyapatite substituée au strontium (Sr-HA) et son évolution après immersion in vitro dans des solutions physiologiques. Les images HRTEM ont montré que la substitution des distorsions locales induites par le Sr dans le réseau d'hydroxyapatite (HA): des niveaux mineurs de dislocations de bords ont été détectés à de faibles teneurs en dopage d'ions Sr (1 at%); lorsque la teneur en Sr dépassait 10% at%, la densité des joints de grains augmentait de manière notable et des triple jonctions étaient clairement observées. La dissolution de l'HA non dopée a été initiée au niveau des surfaces de cristallites, alors que la dissolution de Sr-HA a commencé autour des joints de grains. Une reprécipitation aciculaire de nanocristaux a été observée sur des surfaces de grains immergées dans un fluide corporel simulé (SBF), mais non dans de l'acide chlorhydrique dilué (HCl). Ces découvertes suggèrent que des niveaux appropriés d’incorporation de Sr peuvent introduire des imperfections dans la structure cristalline de l’apatite et ainsi augmenter sa vitesse de dissolution afin d’améliorer les performances physicochimiques dans les applications biomédicales.https://doi.org/10.1016/j.jeurceramsoc.2018.07.05

Analyzing drop coalescence in microfluidic devices with a deep learning generative model

Predicting drop coalescence based on process parameters is crucial for experimental design in chemical engineering. However, predictive models can suffer from the lack of training data and more importantly, the label imbalance problem. In this study, we propose the use of deep learning generative models to tackle this bottleneck by training the predictive models using generated synthetic data. A novel generative model, named double space conditional variational autoencoder (DSCVAE) is developed for labelled tabular data. By introducing label constraints in both the latent and the original space, DSCVAE is capable of generating consistent and realistic samples compared to the standard conditional variational autoencoder (CVAE). Two predictive models, namely random forest and gradient boosting classifiers, are enhanced on synthetic data and their performances are evaluated based on real experimental data. Numerical results show that a considerable improvement in prediction accuracy can be achieved by using synthetic data and the proposed DSCVAE clearly outperforms the standard CVAE. This research clearly provides more insights into handling imbalanced data for classification problems, especially in chemical engineering

Augmented Reality for Enhanced Visualization of MOF Adsorbents

Augmented reality (AR) is an emerging technique used to improve visualization and comprehension of complex 3D materials. This approach has been applied not only in the field of chemistry but also in real estate, physics, mechanical engineering, and many other areas. Here, we demonstrate the workflow for an app-free AR technique for visualization of metal–organic frameworks (MOFs) and other porous materials to investigate their crystal structures, topology, and gas adsorption sites. We think this workflow will serve as an additional tool for computational and experimental scientists working in the field for both research and educational purposes

Analyzing drop coalescence in microfluidic devices with a deep learning generative model

Predicting drop coalescence based on process parameters is crucial for experimental design in chemical engineering. However, predictive models can suffer from the lack of training data and more importantly, the label imbalance problem. In this study, we propose the use of deep learning generative models to tackle this bottleneck by training the predictive models using generated synthetic data. A novel generative model, named double space conditional variational autoencoder (DSCVAE) is developed for labelled tabular data. By introducing label constraints in both the latent and the original space, DSCVAE is capable of generating consistent and realistic samples compared to the standard conditional variational autoencoder (CVAE). Two predictive models, namely random forest and gradient boosting classifiers, are enhanced on synthetic data and their performances are evaluated based on real experimental data. Numerical results show that a considerable improvement in prediction accuracy can be achieved by using synthetic data and the proposed DSCVAE clearly outperforms the standard CVAE. This research clearly provides more insights into handling imbalanced data for classification problems, especially in chemical engineering