SMAN : Stacked Multi-Modal Attention Network for cross-modal image-text retrieval

This article focuses on tackling the task of the cross-modal image-text retrieval which has been an interdisciplinary topic in both computer vision and natural language processing communities. Existing global representation alignment-based methods fail to pinpoint the semantically meaningful portion of images and texts, while the local representation alignment schemes suffer from the huge computational burden for aggregating the similarity of visual fragments and textual words exhaustively. In this article, we propose a stacked multimodal attention network (SMAN) that makes use of the stacked multimodal attention mechanism to exploit the fine-grained interdependencies between image and text, thereby mapping the aggregation of attentive fragments into a common space for measuring cross-modal similarity. Specifically, we sequentially employ intramodal information and multimodal information as guidance to perform multiple-step attention reasoning so that the fine-grained correlation between image and text can be modeled. As a consequence, we are capable of discovering the semantically meaningful visual regions or words in a sentence which contributes to measuring the cross-modal similarity in a more precise manner. Moreover, we present a novel bidirectional ranking loss that enforces the distance among pairwise multimodal instances to be closer. Doing so allows us to make full use of pairwise supervised information to preserve the manifold structure of heterogeneous pairwise data. Extensive experiments on two benchmark datasets demonstrate that our SMAN consistently yields competitive performance compared to state-of-the-art methods

Blockwise perturbation theory for nearly uncoupled Markov chains and its application

AbstractLet P be the transition matrix of a nearly uncoupled Markov chain. The states can be grouped into aggregates such that P has the block form P=(Pij)i,j=1k, where Pii is square and Pij is small for i≠j. Let πT be the stationary distribution partitioned conformally as πT=(π1T,…,πkT). In this paper we bound the relative error in each aggregate distribution πiT caused by small relative perturbations in Pij. The error bounds demonstrate that nearly uncoupled Markov chains usually lead to well-conditioned problems in the sense of blockwise relative error. As an application, we show that with appropriate stopping criteria, iterative aggregation/disaggregation algorithms will achieve such structured backward errors and compute each aggregate distribution with high relative accuracy

Mechanistic Investigation Toward A Microkinetic Model For Decarboxylation Of Gamma Valerolactone Over Silica Alumina

γ-valerolactone (GVL) ring opening- and decarboxylation rates were measured over amorphous silica alumina (SiO2/Al2O3) catalyst in the gas phase, using a down configuration fixed bed reactor, operating under anhydrous and differential conditions. By varying temperature, space time and GVL partial pressure, measured rates were leveraged to determine kinetic parameters (apparent activation barriers and pre-exponential factors) for the ring opening and decarboxylation step, as well as thermodynamic parameters (enthalpy- and entropy of reaction) for the ring opening step. The experimentally measured parameters were applied for the development of a microkinetic model that quantitatively describes the overall kinetics involved in the decarboxylation of GVL over SiO2/Al2O3. The microkinetic model predicts that the adsorption of GVL onto the surface of the catalyst and the subsequent ring opening step are two main elementary steps driving the kinetics of the process. In addition, the equilibrium constant of the adsorption step and apparent forward rate constant for the ring opening step were predicted as key parameters associated with the decarboxylation event. By comparing model predicted apparent forward rate constants for all relevant elementary steps involved in the mechanism of GVL decarboxylation, the ring opening step was identified as the slowest step; hence it’s probably the rate determining step. In previous studies, decarboxylation of GVL over aluminosilicates with comparable apparent activation barriers and deprotonation energies revealed markedly different turn over frequencies; the microkinetic model introduced suggests that the difference in turn over frequencies is an artifact of local structural effects in the vicinity of the acid site. Although the model describes the kinetics involved, testing more aluminosilicates with different pore sizes will provide more information about the relative importance of the surface adsorption and ring opening steps

Dirhodium-Induced Intramolecular C-H Insertion On Diazosulfones/sulfonates And Its Synthetic Applications

This works explores the use of rhodium-catalyzed intramolecular C-H insertion on diazosulfones and sulfonates as a tool to simplify the synthesis of organic molecules. Dirhodium-induced intramolecular C-H insertion on diazo carbonyl compounds is a relatively well-studied reaction. It occurs with an overwhelming bias for the formation of five-membered carbocycles or heterocycles, with the order of reactivity of C-H bonds being methine \u3e methylene \u3e\u3e methyl. This intrinsic preference can be overridden by dissimilar factors such as substrate conformation, insertion site electronic effects and the reactivity of the dirhodium catalyst. This study has disclosed that when diazo- sulfones and sulfonates are employed, dirhodium-induced intramolecular C-H insertion preferentially forms six-membered heterocycles. Apparently, the difference in bond lengths and bond angles around the SO2 fragment incorporated into the newly forming ring allows more distant C-H bonds to be targeted for insertion. The preference for formation of six- versus five-membered heterocyclic sulfones, however, is tenuous in nature, and is dependent on the substrate structure and the nature of the dirhodium catalyst used. Dirhodium catalysts bearing electron poor ligands shift the preference back to formation of five-membered rings, presumably due to forming more reactive carbenoid intermediates. The presence of methyl substituents on the carbon atom adjacent to the sulfone fragment also favors the formation of five-membered sulfones, likely due to the steric compression of the bond angle. Rh2(S-pttl)4 [dirhodium(II)tetrakis(N-phthaloyl-(S)-tert-leucinate)] catalyst has been identified as the current best lead to the development of chiral catalysts for this reaction. Selectivity of 3:1 (50% ee) was achieved with its use, and 20:1 (91% de) when combined with the use of a (-)-menthol-based chiral auxiliary. The synthetic utility of these novel six-membered heterocycles has also been explored. To which end, the enantioselective synthesis of bakuchiol has been achieved in 14 linear steps by employing intramolecular C-H insertion on a diazo sulfonate to install the quaternary center in its structure. The alkylation of these substrates with active electrophiles under relatively mild conditions and with excellent yields has been attained with high diastereoselectivity. Novel methodologies have been originated to transform the obtained δ-sultones into γ-butyrolactones and δ-valerolactones using TBHP/t-BuOK and SmI2/DMPU, respectively. The conversion of carbethoxythiane-1,1-dioxides to cyclic α-sulfonyl oximes has also been demonstrated under relatively mild conditions (NaH, isoamyl nitrite, rt, 8 h). Currently, the versatility of these new transformations is being pursued in the synthesis of the natural products quebrachamine and mesembrine

Pixelated Semantic Colorization

While many image colorization algorithms have recently shown the capability of producing plausible color versions from gray-scale photographs, they still suffer from limited semantic understanding. To address this shortcoming, we propose to exploit pixelated object semantics to guide image colorization. The rationale is that human beings perceive and distinguish colors based on the semantic categories of objects. Starting from an autoregressive model, we generate image color distributions, from which diverse colored results are sampled. We propose two ways to incorporate object semantics into the colorization model: through a pixelated semantic embedding and a pixelated semantic generator. Specifically, the proposed convolutional neural network includes two branches. One branch learns what the object is, while the other branch learns the object colors. The network jointly optimizes a color embedding loss, a semantic segmentation loss and a color generation loss, in an end-to-end fashion. Experiments on PASCAL VOC2012 and COCO-stuff reveal that our network, when trained with semantic segmentation labels, produces more realistic and finer results compared to the colorization state-of-the-art

Deep attentive video summarization with distribution consistency learning

This article studies supervised video summarization by formulating it into a sequence-to-sequence learning framework, in which the input and output are sequences of original video frames and their predicted importance scores, respectively. Two critical issues are addressed in this article: short-term contextual attention insufficiency and distribution inconsistency. The former lies in the insufficiency of capturing the short-term contextual attention information within the video sequence itself since the existing approaches focus a lot on the long-term encoder-decoder attention. The latter refers to the distributions of predicted importance score sequence and the ground-truth sequence is inconsistent, which may lead to a suboptimal solution. To better mitigate the first issue, we incorporate a self-attention mechanism in the encoder to highlight the important keyframes in a short-term context. The proposed approach alongside the encoder-decoder attention constitutes our deep attentive models for video summarization. For the second one, we propose a distribution consistency learning method by employing a simple yet effective regularization loss term, which seeks a consistent distribution for the two sequences. Our final approach is dubbed as Attentive and Distribution consistent video Summarization (ADSum). Extensive experiments on benchmark data sets demonstrate the superiority of the proposed ADSum approach against state-of-the-art approaches

Improving Far-UV CD Prediction With The Dipole Interaction Model

Among the physical methods available for studying protein structures, CD stands out as a property that is easy to measure yet remarkably difficult to predict theoretically. The basic principle underlying the CD of polypeptides and proteins is reasonably understood: the mixing of electronic transitions of monomer groups in the context of a chiral environment of helices gives rise to transitions that are both electronically and magnetically allowed. Despite knowledge on the fundamentals of CD, accurate theoretical prediction is still a major challenge. A better theoretical understanding of protein CD would facilitate a fuller interpretation of protein CD experiments. Extensive theoretical studies have been conducted with some success to predict the CD spectra of polypeptides and proteins. One of such theoretical studies has been with the dipole interaction model, pioneered by J. Applequist. In this model, atoms and chromophores are considered to be point dipole oscillators that interact through mutually induced dipole moments in the presences of an electric field. The dipole interaction model has been assembled into a package (DInaMo) and used to successfully predict the far-UV CD of peptides and proteins. The major limitation of the method has been the neglect of the n-π* transition, and having to deal with a number of parameters, since no single parameter at the moment succeeds with all the different classes of proteins. Herein, in an attempt to improve the far-UV protein CD prediction capability of DInaMo, a number of issues are addressed: (1) Will the predicted CD be improved if mean polarizability values are used? (2) Since excluding methyl hydrogens on the amino acid residues have been successful, what happens if methylene and methylidyne hydrogens are also excluded? (3) How will the predicted spectrum differ upon addition of the n-π* amide transition? To answer these questions, a cyclic peptide (cyclo-(Gly-Pro-Gyl-D-Ala-Pro)) rebuilt with idealized bond angles and lengths, and a set of α-helical proteins obtained from the Protein Data Bank are energy minimized to adjust bond lengths and bond angles. The energy-minimized structures are then used to generate CD spectra with DInaMo. Mean polarizability parameters for methyl, methylene and methylidyne groups are developed and implemented on the cyclic peptide and protein. In addition, the effects of excluding methyl, methylene, and methylidyne hydrogens are investigated. Lastly, the n-π* transition is included in the predictions of α-helical proteins and peptides CD. Calculations using new mean polarizability parameters remove the need for different π-π* transition parameters and improve the CD results in lower RMSDs and better spectra morphology. Excluding more hydrogens improve results with larger protein. In addition, the n-π* transition parameters yield normal modes in the correct region and sign for this transition

Mechanistic Investigation Toward A Microkinetic Model For Decarboxylation Of Gamma Valerolactone Over Silica Alumina

γ-valerolactone (GVL) ring opening- and decarboxylation rates were measured over amorphous silica alumina (SiO2/Al2O3) catalyst in the gas phase, using a down configuration fixed bed reactor, operating under anhydrous and differential conditions. By varying temperature, space time and GVL partial pressure, measured rates were leveraged to determine kinetic parameters (apparent activation barriers and pre-exponential factors) for the ring opening and decarboxylation step, as well as thermodynamic parameters (enthalpy- and entropy of reaction) for the ring opening step. The experimentally measured parameters were applied for the development of a microkinetic model that quantitatively describes the overall kinetics involved in the decarboxylation of GVL over SiO2/Al2O3. The microkinetic model predicts that the adsorption of GVL onto the surface of the catalyst and the subsequent ring opening step are two main elementary steps driving the kinetics of the process. In addition, the equilibrium constant of the adsorption step and apparent forward rate constant for the ring opening step were predicted as key parameters associated with the decarboxylation event. By comparing model predicted apparent forward rate constants for all relevant elementary steps involved in the mechanism of GVL decarboxylation, the ring opening step was identified as the slowest step; hence it’s probably the rate determining step. In previous studies, decarboxylation of GVL over aluminosilicates with comparable apparent activation barriers and deprotonation energies revealed markedly different turn over frequencies; the microkinetic model introduced suggests that the difference in turn over frequencies is an artifact of local structural effects in the vicinity of the acid site. Although the model describes the kinetics involved, testing more aluminosilicates with different pore sizes will provide more information about the relative importance of the surface adsorption and ring opening steps