Apprentissage multimodal de représentation de mots à l'aide de contexte visuel

International audienceReprésenter la sémantique d'un mot est un défi ma-jeur pour pouvoir traiter automatiquement le langage. Jusqu'à présent, une grande partie des méthodes déter-minent le sens d'un mot via ses contextes dans un corpus de texte. Plus récemment, certains auteurs se sont intéressés à l'apparence visuelle d'un objet pour amé-liorer la représentation sémantique du mot correspon-dant. Cependant, ces travaux ignorent l'environnement et le contexte visuel dans lequel l'objet apparaît. Dans cet article, nous proposons d'apprendre la représenta-tion des mots en bénéficiant de la complémentarité des modalités texte et image par la prise en compte simul-tanée des contextes textuels et visuels des mots. Nous explorons plusieurs choix de modélisation de contexte visuel, et présentons une méthode jointe qui intègre le contexte visuel dans un modèle skip-gram multimodal. Enfin, l'apport de ces représentations dans des tâches d'analyse sémantiques est évaluée sur plusieurs jeux de données. Cet article est une traduction de [ZPSG18]

Ultrafast proton-coupled isomerization in the phototransformation of phytochrome

The biological function of phytochromes is triggered by an ultrafast photoisomerization of the tetrapyrrole chromophore biliverdin between two rings denoted C and D. The mechanism by which this process induces extended structural changes of the protein is unclear. Here we report ultrafast proton-coupled photoisomerization upon excitation of the parent state (Pfr) of bacteriophytochrome Agp2. Transient deprotonation of the chromophore’s pyrrole ring D or ring C into a hydrogen-bonded water cluster, revealed by a broad continuum infrared band, is triggered by electronic excitation, coherent oscillations and the sudden electric-field change in the excited state. Subsequently, a dominant fraction of the excited population relaxes back to the Pfr state, while ~35% follows the forward reaction to the photoproduct. A combination of quantum mechanics/molecular mechanics calculations and ultrafast visible and infrared spectroscopies demonstrates how proton-coupled dynamics in the excited state of Pfr leads to a restructured hydrogen-bond environment of early Lumi-F, which is interpreted as a trigger for downstream protein structural changes

Light- and temperature-dependent dynamics of chromophore and protein structural changes in bathy phytochrome Agp2

Bacterial phytochromes are sensoric photoreceptors that transform light absorbed by the photosensor core module (PCM) to protein structural changes that eventually lead to the activation of the enzymatic output module. The underlying photoinduced reaction cascade in the PCM starts with the isomerization of the tetrapyrrole chromophore, followed by conformational relaxations, proton transfer steps, and a secondary structure transition of a peptide segment (tongue) that is essential for communicating the signal to the output module. In this work, we employed various static and time-resolved IR and resonance Raman spectroscopic techniques to study the structural and reaction dynamics of the Meta-F intermediate of both the PCM and the full-length (PCM and output module) variant of the bathy phytochrome Agp2 from Agrobacterium fabrum. In both cases, this intermediate represents a branching point of the phototransformation, since it opens an unproductive reaction channel back to the initial state and a productive pathway to the final active state, including the functional protein structural changes. It is shown that the functional quantum yield, i.e. the events of tongue refolding per absorbed photons, is lower by a factor of ca. two than the quantum yield of the primary photochemical process. However, the kinetic data derived from the spectroscopic experiments imply an increased formation of the final active state upon increasing photon flux or elevated temperature under photostationary conditions. Accordingly, the branching mechanism does not only account for the phytochrome's function as a light intensity sensor but may also modulate its temperature sensitivity.DFG, 221545957, SFB 1078: Protonation Dynamics in Protein FunctionTU Berlin, Open-Access-Mittel – 202

FTIR-spektroskopische Untersuchungen am Phytochrom Agp2

In der vorliegenden Arbeit wurde der lichtinduzierte Reaktionszyklus des bakteriellen Phytochroms Agp2 aus Agrobacterium tumefaciens mit FTIR‑ und UV‑Vis‑Spektroskopie untersucht. Der Photorezeptor besteht aus einem photosensorischen Modul und einer signalgebenden Histidin-Kinase-Domäne. Das photosensorische Modul bindet das Tetrapyrrol Biliverdin als Chromophor. Der Grundzustand von Agp2 (Pfr, 750 nm) ist gegenüber dem lichtaktivierten Zustand (Pr, 700 nm) rotverschoben, weshalb Agp2 den Bathyphytochromen zugeordnet wird. Die Untersuchungen erfolgten unter Verwendung von Isotopenmarkierung, H/D-Austauschexperimenten und ortsspezifischer Mutagenese. Daraus ließen sich folgende molekulare Änderungen charakterisieren, welche im Reaktionszyklus von Agp2 erfolgen: Die lichtinduzierte Isomerisierung des Chromophors führt zu einem Übergang vom Pfr- in den Pr-Zustand, wobei zwei Intermediate, Lumi‑F und Meta‑F, durchlaufen werden. Neben der Konformationsänderung des Chromophor‑D‑Rings ist auch die C‑Ring-Propionsäureseitenkette an der Photoreaktion beteiligt. Die C-Ring-Propionsäureseitenkette ist im Pfr-Zustand protoniert und wird im Übergang von Meta-F zu Pr deprotoniert. Der Pr-Zustand weist eine pH-Abhängigkeit auf, welche auf die pH-abhängige Ladung des Histidins 278 der Chromophortasche zurückzuführen ist. Je nach Ladung des Histidins 278 wird die Keto‑ bzw. Enolform der C(19)=O‑Gruppe des D‑Rings stabilisiert. Die Keto/Enol-Tautomerie ist auf eine innerhalb des Chromophors erfolgende Protontranslokation zurückzuführen und moduliert die Relaxation in den Pfr-Zustand. Änderungen der Amid-I-Absorption im Pfr-Pr-Übergang werden der Umstrukturierung der Tongue-Region des photosensorischen Moduls von einer Alpha-helikalen zu einer Beta‑Faltblatt-Struktur zugeordnet. Diese Strukturänderung wird als möglicher Weg der proteininternen Signaltransduktion zwischen photosensorischem und signalgebendem Modul vorgeschlagen.In this thesis the light-induced reaction cycle of the bacterial phytochrome Agp2 from Agrobacterium tumefaciens was investigated using FTIR and UV‑vis spectroscopy. The photoreceptor comprises a photosensitive module and a signalling histidine kinase domain. The photosensitive module binds the biliverdin tetrapyrrol as chromophore. The Agp2 ground state (Pfr, 750 nm) is red-shifted in comparison with its light-activated state (Pr, 700 nm). Therefore, Agp2 is assigned to the group of bathy phytochromes. The investigations were conducted using isotopically labelled protein, labelled chromophore as well as hydrogen‑deuterium (H‑D) exchange and site-directed mutagenesis. Based on these the following molecular changes could be characterized that occur in the reaction cycle of Agp2: The light-induced isomerization of the chromophore leads to a transition from the Pfr to the Pr state, involving two intermediates, Lumi-F and Meta-F. Besides conformational changes of the chromophore D-ring, the C-ring propionic side chain is involved in the photoreaction as well. The C-ring propionic side chain is protonated in the Pfr state and gets deprotonated in the Meta-F to Pr transition. The Pr state exhibits pH‑dependent alterations which can be explained by pH dependent polarity changes of histidine 278 in the chromophore pocket. Depending on the charge of histidine, the D‑ring C(19)=O group is stabilized either in keto or enol form. The keto/enol tautomerism involves a proton translocation within the chromophore and modulates the relaxation to the Pfr state. The changes in the amide I region in the Pfr-Pr transition are associated with an alpha‑helix to beta‑sheet secondary structure change of the PHY domain tongue‑region. This structural change is proposed as the potential path of signal transduction between the photosensitive and the signalling module

An algebra for probabilistic xml retrieval

In this paper, we describe a new algebra for XML retrieval. We first describe how to transform an XPath-like query in our algebra. The latter contains a vague predicate, about, which defines a set of document parts within an XML document that fullfill a query expressed as in “flat ” Information Retrieval – a query that contains only constraints on content but not on structure. This predicate is evaluated in a probabilistic way: we thus need a probabilistic interpretation of our algebra. Answers to query needs expressed with vague content and vague structure constraints can then be evaluated. 1

Structure, recherche d'information et apprentissage

National audienc

A Machine Learning Model for Information Retrieval with Structured Documents

International audienceMost recent document standards rely on structured representations. On the other hand, current information retrieval systems have been developed for flat document representations and cannot be easily extended to cope with more complex document types. Only a few models have been proposed for handling structured documents, and the design of such systems is still an open problem. We present here a new model for structured document retrieval which allows to compute and to combine the scores of document parts. It is based on bayesian networks and allows for learning the model parameters in the presence of incomplete data. We present an application of this model for ad-hoc retrieval and evaluate its performances on a small structured collection. The model can also be extended to cope with other tasks such as interactive navigation in structured documents or corpus

Représentations Gaussiennes pour le Filtrage Collaboratif

International audienceLa plupart des systèmes de filtrage collaboratifs, comme par exemple la factorisation matricielle, utilisent des représentations vectorielles pour les articles et les utilisateurs. Ces représentations sont déterministes, et ne permettent pas de modéliser l'incertitude des représentations apprises, ce qui peut être utile quand un utilisateur a évalué un petit nombre d'articles (problème du démarrage à froid), ou quand le modèle est confronté à des informations contradictoires concernant le comportement d'un utilisateur ou les évaluations d'un utilisateur. Dans cet article, nous nous appuyons sur des représentations gaussiennes qui modélisent cette incertitude, et définissons un cadre d'apprentissage et d'inférence approprié à ce type de représentations. Nous montrons en outre que ce modèle fonctionne bien sur trois collections représentatives, et analysons les représentations apprises par le modèle