Harnessing AI for Speech Reconstruction using Multi-view Silent Video Feed

Speechreading or lipreading is the technique of understanding and getting phonetic features from a speaker's visual features such as movement of lips, face, teeth and tongue. It has a wide range of multimedia applications such as in surveillance, Internet telephony, and as an aid to a person with hearing impairments. However, most of the work in speechreading has been limited to text generation from silent videos. Recently, research has started venturing into generating (audio) speech from silent video sequences but there have been no developments thus far in dealing with divergent views and poses of a speaker. Thus although, we have multiple camera feeds for the speech of a user, but we have failed in using these multiple video feeds for dealing with the different poses. To this end, this paper presents the world's first ever multi-view speech reading and reconstruction system. This work encompasses the boundaries of multimedia research by putting forth a model which leverages silent video feeds from multiple cameras recording the same subject to generate intelligent speech for a speaker. Initial results confirm the usefulness of exploiting multiple camera views in building an efficient speech reading and reconstruction system. It further shows the optimal placement of cameras which would lead to the maximum intelligibility of speech. Next, it lays out various innovative applications for the proposed system focusing on its potential prodigious impact in not just security arena but in many other multimedia analytics problems.Comment: 2018 ACM Multimedia Conference (MM '18), October 22--26, 2018, Seoul, Republic of Kore

A review of technologies for collaborative online information seeking: On the contribution of collaborative argumentation

In everyday life, people seek, evaluate, and use online sources to underpin opinions and make decisions. While education must promote the skills people need to critically question the sourcing of online information, it is important, more generally, to understand how to successfully promote the acquisition of any skills related to seeking online information. This review outlines technologies that aim to support users when they collaboratively seek online information. Upon integrating psychological–pedagogical approaches on trust in and the sourcing of online information, argumentation, and computer-supported collaborative learning, we reviewed the literature (N = 95 journal articles) on technologies for collaborative online information seeking. The technologies we identified either addressed collaborative online information seeking as an exclusive process for searching for online information or, alternatively, addressed online information seeking within the context of a more complex learning process. Our review was driven by three main research questions: We aimed to understand whether and how the studies considered 1) the role of trust and critical questioning in the sourcing of online information, 2) the learning processes at play when information seekers engage in collaborative argumentation, and 3) what affordances are offered by technologies that support users’ collaborative seeking of online information. The reviewed articles that focused exclusively on technologies for seeking online information primarily addressed aspects of cooperation (e.g., task management), whereas articles that focused on technologies for integrating the processes of information seeking into the entire learning processes instead highlighted aspects of collaborative argumentation (e.g., exchange of multiple perspectives and critical questioning in argumentation). Seven of the articles referred to trust as an aspect of seekers’ sourcing strategies. We emphasize how researchers’, users’, and technology developers’ consideration of collaborative argumentation could expand the benefits of technological support for seeking online information.Peer Reviewe

Mechanistic insights into the metabolization of S-Sulfocysteine by CHO cells using a multi-omics approach

Please click Additional Files below to see the full abstract

Derivation of the Functional Renormalization Group Beta-Function at order 1/N for Manifolds Pinned by Disorder

In an earlier publication, we have introduced a method to obtain, at large N, the effective action for d-dimensional manifolds in a N-dimensional disordered environment. This allowed to obtain the Functional Renormalization Group (FRG) equation for N=infinity and was shown to reproduce, with no need for ultrametric replica symmetry breaking, the predictions of the Mezard-Parisi solution. Here we compute the corrections at order 1/N. We introduce two novel complementary methods, a diagrammatic and an algebraic one, to perform the complicated resummation of an infinite number of loops, and derive the beta-function of the theory to order 1/N. We present both the effective action and the corresponding functional renormalization group equations. The aim is to explain the conceptual basis and give a detailed account of the novel aspects of such calculations. The analysis of the FRG flow, comparison with other studies, and applications, e.g. to the strong-coupling phase of the Kardar-Parisi-Zhang equation are examined in a subsequent publication.Comment: 62 pages, 97 figure

L'œil écoute

Consultable en ligne sur Revue.org, URL : http://perspective.revues.org/1193International audienc

Displacement Operator Formalism for Renormalization and Gauge Dependence to All Orders

We present a new method for determining the renormalization of Green functions to all orders in perturbation theory, which we call the displacement operator formalism, or the D-formalism, in short. This formalism exploits the fact that the renormalized Green functions may be calculated by displacing by an infinite amount the renormalized fields and parameters of the theory with respect to the unrenormalized ones. With the help of this formalism, we are able to obtain the precise form of the deformations induced to the Nielsen identities after renormalization, and thus derive the exact dependence of the renormalized Green functions on the renormalized gauge-fixing parameter to all orders. As a particular non-trivial example, we calculate the gauge-dependence of $\tan\beta$ at two loops in the framework of an Abelian Higgs model, using a gauge-fixing scheme that preserves the Higgs-boson low-energy theorem for off-shell Green functions. Various possible applications and future directions are briefly discussed.Comment: 41 pages, 8 figure

Renormalization Theory for Interacting Crumpled Manifolds

We consider a continuous model of D-dimensional elastic (polymerized) manifold fluctuating in d-dimensional Euclidean space, interacting with a single impurity via an attractive or repulsive delta-potential (but without self-avoidance interactions). Except for D=1 (the polymer case), this model cannot be mapped onto a local field theory. We show that the use of intrinsic distance geometry allows for a rigorous construction of the high-temperature perturbative expansion and for analytic continuation in the manifold dimension D. We study the renormalization properties of the model for 0<D<2, and show that for d<d* where d*=2D/(2-D) is the upper critical dimension, the perturbative expansion is UV finite, while UV divergences occur as poles at d=d*. The standard proof of perturbative renormalizability for local field theories (the BPH theorem) does not apply to this model. We prove perturbative renormalizability to all orders by constructing a subtraction operator based on a generalization of the Zimmermann forests formalism, and which makes the theory finite at d=d*. This subtraction operation corresponds to a renormalization of the coupling constant of the model (strength of the interaction with the impurity). The existence of a Wilson function, of an epsilon-expansion around the critical dimension, of scaling laws for d<d* in the repulsive case, and of non-trivial critical exponents of the delocalization transition for d>d* in the attractive case is thus established. To our knowledge, this provides the first proof of renormalizability for a model of extended objects, and should be applicable to the study of self-avoidance interactions for random manifolds.Comment: 126 pages (+ 24 figures not included available upon request), harvmac, SPhT/92/12

Renormalization of gauge invariant composite operators in light-cone gauge

We generalize to composite operators concepts and techniques which have been successful in proving renormalization of the effective Action in light-cone gauge. Gauge invariant operators can be grouped into classes, closed under renormalization, which is matrix-wise. In spite of the presence of non-local counterterms, an ``effective" dimensional hierarchy still guarantees that any class is endowed with a finite number of elements. The main result we find is that gauge invariant operators under renormalization mix only among themselves, thanks to the very simple structure of Lee-Ward identities in this gauge, contrary to their behaviour in covariant gauges.Comment: 35100 Padova, Italy DFPD 93/TH/53, July 1993 documentstyle[preprint,aps]{revtex

On-line Photoionisation Mass Spectrometry: an Interesting Technique to Study Biomass Pyrolysis

Please click Additional Files below to see the full abstrac

Intestinal DMBT1 Expression Is Modulated by Crohn’s Disease-Associated IL23R Variants and by a DMBT1 Variant Which Influences Binding of the Transcription Factors CREB1 and ATF-2

Objectives: DMBT is an antibacterial pattern recognition and scavenger receptor. In this study, we analyzed the role of DMBT1 single nucleotide polymorphisms (SNPs) regarding inflammatory bowel disease (IBD) susceptibility and examined their functional impact on transcription factor binding and downstream gene expression. Methods: Seven SNPs in the DMBT1 gene region were analyzed in 2073 individuals including 818 Crohn’s disease (CD) patients and 972 healthy controls in two independent case-control panels. Comprehensive epistasis analyses for the known CD susceptibility genes NOD2, IL23R and IL27 were performed. The influence of IL23R variants on DMBT1 expression was analyzed. Functional analysis included siRNA transfection, quantitative PCR, western blot, electrophoretic mobility shift and luciferase assays. Results: IL-22 induces DMBT1 protein expression in intestinal epithelial cells dependent on STAT3, ATF-2 and CREB1. IL-22 expression-modulating, CD risk-associated IL23R variants influence DMBT1 expression in CD patients and DMBT1 levels are increased in the inflamed intestinal mucosa of CD patients. Several DMBT1 SNPs were associated with CD susceptibility. SNP rs2981804 was most strongly associated with CD in the combined panel (p = 3.0×10−7, OR 1.42; 95% CI 1.24–1.63). All haplotype groups tested showed highly significant associations with CD (including omnibus P-values as low as 6.1×10−18). The most strongly CD risk-associated, non-coding DMBT1 SNP rs2981804 modifies the DNA binding sites for the transcription factors CREB1 and ATF-2 and the respective genomic region comprising rs2981804 is able to act as a transcriptional regulator in vitro. Intestinal DMBT1 expression is decreased in CD patients carrying the rs2981804 CD risk allele. Conclusion: We identified novel associations of DMBT1 variants with CD susceptibility and discovered a novel functional role of rs2981804 in regulating DMBT1 expression. Our data suggest an important role of DMBT1 in CD pathogenesis