62 research outputs found
Embedding-Based Speaker Adaptive Training of Deep Neural Networks
An embedding-based speaker adaptive training (SAT) approach is proposed and
investigated in this paper for deep neural network acoustic modeling. In this
approach, speaker embedding vectors, which are a constant given a particular
speaker, are mapped through a control network to layer-dependent element-wise
affine transformations to canonicalize the internal feature representations at
the output of hidden layers of a main network. The control network for
generating the speaker-dependent mappings is jointly estimated with the main
network for the overall speaker adaptive acoustic modeling. Experiments on
large vocabulary continuous speech recognition (LVCSR) tasks show that the
proposed SAT scheme can yield superior performance over the widely-used
speaker-aware training using i-vectors with speaker-adapted input features
Deep Multimodal Learning for Audio-Visual Speech Recognition
In this paper, we present methods in deep multimodal learning for fusing
speech and visual modalities for Audio-Visual Automatic Speech Recognition
(AV-ASR). First, we study an approach where uni-modal deep networks are trained
separately and their final hidden layers fused to obtain a joint feature space
in which another deep network is built. While the audio network alone achieves
a phone error rate (PER) of under clean condition on the IBM large
vocabulary audio-visual studio dataset, this fusion model achieves a PER of
demonstrating the tremendous value of the visual channel in phone
classification even in audio with high signal to noise ratio. Second, we
present a new deep network architecture that uses a bilinear softmax layer to
account for class specific correlations between modalities. We show that
combining the posteriors from the bilinear networks with those from the fused
model mentioned above results in a further significant phone error rate
reduction, yielding a final PER of .Comment: ICASSP 201
Visual securitization of Calais migrants
Since the advent of social media, political communication has become increasingly visual. The increasing use of visuals in political and security discourses presents fundamental challenges to Securitization Theory (ST), which analyses security through the performative aspect of a speech-act. Texts and visuals complement each other to generate security constructions and therefore, ST needs to adopt a multimodal analysis to theorize the performative aspect of text and visuals in combination.
This thesis has two main objectives. First, to present a visual securitization framework that can analyse how discourses of security are constructed through visuals. This will be achieved by drawing theoretical and methodological insights from Rose’s four sites of critical visual methodology, which analyses a visual’s meaning-making along its production, circulation, and consumption stages. The applicability of the framework will be demonstrated through the case study of Calais migrant situation, where visuals were used by truck drivers, travellers and mass media to Calais migrants as security threat.
The second aim of thesis is to analyse how visuals can facilitate the saliency of securitizing moves of actors with insignificant positional power. This thesis argues that for actors with insignificant positional power, visuals are an effective heuristic artefact to gain publicity around their securitizing moves. To this effect, the thesis highlights the importance of publicity in the ST.https://www.ester.ee/record=b5243229*es
Self-critical Sequence Training for Image Captioning
Recently it has been shown that policy-gradient methods for reinforcement
learning can be utilized to train deep end-to-end systems directly on
non-differentiable metrics for the task at hand. In this paper we consider the
problem of optimizing image captioning systems using reinforcement learning,
and show that by carefully optimizing our systems using the test metrics of the
MSCOCO task, significant gains in performance can be realized. Our systems are
built using a new optimization approach that we call self-critical sequence
training (SCST). SCST is a form of the popular REINFORCE algorithm that, rather
than estimating a "baseline" to normalize the rewards and reduce variance,
utilizes the output of its own test-time inference algorithm to normalize the
rewards it experiences. Using this approach, estimating the reward signal (as
actor-critic methods must do) and estimating normalization (as REINFORCE
algorithms typically do) is avoided, while at the same time harmonizing the
model with respect to its test-time inference procedure. Empirically we find
that directly optimizing the CIDEr metric with SCST and greedy decoding at
test-time is highly effective. Our results on the MSCOCO evaluation sever
establish a new state-of-the-art on the task, improving the best result in
terms of CIDEr from 104.9 to 114.7.Comment: CVPR 2017 + additional analysis + fixed baseline results, 16 page
Geometric and strain analyses in folds of the area around Gankot, district Pithoragarh, Uttarakhand, India
The study area around Gankot in Pithoragarh district of Uttarakhand belongs to the Thalkedar Limestone unit of Mandhali Formation, Tejam Group in Inner Sedimentary Zone of Lesser Himalaya, which exposes complexly folded and refolded structures. Geometric analysis carried out on the profile section of the fold tracing using dip isogon and orthogonal thickness parameters revealed presence of all the fold geometry suggested by Ramsay (1967) however the class 3 followed by class 1B are the most dominant classes in the study area when individual layers of the fold were studied. The study of folds as multilayered unit reveals that folds in study area belong to strongly non-analogous fold class of anisodeviatoric folds. In fold, the strain analysis has been done by drawing strain ellipse obtained by Inverse Thickness Method which is useful in estimating flattening strain even when the flattening is imposed obliquely to the fold’s axial trace. The finite two-dimensional flattening strain ratio (Rs) value ranged between 1 and 3.14 with an average Rs value of 1.60. The method of Srivastava and Gairola (2003) has also been used to obtain shear strain and flattening strain for the multilayered folds of study area. The results reveal that the multilayered folds around Gankot area are moderately flattened with mean flatting strain varying between 1.06 and 2.28. A very high degree of variation in shearing ranging about 70o in both clockwise and anticlockwise directions has been noticed. The shear strains (γ) in folds have been found to vary between -2.75 to + 3.27 with an average of +0.33. The shearing and strain patterns are suggestive that the most dominant folding mechanism has been the flexure-shear for the folds of the study area which are overprinted by the fold flattening and other subsequent deformations
- …