Tight Integrated End-to-End Training for Cascaded Speech Translation

A cascaded speech translation model relies on discrete and non-differentiable transcription, which provides a supervision signal from the source side and helps the transformation between source speech and target text. Such modeling suffers from error propagation between ASR and MT models. Direct speech translation is an alternative method to avoid error propagation; however, its performance is often behind the cascade system. To use an intermediate representation and preserve the end-to-end trainability, previous studies have proposed using two-stage models by passing the hidden vectors of the recognizer into the decoder of the MT model and ignoring the MT encoder. This work explores the feasibility of collapsing the entire cascade components into a single end-to-end trainable model by optimizing all parameters of ASR and MT models jointly without ignoring any learned parameters. It is a tightly integrated method that passes renormalized source word posterior distributions as a soft decision instead of one-hot vectors and enables backpropagation. Therefore, it provides both transcriptions and translations and achieves strong consistency between them. Our experiments on four tasks with different data scenarios show that the model outperforms cascade models up to 1.8% in BLEU and 2.0% in TER and is superior compared to direct models.Comment: 8 pages, accepted at SLT202

JAVANESE CULTURE DEPICTED IN THE USE OF KINSHIP ADDRESS TERMS

The Javanese system of kinship terms of address relies tightly on its social construction mirroring not only culture but also point of view of the Javanese people. It is undeniable that the society has long been structured vertically hierarchical. There is always upper to lower social relationship among social practices. Hence it is manifested in the form of Javanese speech levels illustrating that a speaker should consider both role and circumstance. The levels of speech presumably fall into some smaller linguistic features, one of which is address term particularly maintaining kinship relationship. Javanese culture considers ascending generation as the polar asserting that father’s/mother’s big brothers/sisters, father’s/mother’s big brother’s/sister’s sons/daughters, father’s/mother’s big brother’s/sister’s son’s/daughter’s sons/daughters would be addressed as all terms equivalent to big or older sibling

Evorus: A Crowd-powered Conversational Assistant Built to Automate Itself Over Time

Crowd-powered conversational assistants have been shown to be more robust than automated systems, but do so at the cost of higher response latency and monetary costs. A promising direction is to combine the two approaches for high quality, low latency, and low cost solutions. In this paper, we introduce Evorus, a crowd-powered conversational assistant built to automate itself over time by (i) allowing new chatbots to be easily integrated to automate more scenarios, (ii) reusing prior crowd answers, and (iii) learning to automatically approve response candidates. Our 5-month-long deployment with 80 participants and 281 conversations shows that Evorus can automate itself without compromising conversation quality. Crowd-AI architectures have long been proposed as a way to reduce cost and latency for crowd-powered systems; Evorus demonstrates how automation can be introduced successfully in a deployed system. Its architecture allows future researchers to make further innovation on the underlying automated components in the context of a deployed open domain dialog system.Comment: 10 pages. To appear in the Proceedings of the Conference on Human Factors in Computing Systems 2018 (CHI'18

The lexical approach: collocability, fluency and implications for teaching

The lexical approach identifies lexis as the basis of language and focuses on the principle that language consists of grammaticalised lexis. in second language acquisition, over the past few years, this approach has generated great interest as an alternative to traditional grammar-based teaching methods. From a psycholinguistic point of view, the lexical approach consists of the capacity of understanding and producing lexical phrases as non-analysed entities (chunks). A growing body of literature concerning spoken fluency is in favour of integrating automaticity and formulaic language units into classroom practice. in line with the latest theories on SlA, we recommend the inclusion of a language awareness component as an integral part of this approach. The purpose is to induce what Schmidt (1990) calls noticing, i.e., registering forms in the input so as to store them in memory. This paper, which is in keeping with the interuniversity Research Project “Evidentiality in a multidisciplinary corpus of English research papers” of the University of las Palmas de Gran Canaria, provides a theoretical overview on the research of this approach taking into account both the methodological foundations on the subject and its pedagogical implications for SLA.

Can integrated titles improve the viewing experience? Investigating the impact of subtitling on the reception and enjoyment of film using eye tracking and questionnaire data

Historically a dubbing country, Germany is not well-known for subtitled productions. But while dubbing is predominant in Germany, more and more German viewers prefer original and subtitled versions of their favourite shows and films. Conventional subtitling, however, can be seen as a strong intrusion into the original image that can not only disrupt but also destroy the director’s intended shot composition and focus points. Long eye movements between focus points and subtitles decrease the viewer’s information intake, and especially German audiences, who are often not used to subtitles, seem to prefer to wait for the next subtitle instead of looking back up again. Furthermore, not only the placement, but also the overall design of conventional subtitles can disturb the image composition – for instance titles with a weak contrast, inappropriate typeface or irritating colour system. So should it not, despite the translation process, be possible to preserve both image and sound as far as possible? Especially given today’s numerous artistic and technical possibilities and the huge amount of work that goes into the visual aspects of a film, taking into account not only special effects, but also typefaces, opening credits and text-image compositions. A further development of existing subtitling guidelines would not only express respect towards the original film version but also the translator’s work.   The presented study shows how integrated titles can increase information intake while maintaining the intended image composition and focus points as well as the aesthetics of the shot compositions. During a three-stage experiment, the specifically for this purpose created integrated titles in the documentary “Joining the Dots” by director Pablo Romero-Fresco were analysed with the help of eye movement data from more than 45 participants. Titles were placed based on the gaze behaviour of English native speakers and then rated by German viewers dependant on a German translation. The results show that a reduction of the distance between intended focus points and titles allow the viewers more time to explore the image and connect the titles to the plot. The integrated titles were rated as more aesthetically pleasing and reading durations were shorter than with conventional subtitles. Based on the analysis of graphic design and filmmaking rules as well as conventional subtitling standards, a first workflow and set of placement strategies for integrated titles were created in order to allow a more respectful handling of film material as well as the preservation of the original image composition and typographic film identity

Case Studies of Global Transference. Language, Media and Culture Translation in Xu Bing’s Writing-Art

En la era del arte contemporáneo global, la traducibilidad cultural, lingüística y mediática de las obras de arte es esencial para llegar a públicos, instituciones artísticas y mercados artísticos transnacionalmente diversificados. La práctica de la escritura-arte del artista contemporáneo chino Xu Bing, emigrante a los Estados Unidos y remigrante a China, ofrece amplios casos de estudio para discutir la importancia de la traducción como metáfora, práctica y herramienta técnica en las producciones artísticas globales. A partir del análisis de los happenings e instalaciones, entre ellos A Case Study of Transference(1994), Your Surname, Please(1998) y Book from the Ground(2014), este artículo investiga los intentos artísticos de Bing de reemplazar la traducción lingüística con la transcodificación de iconos con el objetivo de adaptarse a la “era pictográfica” digital de la comunicación visual global. La creación de un sistema de iconos translingüísticos y transculturales en Book from the Groundse interpreta por tanto como un nuevo proyecto artístico de Babel, que proporciona una herramienta de traducción visual universal en tiempos de transformación digital. In the era of global contemporary art, the cultural, linguistic and media translatability of artworks is essential for reaching out to transnationally diversified publics, art institutions and art markets. The writing-art of the Chinese contemporary artist Xu Bing – an emigrant to the United States and remigrant to China – provides ample case studies for discussing the significance of translation as a metaphor, practice and technical tool for global artistic performances. Based on the analysis of happenings and installations, among them A Case Study of Transference (1994), Your Surname, Please(1998) and Book from the Ground (2014), the paper investigates Bing’s artistic attempts to replace linguistic translation by icon transcoding in order to adapt to the digital “pictographic age” of global visual communication. The creation of a translinguistic and transcultural icon design system in Book from the Ground  is interpreted as a new artistic Babel project, providing a universal visual translation tool in times of digital transformation.