Tutorial: End-to-End Speech Translation

Speech translation is the translation of speech in one language typically to text in another, traditionally accomplished through a combination of automatic speech recognition and machine translation. Speech translation has attracted interest for many years, but the recent successful applications of deep learning to both individual tasks have enabled new opportunities through joint modeling, in what we today call 'end-to-end speech translation.' In this tutorial we will introduce the techniques used in cutting-edge research on speech translation. Starting from the traditional cascaded approach, we will given an overview on data sources and model architectures to achieve state-of-the art performance with end-to-end speech translation for both high- and low-resource languages. In addition, we will discuss methods to evaluate analyze the proposed solutions, as well as the challenges faced when applying speech translation models for real-world applications

A patient-centred approach in neuro-oncology: definition and measurement of clinical outcome in brain tumor patient

Brain tumors, independently from histology, share a similar progressive growth pattern that often requires multiple treatment modalities. Surgery, radiotherapy, chemotherapy and medications are often administered in close proximity to one another or, sometimes, contemporaneously. For that reason, the assessment of the effectiveness of any single treatment is extremely difficult. Brain tumors are a rare cancer but they are among the most devastating forms of cancer and afflict the very core of the person. Optimizing the life style of patients should become essential and on a par with the life prolongation goal of anti-cancer therapy. A potential solution to this problem is a patient-centered approach to different tumor types and treatments in which the comprehensive patient assessment is included in the outcome set. Traditionally, clinical trials of treatments for gliomas have relied on measures such as the reduction in tumor size, or on time-dependent metrics including the progression-free survival and the overall survival. The procedure used to evaluate of treatment effects by means of these measures can be complemented by the assessment of clinical outcomes such as measurements of the functional or symptomatic effects of the condition on the person. Unfortunately, the most commonly reported outcome measures are not necessarily the most appropriate. They are often selected and motivated since they are widely cited in the literature; this is an incorrect approach on a scientific ground and also precludes new and more effective instruments to be introduced and accepted. The instruments currently used in neuro-oncology are influenced by notions anchored to the type of treatment (such as chemotherapy or radiotherapy) rather than to tumor and, particularly, to patient characteristics. The utility of patient-reported outcome data can be maximized with standardization of methods to assess, analyze, interpret and report results. Nevertheless, neuro-oncological guidelines do not mention instruments and parameters but rather generically prescribe “neurological monitoring”. A paradox is that, while new therapies and diagnostics are improving survival rates, much remains unknown about the patient clinical conditions and performance at an instrumental level. However, in the last years, a growing interest on cognitive response in clinical trials has been observed and the cognitive performance is now considered as an important patient-related outcome to assess response to treatments. Tracking symptoms and function can inform clinicians about whether a treatment results in measurable benefits or adverse effects to patients. 6 We should not forget that patients want to live longer, but they also want to continue to function as well as possible for as long as possible. This research. arose from the considerations set forth above. The aim of the thesis is to provide an overview of the clinical outcome assessment of patients with brain tumors. The thesis analyzes the role of the assessment in collecting information about the direct impact of neoplastic invasion of the cerebral parenchyma and in monitoring the effects of therapies. This thesis is composed of three different studies. The first study describes the characteristics of clinical trials on gliomas and the outcome profile (objective, endpoints, domains, categories and instruments), since the 1990 (the advent of evidence based medicine) to 2019. The second study describes the neuropsychological tests mainly used in the brain tumor clinical trials published over the last 30 years and tries to evaluate the methodological quality of studies on measurement properties by means of the COSMIN checklist. Finally, the third study focuses on the neuropsychological assessment of language in multilingual people. Due to the increasingly widespread integration, this has become a crucial issue in neuro-oncology, especially in the perioperative evaluation of patients undergoing surgical treatment for brain cancer. The aim is to verify to what extent the variables that affect linguistic processing in multilingual speakers have been considered during planning and decision making in awake surgery for brain tumors

Connectionist and Process Modelling of Long-Term Sequence: The Integration of Relative Judgements, Representation and Learning

A large amount of psychological research is devoted to the representation of sequences. It is a fundamental upon which most of the processes of cognition are based. Despite the amount of research into sequencing, there has been relatively little investigation of the types of representations generated in response to sequential information. These representations must allow operations to be performed on individual elements, as well as operations between and among elements. This thesis begins by describing the effects found when subjects are asked to make relative order judgements using sequences which are in long-term storage (e.g. the alphabet or number series). These effects are then used to examine some of the theories and models which have been developed, with a view toward generating a general purpose mechanism with the ability to model all of the different effects found with different types of stimuli. In the course of developing the new model, the neuropsychological findings in the area are examined in Chapter Two. Deficit studies and neuropsychological investigations are able to isolate which aspects of a task appear to be processed in different structures. If a patient loses the ability to perform one aspect of a task but not another, trying to model both of these aspects in one network may be counter-productive. The construction of a new model is begun in Chapter Three. This model is developed in the PDP environment as it offers the ability to change (learn) as a result of experience, and demands a more thorough definition of the mechanisms operating within the network. Chapter Four details a formal definition of the Serial Order Network (SON) model outlined in Chapter Three, including a section devoted to relative order judgements, called the Response Generation Network (RGN) and undertakes a comparison between the SON/RGN and Poltrock's (1990) random walk model described in Chapter One. A review of some of the sequence learning networks developed is undertaken in Chapter Five. This review is used to choose sequence learning networks, which may be used to learn the type of representation needed. These sequence learning networks are investigated in Chapter Six for their ability to learn the sequence incrementally. It is determined that not one of these networks is appropriate. Thus, in Chapter Seven a recitation mechanism is added directly to the representation in the SON. The resulting system is investigated, and it is determined that the system's success in recitation is not dependent on an idiosyncratic setting of the parameters in the network. The definition of the SON and complementary recitation network is not sufficient. The resulting mechanisms should also be compatible with the developmental literature for both children learning sequences for the first time, and adults learning novel sequences. A review of this literature is conducted in Chapter Eight. It is also necessary to explain how this SON representation can be developed, and how the model can be used to explain the seeming hierarchic nature of some sequences. In Chapter Nine a mechanism designed to mimic a hierarchical structure for the representation of a sequence is developed. A learning mechanism is defined for the resulting system. This system is then investigated for its ability to recite both hierarchic and non-hierarchic sequences, and to generate the relative order and developmental effects referred to in Chapters One and Nine. The model developed in this thesis is the only model existent which is able to explain sequence learning, representation and relative order effects, and represents an advance in the approach to modelling sequence information

CHORUS Deliverable 2.1: State of the Art on Multimedia Search Engines

Based on the information provided by European projects and national initiatives related to multimedia search as well as domains experts that participated in the CHORUS Think-thanks and workshops, this document reports on the state of the art related to multimedia content search from, a technical, and socio-economic perspective. The technical perspective includes an up to date view on content based indexing and retrieval technologies, multimedia search in the context of mobile devices and peer-to-peer networks, and an overview of current evaluation and benchmark inititiatives to measure the performance of multimedia search engines. From a socio-economic perspective we inventorize the impact and legal consequences of these technical advances and point out future directions of research

On semantic differences: a multivariate corpus-based study of the semantic field of inchoativity in translated and non-translated Dutch

This dissertation places the study of semantic differences in translation compared to non-translation at the centre of its concerns. To date, much research in Corpus-based Translation Studies has focused on lexical and grammatical phenomena in an attempt to reveal presumed general tendencies of translation. On the semantic level, these general tendencies have rarely been investigated. Therefore, the goal of this study is to explore whether universal tendencies of translation also exist on the semantic level, thereby connecting the framework of translation universals to semantics

Acta Universitatis Sapientiae - Electrical and Mechanical Engineering

Series Electrical and Mechanical Engineering publishes original papers and surveys in various fields of Electrical and Mechanical Engineering

Language beyond language: comics as verbo-visual texts

The investigation proposed in this study is based on the consideration that the nature of "text" is currently undergoing a change whereby verbal components are increasingly being accompanied by visual components, and the two modes of expression co-exist side by side in the same texts. The Internet is symptomatic of this change, with its multi-modal texts, where words, pictures, and sometimes even sounds, interact with one another. One of the main issues that this thesis aims to address is that although the relationship between the verbal and the visual is not an entirely new area of study, what characterises traditional approaches is the fact that the two components have fundamentally been considered as separate entities, while the combination of words and pictures has generally been regarded as having the function of aiding comprehension. This thesis is based on the main hypothesis that the combination of verbal and visual components is a true interaction which creates a type of 'language' that is more than a simple sum of the two codes. This type of verbo-visual interaction characterises media as old as film and comics, both of which came into existence around a century ago. However, while film studies has become an established discipline, comics have never enjoyed much scholarly attention, their expressive potentials having gone largely overlooked, and the publications that deal with them having being essentially socio-historical accounts. This thesis aims to investigate the complex and sophisticated type of interaction between verbal and visual elements that takes place in comics, and suggests that a close scrutiny of this medium enables the researcher to understand better the way in which the verbal/visual interaction works. In doing so, it recognises the necessity for linguistics to expand the notion of 'language' beyond the traditional verbal boundaries and to incorporate other types of codes which exhibit 'language-like' properties. The theoretical discussion is guided by an eclectic approach as it draws from the fields of semiotics, text-linguistics and stylistics. Accordingly, rather than developing one single main analytical model, this study proposes smaller frameworks, one in each of the areas of study drawn from. Finally, the thesis also suggests ways of applying the theoretical finding for pedagogical purposes

The importance of spatial language for early numerical development in preschool: Going beyond verbal number skills.

peer reviewedRecent evidence suggests that spatial language in preschool positively affects the development of verbal number skills, as indexed by aggregated performances on counting and number naming tasks. We firstly aimed to specify whether spatial language (the knowledge of locative prepositions) significantly relates to both of these measures. In addition, we assessed whether the predictive value of spatial language extends beyond verbal number skills to numerical subdomains without explicit verbal component, such as number writing, symbolic magnitude classifications, ordinal judgments and numerosity comparisons. To determine the unique contributions of spatial language to these numerical skills, we controlled in our regression analyses for intrinsic and extrinsic spatial abilities, phonological awareness as well as age, socioeconomic status and home language. With respect to verbal number skills, it appeared that spatial language uniquely predicted forward and backward counting but not number naming, which was significantly affected only by phonological awareness. Regarding numerical tasks that do not contain explicit verbal components, spatial language did not relate to number writing or numerosity comparisons. Conversely, it explained unique variance in symbolic magnitude classifications and was the only predictor of ordinal judgments. These findings thus highlight the importance of spatial language for early numerical development beyond verbal number skills and suggest that the knowledge of spatial terms is especially relevant for processing cardinal and ordinal relations between symbolic numbers. Promoting spatial language in preschool might thus be an interesting avenue for fostering the acquisition of these symbolic numerical skills prior to formal schooling