Real-Time Statistical Speech Translation

This research investigates the Statistical Machine Translation approaches to translate speech in real time automatically. Such systems can be used in a pipeline with speech recognition and synthesis software in order to produce a real-time voice communication system between foreigners. We obtained three main data sets from spoken proceedings that represent three different types of human speech. TED, Europarl, and OPUS parallel text corpora were used as the basis for training of language models, for developmental tuning and testing of the translation system. We also conducted experiments involving part of speech tagging, compound splitting, linear language model interpolation, TrueCasing and morphosyntactic analysis. We evaluated the effects of variety of data preparations on the translation results using the BLEU, NIST, METEOR and TER metrics and tried to give answer which metric is most suitable for PL-EN language pair.Comment: machine translation, polish englis

Simultaneous interpreting : walking a tightrope

Several phenomena associated with the differences in the performance of novice interpreters and semi-professionals have been discussed in the paper. Particular emphasis was placed on the occurrence of imported cognitive load which strongly influenced the performance of the subjects also in places where no intrinsic difficulty had been detected. Nevertheless, too little evidence was provided to establish a more detailed pattern of imported cognitive load, which was due to the limited number of participants in the study. It would be possible to obtain more detailed data and comments from the participants by means of interviews conducted individually with the participants. It would allow asking detailed questions to the participants, which might be a more reliable method than the immediate retrospective accounts. Moreover, in the present study such variables as gender differences, age differences and the possible influence of other foreign languages were not taken into account. Perhaps these variables might shed some light on the issue of the management of cognitive resources. Also, the corpus gathered for the present study may be used for the investigation of other aspects of the SI performance

DISCOURSE-DRIVEN MEANING CONSTRUCTION IN NEOSEMANTIC NOUN-TO-VERB CONVERSIONS [MEANING CONSTRUCTION IN NOUN-TO-VERB CONVERSIONS]

Neosemantic noun-to-verb conversions such as beer → to beer, door → to door, pink → to pink, etc., constitute a particularly interesting field of study for Cognitive Linguistics in that they call for a discourse-guided and context-based analysis of meaning construction. The present article takes a closer look at the cognitive motivation for the conversion process involved in the noun-verb alterations with a view to explaining the semantics of some conversion formations in relation to the user-centred discourse context. The analysis developed in this article draws from the combined insights of Fauconnier and Turner’s (2002) Conceptual Integration Theory and Langacker’s (2005, 2008) Current Discourse Space

An Approach to the Acquisition of Expert Knowledge

This article presents the approach to the acquisition of expert knowledge and includes the following stages: (1) observation-verbal reporting, (2) obtainment of a record of the observation, (3) protocol analysis, (4) formal record of expert knowledge. The first part of the article discusses each element of our approach based on a real-time case study. Next, the structure and exemplar functionalities of the proposed information system for the acquisition of expert knowledge are presented. Our information system supporting the acquisition of expert knowledge, as presented here, is based on the example of service department workers but may be used in other areas where the procedures for performing any given activity are definable

Guidelines for annotating the LUNA corpus with frame information

This document defines the annotation workflow aimed at adding frame information to the LUNA corpus of conversational speech. In particular, it details both the corpus pre-processing steps and the proper annotation process, giving hints about how to choose the frame and the frame element labels. Besides, the description of 20 new domain-specific and language-specific frames is reported. To our knowledge, this is the first attempt to adapt the frame paradigm to dialogs and at the same time to define new frames and frame elements for the specific domain of software/hardware assistance. The technical report is structured as follows: in Section 2 an overview of the FrameNet project is given, while Section 3 introduces the LUNA project and the annotation framework involving the Italian dialogs. Section 4 details the annotation workflow, including the format preparation of the dialog files and the annotation strategy. In Section 5 we discuss the main issues of the annotation of frame information in dialogs and we describe how the standard annotation procedure was changed in order to face such issues. Then, the 20 newly introduced frames are reported in Section 6

MISAR: A Multimodal Instructional System with Augmented Reality

Augmented reality (AR) requires the seamless integration of visual, auditory, and linguistic channels for optimized human-computer interaction. While auditory and visual inputs facilitate real-time and contextual user guidance, the potential of large language models (LLMs) in this landscape remains largely untapped. Our study introduces an innovative method harnessing LLMs to assimilate information from visual, auditory, and contextual modalities. Focusing on the unique challenge of task performance quantification in AR, we utilize egocentric video, speech, and context analysis. The integration of LLMs facilitates enhanced state estimation, marking a step towards more adaptive AR systems. Code, dataset, and demo will be available at https://github.com/nguyennm1024/misar.Comment: Accepted at ICCV 2023 - AV4D, 6 figures, 2 table