Key event detection in video using ASR and visual data

status: publishe

The effect of using same language subtitling (SLS) in content comprehension and vocabulary acquisition in Arabic as a foreign language (AFL)

This study investigates the effects of SLS (Same-Language subtitling) on content comprehension and vocabulary acquisition of MSA (Modern Standard Arabic) as L2 at the intermediate level and addresses three research questions: (1) Does SLS enhance or hinder L2 content comprehension when the writing script of L2 is different than that of L1? (2) Does SLS enhance or hinder L2 vocabulary acquisition when the writing script of L2 is different than that of L1? (3) What is students\u27 attitude towards the use of SLS? Twenty seven students of AUC\u27s ALI program with English as L1 were chosen for the study and divided between a control group (without SLS) and a treatment group (with SLS). Instructions concerning the procedure were given in L1 to rule out any false results due to task misinterpretation. Both groups watched an authentic 3:11 minute documentary twice after completing an individual background questionnaire and taking a multiple choice vocabulary pre-viewing test. Post-viewing, participants took the same vocabulary test, then wrote a summary in L1 based on their notes taken during/and in-between the viewings and completed a questionnaire/questions related to their experience with and -/out captions. L1 summaries were analyzed in terms of 23 semantic units related to content comprehension. The degree of vocabulary acquisition was calculated by comparing the responses between the pre- and post-viewing vocabulary multiply choice tests. Results using t-test and one way ANOVA indicate that SLS neither facilitates nor hinders comprehension and vocabulary acquisition. The majority of students enjoyed captions and wanted to continue using them in class. Pedagogical suggestions and future research recommended training with captions and focus on other text- aids, e.g. reverse subtitling at other proficiency levels. Contains 84 references, 17 figures, 5 pictures and 5 tables

Dual device user interface design for ubiquitous language learning: Mobile phone and interactive television (iTV)

In this paper we describe the design and development of a system that facilitates language learning from a combination of two devices, interactive television (iTV) and mobile phone. We present a number of requirements for technologies to support informal language learning based on theories of language learning, theories of formal and informal learning, our own studies of adult language learners and the affordances of iTV as a medium to support learning. We describe TAMALLE (television and mobile phone assisted language learning environment), a prototype system based on these requirements and discuss some of the user interface design issues that arise in the context of cross platform dual device systems for ubiquitous learning

The effect of using DVD subtitles in English second-language vocabulary recognition and recall development

The aim of this study was to investigate the effectiveness of DVDs in enhancing student vocabulary development in second-language contexts. To this end the study sought students’ perceptions of DVD subtitles and their level of vocabulary knowledge. It also examined the extent to which watching a DVD with or without intralingual subtitles can improve students’ vocabulary recognition and recall. The literature review included a discussion on the variables operant in second-language acquisition; the use of visual media on vocabulary learning; and the effects of subtitling practices as a didactic tool for vocabulary recall and recognition. The study adopted a mixed-method approach and data were collected through a survey and openended questionnaire; a Vocabulary Levels Test; a Vocabulary Knowledge Scale Test; and vocabulary intervention activities. The findings indicate that DVDs can enhance students’ vocabulary in second-language teaching and learning contexts. More importantly the study confirms audio-visual images create greater sensory input that is, “words associated with actual objects or imagery techniques, are learned more easily than those without” (Chun and Plass, 1996:183).English StudiesM.A. (TESOL

Multimedia resource discovery

This chapter examines the challenges and opportunities of Multimedia Information Retrieval and corresponding search engine applications. Computer technology has changed our access to information tremendously: We used to search authors or titles (which we had to know) in library cards in order to locate relevant books; now we can issue keyword searches within the full text of whole book repositories in order to identify authors, titles and locations of relevant books. What about the corresponding challenge of finding multimedia by fragments, examples and excerpts? Rather than asking for a music piece by artist and title, can we hum its tune to find it? Can doctors submit scans of a patient to identify medically similar images of diagnosed cases in a database? Can your mobile phone take a picture of a statue and tell you about its artist and significance via a service that it sends this picture to? In an attempt to answer some of these questions we get to know basic concepts of multimedia resource discovery technologies for a number of different query and document types: piggy-back text search, i.e., reducing the multimedia to pseudo text documents; automated annotation of visual components; content-based retrieval where the query is an image; and fingerprinting to match near duplicates. Some of the research challenges are given by the semantic gap between the simple pixel properties computers can readily index and high-level human concepts; related to this is an inherent technological limitation of automated annotation of images from pixels alone. Other challenges are given by polysemy, i.e., the many meanings and interpretations that are inherent in visual material and the corresponding wide range of a user’s information need. This chapter demonstrates how these challenges can be tackled by automated processing and machine learning and by utilising the skills of the user, for example through browsing or through a process that is called relevance feedback, thus putting the user at centre stage. The latter is made easier by “added value” technologies, exemplified here by summaries of complex multimedia objects such as TV news, information visualisation techniques for document clusters, visual search by example, and methods to create browsable structures within the collection

Mining of Textual Data from the Web for Speech Recognition

Prvotním cílem tohoto projektu bylo prostudovat problematiku jazykového modelování pro rozpoznávání řeči a techniky pro získávání textových dat z Webu. Text představuje základní techniky rozpoznávání řeči a detailněji popisuje jazykové modely založené na statistických metodách. Zvláště se práce zabývá kriterii pro vyhodnocení kvality jazykových modelů a systémů pro rozpoznávání řeči. Text dále popisuje modely a techniky dolování dat, zvláště vyhledávání informací. Dále jsou představeny problémy spojené se získávání dat z webu, a v kontrastu s tím je představen vyhledávač Google. Součástí projektu byl návrh a implementace systému pro získávání textu z webu, jehož detailnímu popisu je věnována náležitá pozornost. Nicméně, hlavním cílem práce bylo ověřit, zda data získaná z Webu mohou mít nějaký přínos pro rozpoznávání řeči. Popsané techniky se tak snaží najít optimální způsob, jak data získaná z Webu použít pro zlepšení ukázkových jazykových modelů, ale i modelů nasazených v reálných rozpoznávacích systémech.The preliminary goals of this project were to get familiar with language modeling for speech recognition and techniques for acquisition of text data from the Web. Speech recognition techniques are introduced and statistical language modeling is described in detail. The text also covers mining models and techniques, information retrieval especially. Specific problems of Web mining are discussed and Google search is introduced. Special attention was paid to detailed description of implementation of the text mining system. However, the main goal of this work was to determine, whether the data acquired from the Web can provide some improvement into the recognition systems. The text is describing experiments, which use the retrieved Web data to update sample language models.

AI-Generated Content (AIGC): A Survey

To address the challenges of digital intelligence in the digital economy, artificial intelligence-generated content (AIGC) has emerged. AIGC uses artificial intelligence to assist or replace manual content generation by generating content based on user-inputted keywords or requirements. The development of large model algorithms has significantly strengthened the capabilities of AIGC, which makes AIGC products a promising generative tool and adds convenience to our lives. As an upstream technology, AIGC has unlimited potential to support different downstream applications. It is important to analyze AIGC's current capabilities and shortcomings to understand how it can be best utilized in future applications. Therefore, this paper provides an extensive overview of AIGC, covering its definition, essential conditions, cutting-edge capabilities, and advanced features. Moreover, it discusses the benefits of large-scale pre-trained models and the industrial chain of AIGC. Furthermore, the article explores the distinctions between auxiliary generation and automatic generation within AIGC, providing examples of text generation. The paper also examines the potential integration of AIGC with the Metaverse. Lastly, the article highlights existing issues and suggests some future directions for application.Comment: Preprint. 14 figures, 4 table