Artificial Intelligence for Multimedia Signal Processing

Artificial intelligence technologies are also actively applied to broadcasting and multimedia processing technologies. A lot of research has been conducted in a wide variety of fields, such as content creation, transmission, and security, and these attempts have been made in the past two to three years to improve image, video, speech, and other data compression efficiency in areas related to MPEG media processing technology. Additionally, technologies such as media creation, processing, editing, and creating scenarios are very important areas of research in multimedia processing and engineering. This book contains a collection of some topics broadly across advanced computational intelligence algorithms and technologies for emerging multimedia signal processing as: Computer vision field, speech/sound/text processing, and content analysis/information mining

Knowledge will Propel Machine Understanding of Content: Extrapolating from Current Examples

Machine Learning has been a big success story during the AI resurgence. One particular stand out success relates to learning from a massive amount of data. In spite of early assertions of the unreasonable effectiveness of data, there is increasing recognition for utilizing knowledge whenever it is available or can be created purposefully. In this paper, we discuss the indispensable role of knowledge for deeper understanding of content where (i) large amounts of training data are unavailable, (ii) the objects to be recognized are complex, (e.g., implicit entities and highly subjective content), and (iii) applications need to use complementary or related data in multiple modalities/media. What brings us to the cusp of rapid progress is our ability to (a) create relevant and reliable knowledge and (b) carefully exploit knowledge to enhance ML/NLP techniques. Using diverse examples, we seek to foretell unprecedented progress in our ability for deeper understanding and exploitation of multimodal data and continued incorporation of knowledge in learning techniques.Comment: Pre-print of the paper accepted at 2017 IEEE/WIC/ACM International Conference on Web Intelligence (WI). arXiv admin note: substantial text overlap with arXiv:1610.0770

Data Showcases: the Data Journal in a Multimodal World

   As an experiment, the Research Data Journal for the Humanities and Social Sciences (RDJ) has temporarily extended the usual format of the online journal with so-called ‘showcases’, separate web pages containing a quick introduction to a dataset, embedded multimedia, interactive components, and facilities to directly preview and explore the dataset described. The aim was to create a coherent hyper document with content communicated via different media (multimodality) and provide space for new forms of scientific publication such as executable papers (e.g. Jupyter notebooks). This paper discusses the objectives, technical implementations, and the need for innovation in data publishing considering the advanced possibilities of today's digital modes of communication. The data showcases experiment proved to be a useful starting point for an exploration of related developments within and outside the humanities and social sciences. It turns out that small-scale experiments are relatively easy to perform thanks to the easy availability of digital technology. However, real innovation in publishing affects organization and infrastructure and requires the joint effort of publishers, editors, data repositories, and authors. It implies a thorough update of the concept of publication and adaptation of the production process. This paper also pays attention to these obstacles to taking new paths

Discourse and Digital Practices

Discourse and Digital Practices shows how tools from discourse analysis can be used to help us understand new communication practices associated with digital media, from video gaming and social networking to apps and photo sharing. This cutting-edge book: draws together fourteen eminent scholars in the field including James Paul Gee, David Barton, Ilana Snyder, Phil Benson, Victoria Carrington, Guy Merchant, Camilla Vasquez, Neil Selwyn and Rodney Jones answers the central question: "How does discourse analysis enable us to understand digital practices?" addresses a different type of digital media in each chapter demonstrates how digital practices and the associated new technologies challenge discourse analysts to adapt traditional analytic tools and formulate new theories and methodologies examines digital practices from a wide variety of approaches including textual analysis, conversation analysis, interactional sociolinguistics, multimodal discourse analysis, object ethnography, geosemiotics, and critical discourse analysis. Discourse and Digital Practices will be of interest to advanced students studying courses on digital literacies or language and digital practices

Deep Multimodal Image-Repurposing Detection

Nefarious actors on social media and other platforms often spread rumors and falsehoods through images whose metadata (e.g., captions) have been modified to provide visual substantiation of the rumor/falsehood. This type of modification is referred to as image repurposing, in which often an unmanipulated image is published along with incorrect or manipulated metadata to serve the actor's ulterior motives. We present the Multimodal Entity Image Repurposing (MEIR) dataset, a substantially challenging dataset over that which has been previously available to support research into image repurposing detection. The new dataset includes location, person, and organization manipulations on real-world data sourced from Flickr. We also present a novel, end-to-end, deep multimodal learning model for assessing the integrity of an image by combining information extracted from the image with related information from a knowledge base. The proposed method is compared against state-of-the-art techniques on existing datasets as well as MEIR, where it outperforms existing methods across the board, with AUC improvement up to 0.23.Comment: To be published at ACM Multimeda 2018 (orals

CMC-core: a schema for the representation of CMC corpora in TEI

Dans cet article, nous décrivons un schéma et des modèles de représentation développés pour structurer les corpus de communication médiée par ordinateur (CMC) en suivant les recommandations de la Text Encoding Initiative (TEI). Nous considérons le discours CMC comme un échange dialogique entre humains, organisé de manière séquentielle. Nous insistons d’abord sur le fait que de nombreuses caractéristiques de la CMC ne sont pas traitées de manière adéquate par les schémas et les outils actuels d’encodage de corpus. Nous formulons donc un ensemble de recommandations pour représenter la CMC avec des schémas d’encodage, en insistant sur le fait que la TEI nous semble être un cadre particulièrement approprié pour l’encodage des corpus CMC. Nous proposons une modélisation des unités de base de la CMC (énoncés, messages et actions non verbales) ainsi que des structures de niveaux macro- et micro des interactions dans les environnements de la CMC. À partir de ces modèles, nous introduisons le CMC-core, un noyau TEI pour l’encodage des corpus CMC, qui définit un ensemble de traits d’encodage spécifiques à la CMC sur quatre niveaux: (i) éléments, (ii) classes de modèles, (iii) classes d'attributs et (iv) modules de l'infrastructure TEI. La description du noyau proposé est illustrée au moyen d’exemples extraits des corpus des chercheurs du groupe SIG TEI CMC, représentant une grande variété de genres de la CMC (le chat, le wiki talk, le tweet, le blog, les interactions Second Life…). Le matériel décrit, i.e. les schémas, les exemples d’encodage et la documentation, est disponible sur le Wiki du SIG CMC TEI et accompagnera une demande d’enrichissement de la TEI (TEI feature request) au conseil de la TEI à la fin de l’année 2019.In this Paper, we describe a schema and models which have been developed for the representation of corpora of computer-mediated communicatin (CMC corpora) using the representation framework provided by the Text Encoding Initiative (TEI). We characterise CMC discourse as dialogic, sequentially organised interchange between humans and point out that many features of CMC are not adequately handled by current corpus encoding schemas and tools. We formulate desiderata for a representation of CMC in encoding schemes and argue why the TEI is a suitable framework for the encoding of CMC corpora. We propose a model of basic CMC units (utterances, posts, and nonverbal activities) and the macro- and micro-level structures of interactions in CMC environments. Based on these models, we introduce CMC-core, a TEI customisation for the encoding of CMC corpora, which defines CMC-specific encoding features on the four levels of elements, model classes, attribute classes, and modules of the TEI infrastructure. The description of our customisation is illustrated by encoding examples from corpora by researchers of the TEI SIG CMC, representing a variety of CMC genres, i.e. chat, wiki talk, twitter, blog, and Second Life interactions. The material described, i.e. schemata, encoding examples, and documentation, is available from the of the TEI CMC SIG Wiki and will accompany a feature request to the TEI council in late 2019

Discourse and Digital Practices

Discourse and Digital Practices shows how tools from discourse analysis can be used to help us understand new communication practices associated with digital media, from video gaming and social networking to apps and photo sharing. This cutting-edge book: draws together fourteen eminent scholars in the field including James Paul Gee, David Barton, Ilana Snyder, Phil Benson, Victoria Carrington, Guy Merchant, Camilla Vasquez, Neil Selwyn and Rodney Jones answers the central question: "How does discourse analysis enable us to understand digital practices?" addresses a different type of digital media in each chapter demonstrates how digital practices and the associated new technologies challenge discourse analysts to adapt traditional analytic tools and formulate new theories and methodologies examines digital practices from a wide variety of approaches including textual analysis, conversation analysis, interactional sociolinguistics, multimodal discourse analysis, object ethnography, geosemiotics, and critical discourse analysis. Discourse and Digital Practices will be of interest to advanced students studying courses on digital literacies or language and digital practices

Using Cloudworks to Support OER Activities

This report forms the third and final output of the Pearls in the Clouds project, funded by the Higher Education Academy. It focuses on evaluation of the use of a social networking site, Cloudworks, to support evidence-based practice. The aim of this project (Pearls in the Clouds) has been to evaluate the ways in which web 2.0 tools like Cloudworks can support evidence-informed practices in relation to learning and teaching. We have reviewed evidence from empirically grounded studies surrounding the uses of web2.0 in higher education and highlighted the gap between using web2.0 to support learning and teaching, and using it to support learning about learning and teaching (in an evidence-informed way) (Conole and Alevizou, 2010). We have reported on findings from a case study focusing on the use of Cloudworks by a community of practice - educational technologists - reflecting upon, and, negotiating their role in enhancing teaching and learning in higher education (Galley et al., 2010). The object of this study is to explore and evaluate the use of the site by individuals and communities involved in the production of, and research on, the development, delivery and use of Open Educational Resources (OER)