Fusing Audio, Textual and Visual Features for Sentiment Analysis of News Videos

This paper presents a novel approach to perform sentiment analysis of news videos, based on the fusion of audio, textual and visual clues extracted from their contents. The proposed approach aims at contributing to the semiodiscoursive study regarding the construction of the ethos (identity) of this media universe, which has become a central part of the modern-day lives of millions of people. To achieve this goal, we apply state-of-the-art computational methods for (1) automatic emotion recognition from facial expressions, (2) extraction of modulations in the participants' speeches and (3) sentiment analysis from the closed caption associated to the videos of interest. More specifically, we compute features, such as, visual intensities of recognized emotions, field sizes of participants, voicing probability, sound loudness, speech fundamental frequencies and the sentiment scores (polarities) from text sentences in the closed caption. Experimental results with a dataset containing 520 annotated news videos from three Brazilian and one American popular TV newscasts show that our approach achieves an accuracy of up to 84% in the sentiments (tension levels) classification task, thus demonstrating its high potential to be used by media analysts in several applications, especially, in the journalistic domain.Comment: 5 pages, 1 figure, International AAAI Conference on Web and Social Medi

A vision-based system to support tactical and physical analyses in futsal

This paper presents a vision-based system to support tactical and physical analyses of futsal teams. Most part of the current analyses in this sport are manually performed, while the existing solutions based on automatic approaches are frequently composed of costly and complex tools, developed for other kind of team sports, making it difficult their adoption by futsal teams. Our system, on the other hand, represents a simple yet efficient dedicated solution, which is based on the analyses of image sequences captured by a single stationary camera used to obtain top-view images of the entire court. We use adaptive background subtraction and blob analysis to detect players, as well as particle filters to track them in every video frame. The system determines the distance traveled by each player, his/her mean and maximum speeds, as well as generates heat maps that describe players’ occupancy during the match. To present the collected data, our system uses a specially developed mobile application. Experimental results with image sequences of an official match and a training match show that our system provides data with global mean tracking errors below 40 cm, demanding on 25 ms to process each frame and, thus, demonstrating its high application potential

Catálogo Taxonômico da Fauna do Brasil: setting the baseline knowledge on the animal diversity in Brazil

The limited temporal completeness and taxonomic accuracy of species lists, made available in a traditional manner in scientific publications, has always represented a problem. These lists are invariably limited to a few taxonomic groups and do not represent up-to-date knowledge of all species and classifications. In this context, the Brazilian megadiverse fauna is no exception, and the Catálogo Taxonômico da Fauna do Brasil (CTFB) (http://fauna.jbrj.gov.br/), made public in 2015, represents a database on biodiversity anchored on a list of valid and expertly recognized scientific names of animals in Brazil. The CTFB is updated in near real time by a team of more than 800 specialists. By January 1, 2024, the CTFB compiled 133,691 nominal species, with 125,138 that were considered valid. Most of the valid species were arthropods (82.3%, with more than 102,000 species) and chordates (7.69%, with over 11,000 species). These taxa were followed by a cluster composed of Mollusca (3,567 species), Platyhelminthes (2,292 species), Annelida (1,833 species), and Nematoda (1,447 species). All remaining groups had less than 1,000 species reported in Brazil, with Cnidaria (831 species), Porifera (628 species), Rotifera (606 species), and Bryozoa (520 species) representing those with more than 500 species. Analysis of the CTFB database can facilitate and direct efforts towards the discovery of new species in Brazil, but it is also fundamental in providing the best available list of valid nominal species to users, including those in science, health, conservation efforts, and any initiative involving animals. The importance of the CTFB is evidenced by the elevated number of citations in the scientific literature in diverse areas of biology, law, anthropology, education, forensic science, and veterinary science, among others

Object-based Image Retrieval using Local Feature Extraction and Relevance Feedback

This paper addresses the problem of object-based image retrieval, by using local feature extraction and a relevance feedback mechanism for quickly narrowing down the image search process to the user needs. This approach relies on the hypothesis that semantically similar images are clustered in some feature space and, in this scenario: (i) computes image signatures that are invariant to scale and rotation using SIFT, (ii) calculates the vector of locally aggregated descriptors (VLAD) to make a fixed length descriptor for the images, (iii) reduce the VLAD descriptor dimensionality with Principal Component Analysis (PCA) and (iv) uses the k-Means algorithm for grouping images that are semantically similar. The proposed approach has been successfully validated using 33,192 images from the ALOI database, obtaining a mean recall value of 47.4 % for searches of images containing objects that are identical to the object query and 20.7 % for searches of images containing different objects (albeit visually similar) to the object query. General Terms Content-based image retrieval, relevance feedback, feature extraction. Keywords Object-based image retrieval, scale invariant feature transform, principal component analysis, vector of locally aggregated descriptors, clustering algorithms. 1

SAPTE : a multimedia information system to support the discourse analysis and information retrieval of television programs.

This paper presents a novel multimedia information system, called SAPTE, for supporting the discourse analysis and information retrieval of television programs from their corresponding video recordings. Unlike most common systems, SAPTE uses both content independent and dependent metadata, which are determined by the application of discourse analysis techniques as well as image and audio analysis methods. The proposed system was developed in partnership with the free-to-air Brazilian TV channel Rede Minas in an attempt to provide TV researchers with computational tools to assist their studies about this media universe. The system is based on the Matterhorn framework for managing video libraries, combining: (1) discourse analysis techniques for describing and indexing the videos, by considering aspects, such as, definitions of the subject of analysis, the nature of the speaker and the corpus of data resulting from the discourse; (2) a state of the art decoder software for large vocabulary continuous speech recognition, called Julius; (3) image and frequency domain techniques to compute visual signatures for the video recordings, containing color, shape and texture information; and (4) hashing and k-d tree methods for data indexing. The capabilities of SAPTE were successfully validated, as demonstrated by our experimental results, indicating that SAPTE is a promising computational tool for TV researchers