1,740 research outputs found
E-MOVIE - Experimental MOVies for Induction of Emotions in Neuroscience: an innovative film database with normative data and sex differences.
The need for a validated set of emotional clips to elicit emotions in more ecological experiments is increasing. Here we present the validation of a new database of emotional films, named E-MOVIE, which includes, in this first validation phase, 39 excerpts arranged in six categories, three negative (Fear, Sadness and Compassion), two positive (Erotic and Scenery) and a Neutral category. Notably, Compassion and Scenery are new in the field as they were not included in other databases. The clips in E-MOVIE are characterized by homogenous durations of approximately two minutes, which make them suitable for psychophysiological research. In order to study the affective profile prompted by each category 174 participants (112 women) rated the movies on multiple dimensions, namely valence and arousal, intensity and discreteness of the induction of one of the six basic emotions and, finally, intensity of the experience of the emotional states defined by a series of emotional adjectives. Erotic clips were effective in the elicitation of a positive emotional state, characterized by high levels of arousal and excitement. On the other hand, Fear clips (selected without blood to avoid disgust reaction) prompted an affect characterized by high arousal, low valence and high levels of reported fear and anxiety. Women reported greater unpleasantness, distress, anxiety and jittery than men to the three negative categories. Compassion clips, characterized by the depiction of crying characters, were able to induce an affective state dominated by sadness and feeling touched, consistent with an empathic reaction to emotional sufferance. Sadness clips, instead, elicited an affective state characterized by sadness together with distress and angst. We also demonstrated that clips depicting natural environments (i.e. Scenery) prompted in the viewer a surprised, inspired affective state, characterized by high valence and arousal (especially in males), a result which suggests that their past categorization as neutral stimuli was inaccurate and problematic
Computer-Assisted Interactive Documentary and Performance Arts in Illimitable Space
This major component of the research described in this thesis is 3D computer
graphics, specifically the realistic physics-based softbody simulation and
haptic responsive environments. Minor components include advanced
human-computer interaction environments, non-linear documentary storytelling,
and theatre performance. The journey of this research has been unusual because
it requires a researcher with solid knowledge and background in multiple
disciplines; who also has to be creative and sensitive in order to combine the
possible areas into a new research direction. [...] It focuses on the advanced
computer graphics and emerges from experimental cinematic works and theatrical
artistic practices. Some development content and installations are completed to
prove and evaluate the described concepts and to be convincing. [...] To
summarize, the resulting work involves not only artistic creativity, but
solving or combining technological hurdles in motion tracking, pattern
recognition, force feedback control, etc., with the available documentary
footage on film, video, or images, and text via a variety of devices [....] and
programming, and installing all the needed interfaces such that it all works in
real-time. Thus, the contribution to the knowledge advancement is in solving
these interfacing problems and the real-time aspects of the interaction that
have uses in film industry, fashion industry, new age interactive theatre,
computer games, and web-based technologies and services for entertainment and
education. It also includes building up on this experience to integrate Kinect-
and haptic-based interaction, artistic scenery rendering, and other forms of
control. This research work connects all the research disciplines, seemingly
disjoint fields of research, such as computer graphics, documentary film,
interactive media, and theatre performance together.Comment: PhD thesis copy; 272 pages, 83 figures, 6 algorithm
Observation of Hand movements by Older Persons with Dementia: Effects on Cognition: a Pilot Study
Background/Aim: Hand movement observation activates mirror neurons, located in brain areas that are vulnerable to Alzheimer's disease. We examined the effects of hand movement observation on cognition in older persons with dementia. Methods: Nursing home residents with dementia (n = 44) watched either videos showing hand movements or videos showing a documentary for 30 min, 5 days a week, for 6 weeks. Neuropsychological tests were performed at baseline, week 6 and week 12. Results: Linear mixed model analyses revealed a significant interaction effect on an attention test, but not on cognitive domains. Additional analyses showed that a face recognition task improved significantly. Conclusion: Although these findings do not support an overall beneficial effect of hand movement observation on cognition in dementia, specific cognitive functions improved. Future studies are warranted. © 2009 S. Karger AG, Basel
Generative AI in the Construction Industry: A State-of-the-art Analysis
The construction industry is a vital sector of the global economy, but it
faces many productivity challenges in various processes, such as design,
planning, procurement, inspection, and maintenance. Generative artificial
intelligence (AI), which can create novel and realistic data or content, such
as text, image, video, or code, based on some input or prior knowledge, offers
innovative and disruptive solutions to address these challenges. However, there
is a gap in the literature on the current state, opportunities, and challenges
of generative AI in the construction industry. This study aims to fill this gap
by providing a state-of-the-art analysis of generative AI in construction, with
three objectives: (1) to review and categorize the existing and emerging
generative AI opportunities and challenges in the construction industry; (2) to
propose a framework for construction firms to build customized generative AI
solutions using their own data, comprising steps such as data collection,
dataset curation, training custom large language model (LLM), model evaluation,
and deployment; and (3) to demonstrate the framework via a case study of
developing a generative model for querying contract documents. The results show
that retrieval augmented generation (RAG) improves the baseline LLM by 5.2,
9.4, and 4.8% in terms of quality, relevance, and reproducibility. This study
provides academics and construction professionals with a comprehensive analysis
and practical framework to guide the adoption of generative AI techniques to
enhance productivity, quality, safety, and sustainability across the
construction industry.Comment: 74 pages, 11 figures, 20 table
FrameNet annotation for multimodal corpora: devising a methodology for the semantic representation of text-image interactions in audiovisual productions
Multimodal analyses have been growing in importance within several approaches to
Cognitive Linguistics and applied fields such as Natural Language Understanding. Nonetheless
fine-grained semantic representations of multimodal objects are still lacking, especially in terms
of integrating areas such as Natural Language Processing and Computer Vision, which are key
for the implementation of multimodality in Computational Linguistics. In this dissertation, we
propose a methodology for extending FrameNet annotation to the multimodal domain, since
FrameNet can provide fine-grained semantic representations, particularly with a database
enriched by Qualia and other interframal and intraframal relations, as it is the case of FrameNet
Brasil. To make FrameNet Brasil able to conduct multimodal analysis, we outlined the
hypothesis that similarly to the way in which words in a sentence evoke frames and organize
their elements in the syntactic locality accompanying them, visual elements in video shots may,
also, evoke frames and organize their elements on the screen or work complementarily with the
frame evocation patterns of the sentences narrated simultaneously to their appearance on screen,
providing different profiling and perspective options for meaning construction. The corpus
annotated for testing the hypothesis is composed of episodes of a Brazilian TV Travel Series
critically acclaimed as an exemplar of good practices in audiovisual composition. The TV genre
chosen also configures a novel experimental setting for research on integrated image and text
comprehension, since, in this corpus, text is not a direct description of the image sequence but
correlates with it indirectly in a myriad of ways. The dissertation also reports on an eye-tracker
experiment conducted to validate the approach proposed to a text-oriented annotation. The
experiment demonstrated that it is not possible to determine that text impacts gaze directly and
was taken as a reinforcement to the approach of valorizing modes combination. Last, we present
the Frame2 dataset, the product of the annotation task carried out for the corpus following both
the methodology and guidelines proposed. The results achieved demonstrate that, at least for
this TV genre but possibly also for others, a fine-grained semantic annotation tackling the
diverse correlations that take place in a multimodal setting provides new perspective in
multimodal comprehension modeling. Moreover, multimodal annotation also enriches the
development of FrameNets, to the extent that correlations found between modalities can attest
the modeling choices made by those building frame-based resources.Análises multimodais vêm crescendo em importância em várias abordagens da
LinguĂstica Cognitiva e em diversas áreas de aplicação, como o da CompreensĂŁo de Linguagem
Natural. No entanto, há significativa carência de representações semânticas refinadas de objetos
multimodais, especialmente em termos de integração de áreas como Processamento de
Linguagem Natural e Visão Computacional, que são fundamentais para a implementação de
multimodalidade no campo da LinguĂstica Computacional. Nesta tese, propomos uma
metodologia para estender o mĂ©todo de anotação da FrameNet ao domĂnio multimodal, uma
vez que a FrameNet pode fornecer representações semânticas refinadas, particularmente com
um banco de dados enriquecido por Qualia e outras relações interframe e intraframe, como é o
caso do FrameNet Brasil. Para tornar a FrameNet Brasil capaz de realizar análises multimodais,
delineamos a hipĂłtese de que, assim como as palavras em uma frase evocam frames e
organizam seus elementos na localidade sintática que os acompanha, os elementos visuais nos
planos de vĂdeo tambĂ©m podem evocar frames e organizar seus elementos na tela ou trabalhar
de forma complementar aos padrões de evocação de frames das sentenças narradas
simultaneamente ao seu aparecimento na tela, proporcionando diferentes perfis e opções de
perspectiva para a construção de sentido. O corpus anotado para testar a hipótese é composto
por episĂłdios de um programa televisivo de viagens brasileiro aclamado pela crĂtica como um
exemplo de boas práticas em composição audiovisual. O gênero televisivo escolhido também
configura um novo conjunto experimental para a pesquisa em imagem integrada e compreensĂŁo
textual, uma vez que, neste corpus, o texto não é uma descrição direta da sequência de imagens,
mas se correlaciona com ela indiretamente em uma mirĂade de formas diversa. A Tese tambĂ©m
relata um experimento de rastreamento ocular realizado para validar a abordagem proposta para
uma anotação orientada por texto. O experimento demonstrou que nĂŁo Ă© possĂvel determinar
que o texto impacta diretamente o direcionamento do olhar e foi tomado como um reforço para
a abordagem de valorização da combinação de modos. Por fim, apresentamos o conjunto de
dados Frame2, produto da tarefa de anotação realizada para o corpus seguindo a metodologia e
as diretrizes propostas. Os resultados obtidos demonstram que, pelo menos para esse gĂŞnero de
TV, mas possivelmente também para outros, uma anotação semântica refinada que aborde as
diversas correlações que ocorrem em um ambiente multimodal oferece uma nova perspectiva
na modelagem da compreensão multimodal. Além disso, a anotação multimodal também
enriquece o desenvolvimento de FrameNets, na medida em que as correlações encontradas entre
as modalidades podem atestar as escolhas de modelagem feitas por aqueles que criam recursos
baseados em frames.CAPES - Coordenação de Aperfeiçoamento de Pessoal de NĂvel Superio
A Probabilistic Multimedia Retrieval Model and its Evaluation
We present a probabilistic model for the retrieval of multimodal documents. The model is based on Bayesian decision theory and combines models for text-based search with models for visual search. The textual model is based on the language modelling approach to text retrieval, and the visual information is modelled as a mixture of Gaussian densities. Both models have proved successful on various standard retrieval tasks. We evaluate the multimodal model on the search task of TREC′s video track. We found that the disclosure of video material based on visual information only is still too difficult. Even with purely visual information needs, text-based retrieval still outperforms visual approaches. The probabilistic model is useful for text, visual, and multimedia retrieval. Unfortunately, simplifying assumptions that reduce its computational complexity degrade retrieval effectiveness. Regarding the question whether the model can effectively combine information from different modalities, we conclude that whenever both modalities yield reasonable scores, a combined run outperforms the individual runs
Music - Media - History: Re-Thinking Musicology in an Age of Digital Media
Music and sound shape the emotional content of audio-visual media and carry different meanings. This volume considers audio-visual material as a primary source for historiography. By analyzing how the same sounds are used in different media contexts at different times, the contributors intend to challenge the linear perspective of (music) history based on canonic authority. The book discusses AV-Documents (analysis in context), methodological questions (implications for research, education, and popularization of knowledge), archives of cultural memory (from the perspective of Cultural Studies) as well as digitalization and its consequences (organization of knowledge)
Cinematic Histospheres
In this Open Access book, film scholar Rasmus Greiner develops a theoretical model for the concept of the histosphere to refer to the “sphere” of a cinematically modelled, physically experienceable historical world. His analysis of practices of modelling and perceiving, immersion and empathy, experience and remembering, appropriation and refiguration, combine approaches from film studies, such as Vivian Sobchack’s phenomenology of film experience, with historiographic theories, such as Frank R. Ankersmit’s concept of historical experience. Building on this analysis, Greiner examines the spatial and temporal organization of historical films and presents discussions of mood and atmosphere, body and memory, and genre and historical consciousness. The analysis is based around three historical films, spanning six decades, that depict 1950s Germany: Helmut Käutner’s Sky Without Stars (1955), Jutta Brückner’s Years of Hunger (1980), and Sven Bohse’s three-part TV series Ku’damm 56 (2016)
Recommended from our members
Understanding image-text relations and news values for multimodal news analysis
The analysis of news dissemination is of utmost importance since the credibility of information and the identification of disinformation and misinformation affect society as a whole. Given the large amounts of news data published daily on the Web, the empirical analysis of news with regard to research questions and the detection of problematic news content on the Web require computational methods that work at scale. Today's online news are typically disseminated in a multimodal form, including various presentation modalities such as text, image, audio, and video. Recent developments in multimodal machine learning now make it possible to capture basic “descriptive” relations between modalities–such as correspondences between words and phrases, on the one hand, and corresponding visual depictions of the verbally expressed information on the other. Although such advances have enabled tremendous progress in tasks like image captioning, text-to-image generation and visual question answering, in domains such as news dissemination, there is a need to go further. In this paper, we introduce a novel framework for the computational analysis of multimodal news. We motivate a set of more complex image-text relations as well as multimodal news values based on real examples of news reports and consider their realization by computational approaches. To this end, we provide (a) an overview of existing literature from semiotics where detailed proposals have been made for taxonomies covering diverse image-text relations generalisable to any domain; (b) an overview of computational work that derives models of image-text relations from data; and (c) an overview of a particular class of news-centric attributes developed in journalism studies called news values. The result is a novel framework for multimodal news analysis that closes existing gaps in previous work while maintaining and combining the strengths of those accounts. We assess and discuss the elements of the framework with real-world examples and use cases, setting out research directions at the intersection of multimodal learning, multimodal analytics and computational social sciences that can benefit from our approach
- …