Structuring lecture videos for distance learning applications. ISMSE

This paper presents an automatic and novel approach in structuring and indexing lecture videos for distance learning applications. By structuring video content, we can support both topic indexing and semantic querying of multimedia documents. In this paper, our aim is to link the discussion topics extracted from the electronic slides with their associated video and audio segments. Two major techniques in our proposed approach include video text analysis and speech recognition. Initially, a video is partitioned into shots based on slide transitions. For each shot, the embedded video texts are detected, reconstructed and segmented as high-resolution foreground texts for commercial OCR recognition. The recognized texts can then be matched with their associated slides for video indexing. Meanwhile, both phrases (title) and keywords (content) are also extracted from the electronic slides to spot the speech signals. The spotted phrases and keywords are further utilized as queries to retrieve the most similar slide for speech indexing. 1

Probabilistic temporal multimedia datamining

Ph.DDOCTOR OF PHILOSOPH

Payment in Credit: Copyright Law and Subcultural Creativity

Copyright lawyers talk and write a lot about the uncertainties of fair use and the deterrent effects of a clearance culture on publishers, teachers, filmmakers, and the like, but know less about the choices people make about copyright on a daily basis, especially when they are not working. Here, Tushnet examines one subcultural group that engages in a variety of practices, from pure copying and distribution of others\u27 works to creation of new stories, art, and audiovisual works: the media-fan community. Among other things, she discusses some differences between fair use and fan practices, focused around attribution as an alternative to veto rights over uses of copyrighted works

ENGLISH READING ABILITY IN YOUNG DEAF SIGNERS AN INVESTIGATION OF SENTENCE COMPREHENSION

We performed a detailed investigation of the robust correlation between ASL and English reading ability in 54 deaf students aged 7;3 to 19;0. Skilled and unskilled signers were assessed on four English sentence structures (actives, passives, pronouns, reflexive pronouns) using a four-alternative forced choice sentence-to-picture-matching task, providing a window into how ASL skill is related to English sentence comprehension. Of interest was the extent to which proficiency in LI provided a foundation for L2 learning as predicted by Cummins’ developmental interdependence hypothesis. Skilled signers outperformed unskilled signers on all sentence types. Error analysis indicated greater word recognition difficulties in unskilled signers. Syntactic structures mapping directly from LI to L2 were more accurately understood than structures mapping in less obvious ways, consistent with MacWhinney’s unified competition model. Our findings provide evidence that increased ASL ability supports English sentence comprehension at the levels of individual words and syntax

CAN ATTENTIONAL PATTERNS PREDICT THE FUTURE? A NOVEL VIEW INTO WHO IS LIKELY TO EXPERIENCE CHANGE BLINDNESS

60 pagesPeople witnessing identical streams of information can experience that information very differently. This phenomenon was strikingly documented in a famous psychological experiment: one group of research participants watching a video of a crowded area failed to notice a man in a gorilla suit meander across the room, although another group described the man in the gorilla suit as the most salient aspect of the video. How do we account for such diversity in experience? My research investigates this general question via a new technique: the dwell-time paradigm, in which viewers advance at their own pace through slideshows depicting dynamic events while the time they spend dwelling on each image is measured. We hypothesize that patterns of dwelling across time will clarify which aspects of events viewers are prioritizing in their processing, and thus we will be able to predict – well in advance -- who will subsequently report salient features of interest. Following the completion of two different studies, analysis of the results of this research indicate that dwell-time patterns do in fact provide predictive information about the probability a viewer will be subject to change blindness. This finding has wide reaching implications. Specifically, it will be possible to utilize dwell-time patterns across a range of situations where monitoring the focus and adequacy of people’s attention is crucial. For example, applications could include a) refinements to diagnosis in those with attentional impairments, and b) the creation of systems that alert people when their attentional patterns have become suboptimal for an essential task, such as drivers, train operators, and pilots

Got rhythm? Better inhibitory control is linked with more consistent drumming and enhanced neural tracking of the musical beat in adult percussionists and nonpercussionists

Musical rhythm engages motor and reward circuitry that is important for cognitive control, and there is evidence for enhanced inhibitory control in musicians. We recently revealed an inhibitory control advantage in percussionists compared with vocalists, highlighting the potential importance of rhythmic expertise in mediating this advantage. Previous research has shown that better inhibitory control is associated with less variable performance in simple sensorimotor synchronization tasks; however, this relationship has not been examined through the lens of rhythmic expertise. We hypothesize that the development of rhythm skills strengthens inhibitory control in two ways: by fine-tuning motor networks through the precise coordination of movements “in time” and by activating reward-based mechanisms, such as predictive processing and conflict monitoring, which are involved in tracking temporal structure in music. Here, we assess adult percussionists and nonpercussionists on inhibitory control, selective attention, basic drumming skills (self-paced, paced, and continuation drumming), and cortical evoked responses to an auditory stimulus presented on versus off the beat of music. Consistent with our hypotheses, we find that better inhibitory control is correlated with more consistent drumming and enhanced neural tracking of the musical beat. Drumming variability and the neural index of beat alignment each contribute unique predictive power to a regression model, explaining 57% of variance in inhibitory control. These outcomes present the first evidence that enhanced inhibitory control in musicians may be mediated by rhythmic expertise and provide a foundation for future research investigating the potential for rhythm-based training to strengthen cognitive function

Comprehension in-situ: how multimodal information shapes language processing

The human brain supports communication in dynamic face-to-face environments where spoken words are embedded in linguistic discourse and accompanied by multimodal cues, such as prosody, gestures and mouth movements. However, we only have limited knowledge of how these multimodal cues jointly modulate language comprehension. In a series of behavioural and EEG studies, we investigated the joint impact of these cues when processing naturalistic-style materials. First, we built a mouth informativeness corpus of English words, to quantify mouth informativeness of a large number of words used in the following experiments. Then, across two EEG studies, we found and replicated that native English speakers use multimodal cues and that their interactions dynamically modulate N400 amplitude elicited by words that are less predictable in the discourse context (indexed by surprisal values per word). We then extended the findings to second language comprehenders, finding that multimodal cues modulate L2 comprehension, just like in L1, but to a lesser extent; although L2 comprehenders benefit more from meaningful gestures and mouth movements. Finally, in two behavioural experiments investigating whether multimodal cues jointly modulate the learning of new concepts, we found some evidence that presence of iconic gestures improves memory, and that the effect may be larger if information is presented also with prosodic accentuation. Overall, these findings suggest that real-world comprehension uses all cues present and weights cues differently in a dynamic manner. Therefore, multimodal cues should not be neglected for language studies. Investigating communication in naturalistic contexts containing more than one cue can provide new insight into our understanding of language comprehension in the real world

Linguistic Features of Lectures and their Relationship with Student Performance

Lectures are an important part of the post-secondary experience. Optimizing various aspects of this experience for the benefit of students’ learning has been examined (Mayer, 2019). However, the linguistic features of lectures and how these features might affect student learning have been overlooked in the extant literature. Recent studies have utilised Coh-Metrix, an automated text analyzer, to examine discourse in both texts and lecture discourse (Graesser, McNamara, & Kulikowich 2011; McNamara, Graesser, McCarthy, & Cai, 2014; Medimorec, Palvik Jr, Oleny, Gaesser, & Risko, 2015; Morgan, Burkett, Bagley, Graesser, 2011). We extend this effort here by analyzing linguistic features of lectures and how they are associated with student performance. In particular, we were interested in determining whether (a) computationally generated measures of language are associated with student performance and (b) whether different associations are observed with different testing methods (multiple-choice vs. short answer). We demonstrate that a lecturer's narrativity, syntactic simplicity, and referential cohesion are associated with performance on multiple-choice tests. Preliminary results suggest a different pattern of association for short answer tests