Real-time activity recognition by discerning qualitative relationships between randomly chosen visual features

In this paper, we present a novel method to explore semantically meaningful visual information and identify the discriminative spatiotemporal relationships between them for real-time activity recognition. Our approach infers human activities using continuous egocentric (first-person-view) videos of object manipulations in an industrial setup. In order to achieve this goal, we propose a random forest that unifies randomization, discriminative relationships mining and a Markov temporal structure. Discriminative relationships mining helps us to model relations that distinguish different activities, while randomization allows us to handle the large feature space and prevents over-fitting. The Markov temporal structure provides temporally consistent decisions during testing. The proposed random forest uses a discriminative Markov decision tree, where every nonterminal node is a discriminative classifier and the Markov structure is applied at leaf nodes. The proposed approach outperforms the state-of-the-art methods on a new challenging video dataset of assembling a pump system

Coarse Temporal Attention Network (CTA-Net) for Driver’s Activity Recognition

There is significant progress in recognizing traditional human activities from videos focusing on highly distinctive actions involving discriminative body movements, body-object and/or human-human interactions. Driver's activities are different since they are executed by the same subject with similar body parts movements, resulting in subtle changes. To address this, we propose a novel framework by exploiting the spatiotemporal attention to model the subtle changes. Our model is named Coarse Temporal Attention Network (CTA-Net), in which coarse temporal branches are introduced in a trainable glimpse network. The goal is to allow the glimpse to capture high-level temporal relationships, such as 'during', 'before' and 'after' by focusing on a specific part of a video. These branches also respect the topology of the temporal dynamics in the video, ensuring that different branches learn meaningful spatial and temporal changes. The model then uses an innovative attention mechanism to generate high-level action specific contextual information for activity recognition by exploring the hidden states of an LSTM. The attention mechanism helps in learning to decide the importance of each hidden state for the recognition task by weighing them when constructing the representation of the video. Our approach is evaluated on four publicly accessible datasets and significantly outperforms the state-of-the-art by a considerable margin with only RGB video as input.Comment: Extended version of the accepted WACV 202

Trying to break new ground in aerial archaeology

Aerial reconnaissance continues to be a vital tool for landscape-oriented archaeological research. Although a variety of remote sensing platforms operate within the earth’s atmosphere, the majority of aerial archaeological information is still derived from oblique photographs collected during observer-directed reconnaissance flights, a prospection approach which has dominated archaeological aerial survey for the past century. The resulting highly biased imagery is generally catalogued in sub-optimal (spatial) databases, if at all, after which a small selection of images is orthorectified and interpreted. For decades, this has been the standard approach. Although many innovations, including digital cameras, inertial units, photogrammetry and computer vision algorithms, geographic(al) information systems and computing power have emerged, their potential has not yet been fully exploited in order to re-invent and highly optimise this crucial branch of landscape archaeology. The authors argue that a fundamental change is needed to transform the way aerial archaeologists approach data acquisition and image processing. By addressing the very core concepts of geographically biased aerial archaeological photographs and proposing new imaging technologies, data handling methods and processing procedures, this paper gives a personal opinion on how the methodological components of aerial archaeology, and specifically aerial archaeological photography, should evolve during the next decade if developing a more reliable record of our past is to be our central aim. In this paper, a possible practical solution is illustrated by outlining a turnkey aerial prospection system for total coverage survey together with a semi-automated back-end pipeline that takes care of photograph correction and image enhancement as well as the management and interpretative mapping of the resulting data products. In this way, the proposed system addresses one of many bias issues in archaeological research: the bias we impart to the visual record as a result of selective coverage. While the total coverage approach outlined here may not altogether eliminate survey bias, it can vastly increase the amount of useful information captured during a single reconnaissance flight while mitigating the discriminating effects of observer-based, on-the-fly target selection. Furthermore, the information contained in this paper should make it clear that with current technology it is feasible to do so. This can radically alter the basis for aerial prospection and move landscape archaeology forward, beyond the inherently biased patterns that are currently created by airborne archaeological prospection

Learning to Code: Effects of Programming Modality in a Game-based Learning Environment

As new introductory block-based coding applications for young students to learn basic computer science concepts, such as, loops and conditionals, continue to increase in popularity, it is necessary to consider the best method of teaching students these skills. Many of these products continue to exhibit programmatic misconceptions of these concepts and many students struggle with how to apply what they learn to a text-based format due to the difficulties with learning the syntactic structure not present in block-based programming languages. If the goal of teaching young students how to program is meant to develop a set of skills they may apply when learning more complex programming languages, then discerning how they are introduced to those practices is imperative. However, few studies have examined how the specific modality in which students are taught to program effects how they learn and what skills they develop. More specifically, research has yet to effectively investigate modality in the context of an educational coding game where the modality feature is controlled, and content is consistent throughout game-play. This is mainly due to the lack of available games with this feature designed into the application. This dissertation explores whether programming modality effects how well students can learn and transfer computer science concepts and practices from an educational programming game. I proposed that by being guided from a blocks-based to text-based programming language would instill a deeper understanding of basic computer science concepts and would support learning and improve transfer and performance on new challenging tasks. Two experimental studies facilitated game-play sessions on the developed application for this project. The first study was a 2x2 between subjects design comparing educational module (game versus basic) and programming modality (guided versus free choice). The findings from Study 1 informed the final version design for the module used in the second study where only the game module was used in order to focus the comparison between programming modality. Findings showed that students who coded using the game module performed better on a learning test. Study 2 results showed that students who are transitioned from blocks-based to text-based programming language learn basic computer science concepts with greater success than those with the free choice modality. A comparative study was conducted using quantitative data from learning measures and qualitative video data from the interviews during the challenge task of the second study. This study examined how students at the extreme levels of performance utilized the toggle switch feature during game-play and how the absence of the feature impacted how they completed the challenge task. This analysis showed two different methods of toggle switch usage being implemented by a high and low performing student. The high performing student utilized the resources more often during the challenge tasks in lieu of leveraging the toggle switch and were still able to submit high level code. Results suggest that a free choice student who uses the feature as a tool to check their prewritten code rather than a as short cut for piecing code together as blocks and submitting the text upon the final attempt. This practice leads to a shallower understanding of the basic concepts and make it extremely difficult to expand and apply that knowledge to a more difficult task. This dissertation includes five chapters: an introduction and theoretical framework, a game design framework and implementation description, two experimental investigations, and a quantitative and qualitative comparative analysis. Chapter one provides the conceptual and theoretical framework for the two experimental investigations. Chapter two describes the theory and design structure for the game developed for this dissertation work. Chapter three and four will discuss the effects of programming modality on learning outcomes. Specifically, chapter 3 focuses on implications of programming modality when determining how to implement changes for the design of the game for Study 2. Chapter five discusses a comparative analysis that investigated differing work flow patterns within the free choice condition between high and low performing students. Results from these three chapters illustrate the importance of examining this component of the computer science education process in supplemental games for middle and high school students. Additionally, this work contributes in furthering the investigation of these educational games and discusses implications for design of similar applications

An EEG-based neural decoding approach for investigating statistical learning between modalities

We researched cross-modal statistical learning conducting to experiments: a behavioural one and a neuroimaging one. In the analysis of the later we used neural decoding with temporal generalization

Communicating Multiplicative Reasoning Through Semiotic Resources

The importance of fostering in students the requisite language to understand what is being communicated and how to communicate their understanding requires educators to conceptualize themselves as teachers of language and content. It is possible to engage in activities of the mathematics classroom and through that participation engage in language practices and mathematical practices simultaneously. The purpose of this study was to explore the use of semiotic resources, and modality, with a student-generated tool on students’ communication of multiplicative reasoning. The study design was a qualitative case study that included a single third-grade class with an in-depth look at six students of varying knowledge levels. Two students, one male and one female, were randomly selected from Beyond, On, and Approaching levels. Discourse analysis served dual purposes for the data collected: first, it explored a socially constructed multi-modal tool utilized as an activity to enhance language use individually and interactively during mathematical discourse; second, it supported investigating the language used by participants during the studied activities and how they relate to Communication About and Communication In multiplication. The findings support the utilization of semiotic resources, inclusive of visual representations, signs, symbolic notations, and receptive and expressive language elements as fundamental to the learning and communication we are asking of our students. Through the interplay of semiotic resources, a multimodal student-generated tool can support students in summarizing their learning, individually and interactively, enhancing their means of communicating discursively in mathematics

The Effect of Gender on Spatial Ability and Spatial Reasoning Among Students in Grades 2-8

The purpose of this study was to examine gender differences across three types of spatial ability; namely, spatial perception, spatial visualization, and mental rotation in conjunction with working memory. The study utilized a causal-comparative research design involving group comparisons. In this design researchers collect data about variables that they have conceptualized to be in a causal relationship to each other, but there is no intervention as in experimental research. Participants in this study included approximately 200 students in second through eighth grades at one public school and one public charter school, all located in the same school district/county. Spatial ability was measured by four categories of spatial relations tests based upon spatial cognition research proposing that spatial cognition is comprised of “three separable dimensions:” the Mental Folding Test for Children (spatial visualization), an adaptation of the Differential Aptitude Test: Space Relations (DAT: SR), Mental Rotation for Children, an adaptation of the Mental Rotations Test (MRT), Manikin Test (spatial orientation and transformation), and Mr. Peanut Test (visuo-spatial working memory). The resultant scores were used as measures of mathematical achievement and cognitive ability. Data were analyzed using MANOVA and ANOVA statistical analysis. Results suggested that mostly non-significant differences exist for spatial visualization abilities between males and females. The sole example of a significant difference between male and females was noted on the Mr. Peanut test in the fourth and fifth grades, accompanied with a partial Eta Squared (ղ2) of .10