Periodic Motion Detection and Estimation via Space-Time Sampling

A novel technique to detect and localize periodic movements in video is presented. The distinctive feature of the technique is that it requires neither feature tracking nor object segmentation. Intensity patterns along linear sample paths in space-time are used in estimation of period of object motion in a given sequence of frames. Sample paths are obtained by connecting (in space-time) sample points from regions of high motion magnitude in the first and last frames. Oscillations in intensity values are induced at time instants when an object intersects the sample path. The locations of peaks in intensity are determined by parameters of both cyclic object motion and orientation of the sample path with respect to object motion. The information about peaks is used in a least squares framework to obtain an initial estimate of these parameters. The estimate is further refined using the full intensity profile. The best estimate for the period of cyclic object motion is obtained by looking for consensus among estimates from many sample paths. The proposed technique is evaluated with synthetic videos where ground-truth is known, and with American Sign Language videos where the goal is to detect periodic hand motions.National Science Foundation (CNS-0202067, IIS-0308213, IIS-0329009); Office of Naval Research (N00014-03-1-0108

Sign Spotting using Hierarchical Sequential Patterns with Temporal Intervals

This paper tackles the problem of spotting a set of signs occuring in videos with sequences of signs. To achieve this, we propose to model the spatio-temporal signatures of a sign using an extension of sequential patterns that contain temporal intervals called Sequential Interval Patterns (SIP). We then propose a novel multi-class classifier that organises different sequential interval patterns in a hierarchical tree structure called a Hierarchical SIP Tree (HSP-Tree). This allows one to exploit any subsequence sharing that exists between different SIPs of different classes. Multiple trees are then combined together into a forest of HSP-Trees resulting in a strong classifier that can be used to spot signs. We then show how the HSP-Forest can be used to spot sequences of signs that occur in an input video. We have evaluated the method on both concatenated sequences of isolated signs and continuous sign sequences. We also show that the proposed method is superior in robustness and accuracy to a state of the art sign recogniser when applied to spotting a sequence of signs.This work was funded by the UK government

Children and adults' understanding and use of sound-symbolism in novel words

Sound-symbolism is the inherent link between the sound of a word and its meaning. The aim of this thesis is to gain an insight into the nature of sound-symbolism. There are five empirical chapters, each of which aims to uncover children and adults’ understanding of sound-symbolic words. Chapter 1 is a literature review of sound-symbolism. Chapter 2 is a cross-linguistic developmental study looking at the acquisition of sound-symbolism. Chapter 3 looks at childrens use of sound-symbolism in a verb-learning task. Chapter 4 looks at childrens use of sound-symbolism when learning and memorising novel verbs. Chapter 5 consists of two experiments looking at what exact part of a word is sound-symbolic. This study compared different types of consonants and vowels, across a number of domains in an attempt to gain an understanding of the nature of sound-symbolism. Chapter 6 looks at the potential mechanisms by which sound-symbolism is understood. This study is a replication of previous research, which found that sound-symbolic sensitivity is increased when the word is said and not just heard. There are therefore a total of five empirical chapters each of which attempts to look at the nature of sound-symbolic meaning from a slightly different angle

Language and modality: Effects of the use of space in the agreement system of lengua de signos española (Spanish Sign Language)

393 p.Esta tesis examina la concordancia en la lengua de signos española (LSE) y ofrece una extensa descripción de los mecanismos que la gobiernan en base a datos recogidos de usuarios de LSE del País Vasco. Esta descripción brinda la posibilidad de comparar la concordancia en LSE con este mismo fenómeno en otras lenguas de signos, además de realizar una comparativa entre modalidades, es decir, una comparativa entre la concordancia en una lengua de signos y la concordancia en las lenguas orales. Esta comparativa nos obliga a preguntarnos si el término "concordancia" tiene el mismo significado cuando lo aplicamos a las lenguas de signos o a las lenguas orales. Así, nos permite profundizar en la cuestión de la modalidad e identificar aquellas propiedades del sistema de concordancia que son producto de la modalidad de la lengua. Por otro lado, es posible que las características comunes a los dos tipos de lenguas representen propiedades universales de lenguaje, sea cual sea la modalidad. Esta tesis pretende delinear un aspecto del uso de espacio en lengua de signos, pero queda mucho espacio por conquistar

WATCHING PEOPLE: ALGORITHMS TO STUDY HUMAN MOTION AND ACTIVITIES

Nowadays human motion analysis is one of the most active research topics in Computer Vision and it is receiving an increasing attention from both the industrial and scientific communities. The growing interest in human motion analysis is motivated by the increasing number of promising applications, ranging from surveillance, human–computer interaction, virtual reality to healthcare, sports, computer games and video conferencing, just to name a few. The aim of this thesis is to give an overview of the various tasks involved in visual motion analysis of the human body and to present the issues and possible solutions related to it. In this thesis, visual motion analysis is categorized into three major areas related to the interpretation of human motion: tracking of human motion using virtual pan-tilt-zoom (vPTZ) camera, recognition of human motions and human behaviors segmentation. In the field of human motion tracking, a virtual environment for PTZ cameras (vPTZ) is presented to overcame the mechanical limitations of PTZ cameras. The vPTZ is built on equirectangular images acquired by 360° cameras and it allows not only the development of pedestrian tracking algorithms but also the comparison of their performances. On the basis of this virtual environment, three novel pedestrian tracking algorithms for 360° cameras were developed, two of which adopt a tracking-by-detection approach while the last adopts a Bayesian approach. The action recognition problem is addressed by an algorithm that represents actions in terms of multinomial distributions of frequent sequential patterns of different length. Frequent sequential patterns are series of data descriptors that occur many times in the data. The proposed method learns a codebook of frequent sequential patterns by means of an apriori-like algorithm. An action is then represented with a Bag-of-Frequent-Sequential-Patterns approach. In the last part of this thesis a methodology to semi-automatically annotate behavioral data given a small set of manually annotated data is presented. The resulting methodology is not only effective in the semi-automated annotation task but can also be used in presence of abnormal behaviors, as demonstrated empirically by testing the system on data collected from children affected by neuro-developmental disorders

An Examination of the Influence of Age on L2 Acquisition of English Sound-Symbolic Patterns

A number of researchers (DeKeyser, 2012; J. S. Johnson & Newport, 1989; Long, 1990) have argued that age is a critical factor in second language acquisition. This conclusion is based on extensive research over the last two decades that has demonstrated age-related effects in learners’ nonnativelike acquisition of phonology, morphosyntax, pragmatics, and discourse-level features of language. In the wake of such findings, there has recently been an increased interest in determining the precise linguistic areas that are difficult for adult learners and the cognitive mechanisms implicated in age-related effects. Because implicit learning plays a key role in first-language (L1) acquisition, particularly in the acquisition of statistical patterns in language, it has been proposed that age effects may be the result of attenuated implicit learning capabilities in late-teen and adult learners (DeKeyser, 2000; Janacsek, Fiser, & Nemeth, 2012). If this is true, age-related effects should be significant in linguistic areas that are not readily amenable to conscious learning processes and explicit instruction. To determine whether this is in fact the case, this study examined the linguistic knowledge of native speakers (NSs), early L2 learners, and learners who acquired English as adults. In particular, it examined these groups’ knowledge related to an area of English that is hypothesized to be difficult to learn explicitly, namely, English sound-symbolic (SS) patterns. Participants were composed of English NSs (n = 20) and three NNS groups with L1 Korean and L2 of English. The NNS groups were divided into three groups based on age of onset (AO) , with an AO range from 3 to 9 years of age (n = 20), 10 to 16 (n = 20), and > 17 (n = 20). Three experiments were performed that tested the participants’ English magnitude SS sensitivities when forming assumptions about nonce words (Experiment 1 and 2) and their ability to utilize English SS patterns to bootstrap their learning of new vocabulary (Experiment 3). The two late L2 learner groups (AO 10-16; 17+) were found to have significantly reduced levels of SS knowledge compared to the early L2 learners (AO 3-9) and NSs in all experiments. Only in Experiment 1 and 2, the early L2 learners had diminished magnitude SS sensitivities compared to NSs, but not for Experiment 3. Explicit and implicit aptitudes as measured by LLAMA (Meara, 2005) were also tested for potential relationships with test scores. Explicit aptitudes (LLAMA B, E, and F) did not have a significant effect on the performance of all AO groups, whereas implicit aptitude (LLAMA D) did have a moderate to strong correlation for test scores in only the two late learner groups. The early learner group was not affected by language aptitude levels during the experiments. In sum, the study has found that there is evidence for SPE in the areas of magnitude and English phonesthemic SS patterns. Implicit language-learning aptitudes appeared to have a facilitative effect on the acquisition of these SS sensitivities for the two late L2 learner groups, but not for the early L2 learners