When I Look into Your Eyes: A Survey on Computer Vision Contributions for Human Gaze Estimation and Tracking

The automatic detection of eye positions, their temporal consistency, and their mapping into a line of sight in the real world (to find where a person is looking at) is reported in the scientific literature as gaze tracking. This has become a very hot topic in the field of computer vision during the last decades, with a surprising and continuously growing number of application fields. A very long journey has been made from the first pioneering works, and this continuous search for more accurate solutions process has been further boosted in the last decade when deep neural networks have revolutionized the whole machine learning area, and gaze tracking as well. In this arena, it is being increasingly useful to find guidance through survey/review articles collecting most relevant works and putting clear pros and cons of existing techniques, also by introducing a precise taxonomy. This kind of manuscripts allows researchers and technicians to choose the better way to move towards their application or scientific goals. In the literature, there exist holistic and specifically technological survey documents (even if not updated), but, unfortunately, there is not an overview discussing how the great advancements in computer vision have impacted gaze tracking. Thus, this work represents an attempt to fill this gap, also introducing a wider point of view that brings to a new taxonomy (extending the consolidated ones) by considering gaze tracking as a more exhaustive task that aims at estimating gaze target from different perspectives: from the eye of the beholder (first-person view), from an external camera framing the beholder’s, from a third-person view looking at the scene where the beholder is placed in, and from an external view independent from the beholder

MULTIMEDIALITÀ WEB E DIDATTICA DELLA PROSODIA

La presenza del Web e l’introduzione di tecnologie informatiche di tipo immersivo e collaborativo nella didattica delle lingue straniere servono a ricreare un ambiente cognitivo tipico di un contesto extra-scolastico, nel tentativo di superare quel gap esistente tra apprendimento formale e apprendimento spontaneo. E’ importante prestare attenzione alle ‘modalità’ con cui le tecnologie contribuiscono a creare un ambiente di apprendimento, in cui gli aspetti cognitivi e sociali dell’interazione uomo-macchina si intrecciano formando un unico contesto operativo. In questa prospettiva la multimedialità avvolgente del Web opera una sensibilizzazione diversa delle varie parti della macchina cerebrale modificandone lentamente procedure di percezione e strategie cognitive, quasi fosse lo stimolo costante da parte di un habitat naturaliter multimediale. Alla base di questa prospettiva c’è una relazione stretta tra grammatiche dei media, funzionamento delle sensorialità umane e schemi cognitivi che le influenzano. L’obiettivo del progetto di ricerca è quello di sperimentarne la ricaduta nella didattica della lingua seconda/straniera (L2/LS) ad adulti, in particolare dell’italiano come lingua straniera, limitatamente all’aspetto dialogico e prosodico/intonativo attraverso il training nella capacità di ascolto-produzione dei suoni e dei pattern intonativi con il supporto degli speech analysis tool e della multimedialità Web. L’ipotesi principale su cui è basata la ricerca è che questa tecnologia, spesso di tipo individuale, nelle classi di lingua ottimizzi le occasioni di apprendimento immersivo e di gruppo, recuperando ed estendendo modalità di conoscenza linguistica della L1 nella didattica della L2 e rispettando le indicazioni del natural approach. I risultati, raccolti durante la sperimentazione, usando materiali autentici con studenti principianti della National University of Ireland - Galway, indicano che l’uso di un feedback audio-visivo aiuta gli studenti a migliorare la loro produzione in LS e ad avvicinarsi alla frase obiettivo, grazie all’azione immediata di un’immagine delle differenze tra L1 e LS. Il campo di ricerca è interdisciplinare sia all’interno della linguistica sia del Computer Assisted Language Learning. Il settore della sperimentazione riguarda l’insegnamento dell’italiano LS in modalità blended learning, a metà tra la formazione in presenza e quella a distanza, attraverso l’utilizzo integrato del metodo percettivo e di quello strumentale nell’acquisizione dell’intonazione.The presence of the Web and the introduction of immersive and collaborative computer technology in the teaching of foreign languages can help to re-create a relaxed cognitive environment and overcome the gap between formal and spontaneous learning. However, it is important to consider the 'mode' in which technologies help to create a learning environment in which the cognitive and social aspects of the human - machine interaction are intertwined to form a single operating environment. In this perspective the enveloping multimediality of the Web stimulates different types of awareness in the various parts of the brain slowly modifying processes of perception and cognitive strategies, and submitting them to constant stimuli from a multimedial habitat naturaliter. In this perspective, there is a close relationship between the grammars of the media, the functioning of the human sensory systems and the cognitive schemata that influence them. The aim of this research study is to apply collaborative computer technology and multimedial environments to adult Foreign Language (FL) and Second Language (SL) teaching (in particular Italian as a foreign language), in order to develop and enhance dialogical and prosodic/intonational awareness through training in listening/production of sounds and intonation patterns with the support of speech analysis tools. The main hypothesis on which the research is based, is that technology, normally used in individual training, can in fact maximize opportunities for immersive group learning, by recovering and extending modes of L1 language knowledge into SL teaching and learning, in accordance with Krashen’s theory of natural approach. The data collected during the trial, which used authentic oral texts with ab initio students from the National University of Ireland - Galway, indicate that the implementation of audio-visual feedback helps learn¬ers to improve their FL production and to get closer to the target utterance. This is done through the support of an immediate and easy-to-read visual image of the differences between L1 and FL. The field of research of this paper is interdisciplinary and involves both Linguistics and Computer Assisted Language Learning. The field of the experiment concerns the teaching of Italian as FL in blended learning, therefore halfway between face-to-face and distance learning, through the integrated use of the perceptive and the instrumental method in the acquisition of intonation

Un modèle pour la gestion et la capitalisation d'analyses de traces d'activités en interaction collaborative

We present our three main results in adressing the problem of assisting the socio-cognitive analysis of human interaction. First, we propose a description of the process of analysis of such data, as well as a generic artefact which covers a large number of the analytic artefacts we have observed and which we call a replayable. Second, we present a study and a modelling of replayables, and describe the four fundamental operations which can be applied to them: synchronisation, visualisation, transformation and enrichment. Finally, we describe the implementation of this model in an environment that assists analysis through the manipulation of replayables, which we evaluate in real-life research situations. Tatiana (http://code.google.com/p/tatiana), the resulting software environment, is based on these four operations and integrates numerous possibilities for extending these operations to adapt to new kinds of analysis while staying within the analytic framework afforded by replayables.Nous présentons nos trois résultats principaux face à la difficulté d'assister l'analyse socio-cognitive d'interactions humaines. D'une part, nous proposons une description du processus d'analyse de ce genre données ainsi qu'un artefact générique permettant de recouvrir un grand nombre d'artefacts analytiques que nous avons pu observer et que nous nommons rejouable. D'autre part, nous présentons une étude et modélisation informatique des rejouables, et décrivons quatre opérations fondamentales qui peuvent s'y appliquer : synchronisation, visualisation, transformation et enrichissement. Enfin, nous décrivons l'implémentation de cette modélisation dans un environnement d'aide à l'analyse par manipulation de rejouables que nous évaluons dans des situations de recherche réelles. Tatiana (http://code.google.com/p/tatiana), l'environnement logiciel résultant, est basé sur ces quatre opérations et permet l'extension de ces opérations pour s'adapter à de nouvelles formes d'analyse

What the body knows about teaching music. The specialist preschool music teacher's pedagogical content knowing regarding teaching and learning rhythm skills viewed from an embodied cognition perspective

This thesis presents an investigation into the pedagogical content knowing (PCKg) of Dutch experienced specialist preschool music teachers with regard to teaching and learning rhythm skills viewed from an embodied cognition perspective. An embodied cognition perspective stresses the intimate relationship between body, mind and environment. In a multiple case study the research methods – stimulated recall interviews, gesture analysis tasks, physical action analysis tasks, notebooks and semi-structured interviews – were used to elicit the PCKg of six specialist preschool music teachers regarding rhythm skills. The data of these different methods were inductively analysed but sensitising concepts derived from the literature review on PCKg were also used in the analysis. Furthermore, the data were triangulated to gain a comprehensive understanding of the participants’ PCKg. As for the nature of the specialist preschool music teachers’ PCKg regarding rhythm skills the findings illustrated that PCKg is distributed over language, sound, gestures, body positioning and physical actions. Respecting the content of PCKg a new form of (non-verbal) knowledge was explored: “musical communication and musical interaction” that facilitates the learning of rhythm skills of preschoolers. The study is first of all significant for offering a new perspective on the nature of the specialist preschool music teachers’ PCKg: a multimodal and dynamic way of knowing that emerges from the interrelated role between the social, cultural and physical classroom environment, the teaching task and the teacher’s body. Beyond the classroom, these teachers’ bodies form a source for recalling, re-enacting and eliciting classroom experiences to develop and communicate their PCKg. Secondly, it offers a new perspective on the content of the specialist preschool music teachers’ PCKg: these teachers’ bodies take on different roles to mediate the preschoolers’ learning process regarding rhythm skills. These findings have implications for further research, teacher education, practice and policy

The interactive ecology of construal in gesture: a microethnographic analysis of peer learning at an EMI university in China

Depictive manual gestures do not appear in isolation, but are motivated by a complex of experiential knowledge, communicative goals, and contextual-environmental factors (Harrison 2018; Kendon 2004; Müller 2014; Streeck 1993, 1994, 2009b). However, little is known about the incremental, moment-by-moment formulation of depictions in elaborate sequences of talk. Furthermore, questions endure about depiction as a learning resource within the contingent interactivity of the foreign language academic classroom. This study explores these questions in the context of subject-related student talk at a Sino-foreign university in China by focusing on how gesturers build expositions through intercorporeal and intersubjective sense making (cf. Merleau-Ponty 1945/2012). Drawing on empirical material from the corpus of Chinese Academic Written and Spoken English (CAWSE), I aim to contribute greater understanding of the intersubjective ecology of depictive gesturing. The study builds on previous research on depictive gestures in the classroom (e.g. Rosborough 2014; Roth & Lawless 2002) by focusing on sequences of gesturing within two distinct classroom tasks: i) dialogic explanations of complex systems and ii) interactional multi-party group discussions. By converging theories of intersubjectivity drawing on Cognitive Grammar (e.g. Langacker 2008; Blomberg & Zlatev 2014) and Conversation Analysis (Heritage & Atkinson 1984; Schegloff 1992), I use microethnography for the investigation of gesture as a cognitive practice (Streeck 2009b; cf. Erickson 1995; Streeck & Mehus 2005). The analysis engages concepts in phenomenology, ecological cognition and enactivism in order to illustrate the publicly displayable achievement of enactive construal in spoken exposition. These analyses expose the ways that speakers depict for intersubjective visualization of the topic-at-hand, and anticipate and react to affordances that occur within the landscape of interaction. Speakers design their depictions, by manipulating construal dimensions in three ways: i) depictions are integrated into the exposition for projecting and delimiting epistemic arenas where construal relations are tailored for specific structural aspects of the depictions, ii) depictions invite participatory frameworks for co-analysis of the topic-at-hand, and iii) speakers refashion their depictions to anticipate previous trouble. Furthermore, the analysis of the interactional order of the tasks illustrates the intercorporeality, the pre-reflective disposition towards sense-making, of construal in the moment-by-moment construction of academic classroom talk. This study has implications that problematize the notion of the body as a communicative resource by obscuring the notions of planning and strategy. Overall, the analysis shows that explanations and discussions involve finely grained attenuation of the corporeal dimensions of spoken language

Le discours explicatif de l'enseignant en classe de langue Contextes interactionnels et processus cognitifs

This research finds its origin in the difficulties encountered by all teachers when explaining in classrooms of French as a Native Language or French as a Foreign and Second Language. As the teacher's oral explanatory discourse has not yet been the subject of comparative studies involving these two domains, we have collected corpora of oral pedagogic discourses in classrooms of native early learners (French secondary schools) and foreign young adults (B1-B2 level in homoglot environment). Thus, this thesis aims to present an experimentation methodology for examining the teacher's explanatory discourse by using the cognitive-didactic approach along with interactional linguistics' concepts and lexicosemantic tools. Initially, we will define explanatory discourse in general, as well as vocabulary teaching in context, in particular to non-beginners. Then, by applying our specific observation methodology, we will analyze explanatory discourse functioning in interactional context and will identify various teachers' explanatory techniques aiming the vocabulary comprehension. Furthermore, we were interested in learners' comprehension of lexical explanations. The impacts of our study could be involved in teacher training programs by drawing their attention to the stakes of explanatory interactions at learners' acquisition of a new linguistic knowledge.Notre recherche trouve son origine dans les difficultés rencontrées par tout enseignant à expliquer en classe de langue de français langue maternelle (FLM) ou de français langue étrangère et seconde (FLES). Le discours explicatif oral de l'enseignant n'ayant pas fait l'objet d'étude comparative au regard des deux disciplines énoncées précédemment, nous avons recueilli des corpus de discours pédagogiques s'adressant à de jeunes natifs (au collège et lycée français) et à de jeunes adultes étrangers (de niveau B1-B2, en milieu homoglotte). Cette thèse a alors pour objectif de présenter une méthodologie d'expérimentation qui permet d'étudier le discours explicatif de l'enseignant en utilisant les concepts de la didactique cognitive des langues et en s'appuyant sur les outils de la linguistique interactionnelle et de la lexico-sémantique. En premier lieu, nous définirons ce qu'est un discours explicatif en classe de langue en général et ce que signifie expliquer du lexique à des apprenants non-débutants en particulier. Ensuite, à l'aide d'une méthodologie d'observation élaborée pour l'étude, nous analyserons le fonctionnement du discours explicatif en contexte interactionnel et identifierons les différentes techniques utilisées par les enseignants pour arriver à la compréhension du lexique. De surcroît, nous nous sommes intéressée au fonctionnement de la compréhension des explications lexicales par les apprenants. Les retombées de notre étude peuvent s'envisager dans la formation des futurs enseignants en attirant leur attention sur les enjeux des interactions explicatives dans l'acquisition de nouvelles connaissances linguistiques par l'apprenant