A Two-Level Approach to Characterizing Human Activities from Wearable Sensor Data

International audienceThe rapid emergence of new technologies in recent decades has opened up a world of opportunities for a better understanding of human mobility and behavior. It is now possible to recognize human movements, physical activity and the environments in which they take place. And this can be done with high precision, thanks to miniature sensors integrated into our everyday devices. In this paper, we explore different methodologies for recognizing and characterizing physical activities performed by people wearing new smart devices. Whether it's smartglasses, smartwatches or smartphones, we show that each of these specialized wearables has a role to play in interpreting and monitoring moments in a user's life. In particular, we propose an approach that splits the concept of physical activity into two sub-categories that we call micro-and macro-activities. Micro-and macro-activities are supposed to have functional relationship with each other and should therefore help to better understand activities on a larger scale. Then, for each of these levels, we show different methods of collecting, interpreting and evaluating data from different sensor sources. Based on a sensing system we have developed using smart devices, we build two data sets before analyzing how to recognize such activities. Finally, we show different interactions and combinations between these scales and demonstrate that they have the potential to lead to new classes of applications, involving authentication or user profiling

An Open Dataset for Human Activity Analysis using Smart Devices

The study of human mobility and activities has opened up to an incredible number of studies in the past, most of which included the use of sensors distributed on the body of the subject. More recently, the use of smart devices has been particularly relevant because they are already everywhere and they come with accurate miniaturized sensors. Whether it is smartphones, smartwatches or smartglasses, each device can be used to describe complementary information such as emotions, precise movements, or environmental conditions. In this short paper, we release the applications we have developed and an example of a collected dataset. We propose that opening multi-sensors data from daily activities may enable new approaches to studying human behavior

Le contrôle de l'interception d'un créneau en mouvement : Le cas de la traversée d'intersection

This Ph.D. project aims at studying drivers' road crossing behaviour under the perspective of visually guided actions. In the introduction, we present that former studies related to road crossing behaviour mainly focused on perceptual judgement processes. However, we demonstrate that this kind of task would be better understood with theories issued from the ecological approach of perception and motor control studies. Our literature review leads us to make a bridge between the task of crossing a road inside a free moving traffic gap and the task of intercepting an horizontally moving target. In this regard, it has been demonstrated that the Constant Bearing Angle strategy (CBA) has a high explanatory power in respect to experimental data. We used driving simulation tools in order to assess the hypothesis of the use of a bearing angle based strategy by drivers. In the first experimental chapter, we show that studying perceptual-motor coupling in this task is relevant and that observed behaviour is close to which observed in horizontally moving target interception tasks. The second experimental chapter demonstrates that drivers' behaviour relies both on visual information related to the moving gap (global contribution) and that related to the independent vehicles (local contributions). In the last experimental chapter, we evidenced that drivers' behaviour does not exclusively rely on the changes of the bearing angle but also on traffic vehicles' optical expansion. We conclude on how our results shed new light on former obtained results with perceptual judgements based methodology. Finally, we suggest further researches by considering experimental manipulation of drivers' action capabilities and we also consider how our results could be used in order to design driving assistance systems.L'objectif de cette thèse est d'étudier le comportement de traversée d'intersection en conduite automobile sous l'angle des actions visuellement guidées. Dans l'introduction, nous montrons que les principales études portant sur la traversée d'intersection se sont focalisées sur les processus de jugement perceptif ; or nous montrons que la compréhension de cette activité pourrait être affinée en utilisant les concepts de la théorie de la perception directe et ceux issus de l'étude du controle moteur. Notre revue de questions nous conduit à rapprocher la tache de traversée d'intersection à l'intérieur d'un créneau libre du trafic à celle d'interception de cible se déplaçant horizontalement. Or, dans le cadre des taches d'interception, il a été démontré que la stratégie d'angle de relèvement constant (i.e., Constant Bearing Angle, CBA) a un fort pouvoir explicatif vis-à-vis des données expérimentales. Nous avons donc tiré partie des outils de simulation automobile pour tester la plausibilité de l'utilisation d'une telle stratégie par les conducteurs. Dans le premier chapitre expérimental, nous montrons qu'il est légitime d'étudier le couplage perception-action dans ce type de tache et que les conducteurs adoptent des régulations comportementales proches de celles observées dans une tache d'interception de mobile. Le second chapitre expérimental nous permet de mettre en évidence l'utilisation conjointe par les conducteurs d'informations relatives au créneau du trafic (supports globaux) et aux véhicules (supports locaux). Enfin, dans le dernier chapitre expérimental, nous mettons en évidence que les conducteurs n'utilisent pas exclusivement le taux de changement de l'angle de relèvement comme information visuelle mais qu'ils utilisent aussi l'expansion optique des véhicules du trafic. Nous concluons ce travail en montrant comment nos résultats nous permettent d'interpréter sous un autre angle ceux obtenus avec une méthodologie fondée uniquement sur les jugements perceptifs. Enfin, nous proposons des perspectives expérimentales en envisageant la manipulation des capacités d'action des conducteurs ainsi que des perspectives appliquées en envisageant l'utilisation de nos résultats dans le contexte de la conception d'un système d'assistance à la conduite

Continuous control of the interception of a moving gap : The case of road crossing at intersection

L'objectif de cette thèse est d'étudier le comportement de traversée d'intersection en conduite automobile sous l'angle des actions visuellement guidées. Dans l'introduction, nous montrons que les principales études portant sur la traversée d'intersection se sont focalisées sur les processus de jugement perceptif ; or nous montrons que la compréhension de cette activité pourrait être affinée en utilisant les concepts de la théorie de la perception directe et ceux issus du contrôle continu. Notre revue de questions nous conduit à rapprocher la tâche de traversée d'intersection à l'intérieur d'un créneau libre du trafic à celle d'interception de cible se déplaçant horizontalement. Or, dans le cadre des tâches d'interception, il a été démontré que la stratégie d'angle de relèvement constant (i.e., Constant Bearing Angle, CBA) a un fort pouvoir explicatif vis-à-vis des données expérimentales. Nous avons donc tiré partie des outils de simulation automobile pour tester la plausibilité de l'utilisation d'une telle stratégie par les conducteurs. Dans le premier chapitre expérimental, nous montrons qu'il est légitime d'étudier le couplage perception-action dans ce type de tâche et que les conducteurs adoptent des régulations comportementales proches de celles observées dans une tâche d'interception de mobile. Le second chapitre expérimental nous permet de mettre en évidence l'utilisation conjointe par les conducteurs d'informations relatives au créneau du trafic (supports globaux) et aux véhicules (supports locaux).This Ph.D. project aims at studying drivers' road crossing behaviour under the perspective of visually guided actions. In the introduction, we present that former studies related to road crossing behaviour mainly focused on perceptual judgement processes. However, we demonstrate that this kind of task would be better understood with theories from the ecological approach of perception and motor control studies. Our literature review lead us to make a bridge between the task of crossing a road inside a free moving traffic gap and the task of intercepting an horizontally moving target. In this regard, it has been demonstrated that the Constant Bearing Angle strategy (CBA) has a high explanatory power in respect to experimental data. We used driving simulation tools in order to assess the hypothesis of the use of bearing angle based strategy by drivers. In the first experimental chapter, we show that studying perceptual-motor coupling in this task is relevant and that observed behaviour is close to taht observed in horizontally moving target interception tasks. The second experimental chapter demonstrates that drivers' behaviour relies both on visual information related to the moving gap (global contribution) and that related to the independent vehicles (local contributions). In the last experimental chapter, we evidenced that drivers' behaviour does not exclusively rely on the changes of the bearing angle but also on traffic vehicles' optical expansion. We conclude on how our results shed new light on former results obtained with perceptual-judgements based methodology

Does full-page view when reading comics improve understanding? Studying the effect of panels' visibility and reader's expertise on reading time and understanding of two comic books excerpts: Studying the effect of panels’ visibility and reader’s expertise on reading time and understanding of two comic books excerpts

International audience1)IntroductionComics (or mangas, bandes dessinées) are a popular narrative form in many countries. They are read for leisure but also for pedagogical purposes. Cognitive processes underlying their reading and understanding need more research, though. For those reasons, this is not surprising that the number of publications on the topic grew steadily in a decade. Indeed, understanding how readers effectively read a comic rises conceptual and methodological challenges. Particularly, when it comes to understand how the reader accesses the different layers of information in a page.To answer that question, Cohn’s study (2013) asked participants to sort panels, in real-world comics’ page layout, in the order they would read them. Those pages were emptied of their drawing and text content. Results showed that, while readers generally follow a Z-path to navigate panels, alternative paths are also possible depending on size or proximity variations. In their study, Mikkonen & Lautenbacher (2019) used eye-tracking to understand the effects of both layout and narrative content during comic book reading. They also found a predominance of the Z-shape pattern, although readers had a much less structured eye movement pattern at the very beginning and at the very end of each page. They made the hypothesis that readers use both a global and a local attention scales.However, this is not clear yet what it is the functional purpose of those eye movement patterns: are they merely erratic or do they serve understanding? Picking relevant information in a comic page surely involve expertise. Indeed, Nakazawa (2016) found that the ability to understand comics was modulated by expertise. In addition, Zhao & Mahrt (2018) showed that advanced readers have shorter eye fixations than novice readers and that advanced readers have more correct responses to the comic book comprehension test than novices. In this work, we tested two hypotheses. The first one is that “global” reading does serve a functional purpose for understanding. Thus, preventing such global reading will impair the reader’s performance. The second hypothesis is that reader’s familiarity with visual narrative forms will improve performance and modulate the effect of the latter manipulations.2)MethodologySixty participants took part to our study. They were to read two digitalized double-page comic excerpts. We created three between-subjects experimental conditions: reading the two pages through one panel at a time, two sequential full-page read, and two full-page available at once. Additionally, participants completed the Visual Language Fluency Index (VLFI; Cohn, 2014) allowing us to split them into two groups: novice and experts. Due to the COVID-19 pandemic, we ran the experiment using a videoconferencing tool.We choose two comic books that were very different in aesthetic and visual design choices. For each, we selected two consecutive pages. We selected pages 46-47 from “V for Vendetta” (Moore & Lloyd, 2012) whose design relies on few darker colors, many text balloons and panels, and a systematic grid layout. We also selected pages 80-81 from “The Two Lives of Penelope” (Vanistendael, 2019) whose design relies on colorful drawing, few text balloons and panels; and a loosely defined panel layout.We measured reading performance through reading time and a score of understanding. Reading time is the time required for a participant to read the entire content of the two-page selection from comic books. We calculated the score of understanding based on correct answers from a fifteen-item questionnaire. We created a questionnaire for each of the comic book excerpt.3)ResultsFirst, we found a correlation between the two comic books for both reading time (r = 0,292, p < 0.05) and understanding scores (r = 0,361, p < 0.05). Overall, “V for Vendetta” was harder to read (mean understanding score: 73%, mean reading time: 183.89s) than “The Two Lives of Penelope” (76.5% and 119.95s, respectively).According to our hypotheses, we expected that participant would perform worst when they had to read smaller bits of the comic than full-page views (single-page or two-pages at once). We could not corroborate that hypothesis based on our data (no effect in ANOVA tests). Participants’ reading time and understanding scores were equivalent in the three reading conditions, for both comic books.As we expected, we found an effect of expertise on reading performance. Our findings showed that the expert group had a significantly better reading performance. We found a better understanding score from experts compared to novices on “V for Vendetta” (F(1,54) = 13.17 ; p < 0.05) and “The Two Lives of Penelope” (F(1,54)=7.80; p < 0.05). We also found a reading time significantly faster for “V for Vendetta” (F(1,54) = 6.76; p < 0.05) from experts compared to novices, but not for the other comics excerpt.4)DiscussionLiterature pointed out that reading comics may involve both a local and a global reading scale. We assumed that global visual scan plays a role in understanding pages of a comic. We manipulated the number of available panels at a given time and we measured reader’s expertise in visual narrative reading. Our results did not demonstrate that comics were harder to understand in local reading condition (single-panel) than more global ones (single-page or double-page). However, our results showed that expertise, measured as a score of VLFI scale, implied a better understanding or two excerpts. A faster reading time is also observed with “V for Vendetta” which had more text content to read, but also had a more conventional layout.More must be done. Participants were reading comics on a desktop screen. While reading digital comics is growing, such conditions might not have allowed to observer optimal global visual reading patterns. A comparison between different screen sizes and paper formats will provide more insights on this question. Of course, our work needs to be expanded over a larger range of comics with a better control of aesthetic and narrative styles. Extending this work will allow for a higher generalization on how visual design of comics affects their reading. Such knowledge may help comics’ creators, editors and other users such as teachers to provide optimal reading experience.Bibliography:Cohn, N. (2013). Navigating Comics: An Empirical and Theoretical Approach to Strategies of Reading Comic Page Layouts. Frontiers in Psychology, 4. https://doi.org/10.3389/fpsyg.2013.00186Cohn, N. (2014). The Visual Language Fluency Index: A measure of comic reading expertise. Visual Language Lab. Resources. www.visuallanguagelab.com/resources.html. First posted online April 2014.Cohn, N., Axnér, J., Diercks, M., Yeh, R., & Pederson, K. (2017). The cultural pages of comics: Cross-cultural variation in page layouts. Journal of Graphic Novels and Comics, 1 20. https://doi.org/10.1080/21504857.2017.1413667 Mikkonen, K. H., & Lautenbacher, O. P. (2019). Global Attention in Reading Comics: Eye movement indications of interplay between narrative content and layout, ImageTexT 10 (2). Moore, A. & Lloyd, D. (2012). V pour Vendetta. Urban ComicsNakazawa, J. (2016). Manga literacy and manga comprehension in Japanese children. In N. Cohn (Ed.), The visual narrative reader (pp. 157–184). London: BloomsburyVanistendael, J. (2019).Zhao, F., Mahrt, N. (2018). Influences of Comics Expertise and Comics Types in Comics Reading. International Journal of Innovation and Research in Educational Sciences. Volume 5

Colorizer: Smart Glasses Aid for the Colorblind

We present a smart glasses application for helping colorblind people to distinguish problematic colors in daily life. The prototype processes a live video stream from the mobile camera, remaps colors according to the user needs, and displays the augmented result. Color transformation ensures high contrast between colors which are otherwise indistinguishable for the user

Formation immersive dans la gestion des crises d'origine cyber : Le cas du projet RESISTECC

International audienceThe “Resilience through Immersive Strategic and Technical Simulation of Cyber Crises” project aims to train staff from small communities and small socio-economic entities in the proper management of cyber crises. To carry this out, a Cyber-Range will be used as an experiential learning setting, which will simulate the learners' working environment as well as their databases and digital work tools. The main objective of this work is to position cognitive ergonomics within the RESISTECC project. This positioning will be based on the evaluation of three variables: immersion, situation awareness and cognitive load. The evaluation of these three variables will be a lever to achieve the instructional objectives of the project, both in the creation of simulation scenarios and in understanding how learners make decisions during stressful situations.Le projet « Résilience par la Simulation Immersive Stratégique et Technique de Crises Cyber » vise à former le personnel des petites collectivités et des petites entités socio-économiques à la bonne gestion des crises d'origine cyber. Pour ce faire, un Cyber-Range sera utilisé comme dispositif d'apprentissage expérientiel. Il simulera l'environnement de travail des apprenants ainsi que leurs bases de données et outils numériques de travail. L'objectif principal de ce travail est de positionner l'ergonomie cognitive au sein du projet RESISTECC. Ce positionnement se fera sur la base de l'évaluation de trois variables : l'immersion, la conscience de la situation et la charge cognitive. L'évaluation de ces trois variables constituera un levier pour atteindre les objectifs pédagogiques du projet, tant dans la création de scénarios de simulation que dans la compréhension de la façon dont les apprenants prennent des décisions lors de situations stressantes

Colorizer

International audienc

Synchronizing self-displacement with a cross-traffic gap: How does the size of traffic vehicles impact continuous speed regulations?

International audienceIn this article, we investigated what visual information is used by drivers at a road crossing when they want to synchronize their displacement with that of an incoming traffic train. We made the hypothesis that synchronizing self-displacement with that of a traffic gap shares the same perceptual-motor basis as interception tasks. While a large body of literature demonstrates that bearing angle is used to control interception, another range of studies points to optical size and expansion as playing a critical role in collision avoidance. In order to test the hypothesis of the exclusive use of bearing angle in road crossing task, we manipulated the optical size and expansion of oncoming traffic elements independently of bearing angle variations. We designed a driving simulator study in which participants were to adjust their approach speed in order to cross a road junction within a moving traffic gap. We manipulated the initial offset of participants with the traffic gap, the geometry of the road junction and the way optical size of oncoming traffic elements evolves over the course of a trial. Our results showed an effect of optical size and optical expansion manipulations eventhough, we also found similar displacement profiles as in interception studies. This demonstrates that bearing angle could not explain alone the control of such a complex perceptual-motor task. We discuss these results with regard to similar results in other fields of literature.In this article, we investigated what visual information is used by drivers at a road crossing when they want to synchronized their displacement with that of an incoming traffic train. We made the hypothesis that synchronizing self-displacement with that of a traffic gap shares the same perceptual-motor basis as interception tasks. While a large body of literature demonstrated that bearing angle is used to control interception, another range of studies points to optical size and expansion as playing a critical role in collision avoidance. In order to test the hypothesis of the exclusive use of bearing angle in road crossing task, we manipulated the optical size and expansion of oncoming traffic elements independently of bearing angle variations. We designed a driving simulator study in which participants were to adjust their approach speed in order to cross a road junction within a moving traffic gap. We manipulated the initial offset of participants with the traffic gap, the geometry of the road junction and the way optical size of oncoming traffic elements evolves over the course of a trial. Our results showed an effect of optical size and optical expansion manipulations eventhough, we also found similar displacement profiles as in interception studies. This demonstrates that bearing angle could not explain alone the control of such a complex perceptual-motor task. We discuss these results with regard to similar results in other fields of literature

D2.1 The Specification and Overall Requirements of the eGlasses Platform

The purpose of this document is to review the requirements of the platform in terms of human factors issues that to be addressed, provide an update on the state of the art in augmented reality requirements and to provide an early technical specification. Addressing these issues will ensure that the platform can deliver its full potential when actually operated by the end-users. Indeed, a poorly designed system will not only reduce utility and acceptance of the platform but also induce discomfort and frustration. The document keeps the requirements at a general level as it does not support specific use-case implementations. Instead this document provides a list of general specifications and requirements that must be addressed in order for the eGlasses platform to be a success