MUVTIME: a Multivariate time series visualizer for behavioral science

As behavioral science becomes progressively more data driven, the need is increasing for appropriate tools for visual exploration and analysis of large datasets, often formed by multivariate time series. This paper describes MUVTIME, a multimodal time series visualization tool, developed in Matlab that allows a user to load a time series collection (a multivariate time series dataset) and an associated video. The user can plot several time series on MUVTIME and use one of them to do brushing on the displayed data, i.e. select a time range dynamically and have it updated on the display. The tool also features a categorical visualization of two binary time series that works as a high-level descriptor of the coordination between two interacting partners. The paper reports the successful use of MUVTIME under the scope of project TURNTAKE, which was intended to contribute to the improvement of human-robot interaction systems by studying turn- taking dynamics (role interchange) in parent-child dyads during joint action.Marie Curie International Incoming Fellowship PIIF-GA-2011- 301155; Portuguese Foundation for Science and Technology (FCT) project PTDC/PSI- PCO/121494/2010; AFP was also partially funded by the FCT project (IF/00217/2013)This research was supported by: Marie Curie International Incoming Fellowship PIIF-GA-2011301155; Portuguese Foundation for Science and Technology (FCT) Strategic program FCT UID/EEA/00066/2013; FCT project PTDC/PSIPCO/121494/2010. AFP was also partially funded by the FCT project (IF/00217/2013). REFERENCE

Un modèle pour la gestion et la capitalisation d'analyses de traces d'activités en interaction collaborative

We present our three main results in adressing the problem of assisting the socio-cognitive analysis of human interaction. First, we propose a description of the process of analysis of such data, as well as a generic artefact which covers a large number of the analytic artefacts we have observed and which we call a replayable. Second, we present a study and a modelling of replayables, and describe the four fundamental operations which can be applied to them: synchronisation, visualisation, transformation and enrichment. Finally, we describe the implementation of this model in an environment that assists analysis through the manipulation of replayables, which we evaluate in real-life research situations. Tatiana (http://code.google.com/p/tatiana), the resulting software environment, is based on these four operations and integrates numerous possibilities for extending these operations to adapt to new kinds of analysis while staying within the analytic framework afforded by replayables.Nous présentons nos trois résultats principaux face à la difficulté d'assister l'analyse socio-cognitive d'interactions humaines. D'une part, nous proposons une description du processus d'analyse de ce genre données ainsi qu'un artefact générique permettant de recouvrir un grand nombre d'artefacts analytiques que nous avons pu observer et que nous nommons rejouable. D'autre part, nous présentons une étude et modélisation informatique des rejouables, et décrivons quatre opérations fondamentales qui peuvent s'y appliquer : synchronisation, visualisation, transformation et enrichissement. Enfin, nous décrivons l'implémentation de cette modélisation dans un environnement d'aide à l'analyse par manipulation de rejouables que nous évaluons dans des situations de recherche réelles. Tatiana (http://code.google.com/p/tatiana), l'environnement logiciel résultant, est basé sur ces quatre opérations et permet l'extension de ces opérations pour s'adapter à de nouvelles formes d'analyse

Understanding cognitive work

Decades of research into the impact of new technologies in domains such as power generation, aviation, critical care, and other domains makes it abundantly clear that joint cognitive systems are not always designed in a way that avoids the pitfalls captured in some of the above laws. Therefore, an efficient way to investigate the impact of change on cognitive work is to be guided by search for instances where these laws have been respected or violated. In other words, the analyst should not only be prepared to find instances where the laws are in operation but also be prepared to find instances where the relationships described by the laws are present in new, surprising ways, or are absent

PAPER Developmental continuity? Crawling, cruising, and walking

Abstract This research examined developmental continuity between 'cruising' (moving sideways holding onto furniture for support) and walking. Because cruising and walking involve locomotion in an upright posture, researchers have assumed that cruising is functionally related to walking. Study 1 showed that most infants crawl and cruise concurrently prior to walking, amassing several weeks of experience with both skills. Study 2 showed that cruising infants perceive affordances for locomotion over an adjustable gap in a handrail used for manual support, but despite weeks of cruising experience, cruisers are largely oblivious to the dangers of gaps in the floor beneath their feet. Study 3 replicated the floor-gap findings for infants taking their first independent walking steps, and showed that new walkers also misperceive affordances for locomoting between gaps in a handrail. The findings suggest that weeks of cruising do not teach infants a basic fact about walking: the necessity of a floor to support the body. Moreover, this research demonstrated that developmental milestones that are temporally contiguous and structurally similar might have important functional discontinuities

Gesture generation by imitation : from human behavior to computer character animation

This dissertation shows how to generate conversational gestures for an animated agent based on annotated text input. The central idea is to imitate the gestural behavior of human individuals. Using TV show recordings as empirical data, gestural key parameters are extracted for the generation of natural and individual gestures. For each of the three tasks in the generation pipeline a software was developed. The generic ANVIL annotation tool allows to transcribe gesture and speech in the empirical data. The NOVALIS module uses the annotations to compute individual gesture profiles with statistical methods. The NOVA generator creates gestures based on these profiles and heuristic rules, and outputs them in a linear script. In all, this work presents a complete work pipeline from collecting empirical data to obtaining an executable script and provides the necessary software, too.Die vorliegende Dissertation stellt einen Ansatz zur Generierung von Konversationsgesten für animierte Agenten aus annotatiertem Textinput vor. Zentrale Idee ist es, die Gestik menschlicher Individuen zu imitieren. Als empirisches Material dient eine Fernsehsendung, aus der Schlüsselparameter zur Generierung natürlicher und individueller Gesten extrahiert werden. Die Generierungsaufgabe wurde in drei Schritten mit eigens entwickelter Software gelöst. Das generische ANVIL-Annotationswerkzeug ermöglicht die Transkription von Gestik und Sprache in den empirischen Daten. Das NOVALIS-Modul berechnet aus den Annotationen individuelle Gestenprofile mit Hilfe statistischer Verfahren. Der NOVAGenerator erzeugt Gesten anhand dieser Profile und allgemeiner Heuristiken und gibt diese in Skriptform aus. Die Arbeit stellt somit einen vollständigen Arbeitspfad von empirischer Datenerhebung bis zum abspielfertigen Skript vor und liefert die entsprechenden Software-Werkzeuge dazu

Gesture generation by imitation : from human behavior to computer character animation

This dissertation shows how to generate conversational gestures for an animated agent based on annotated text input. The central idea is to imitate the gestural behavior of human individuals. Using TV show recordings as empirical data, gestural key parameters are extracted for the generation of natural and individual gestures. For each of the three tasks in the generation pipeline a software was developed. The generic ANVIL annotation tool allows to transcribe gesture and speech in the empirical data. The NOVALIS module uses the annotations to compute individual gesture profiles with statistical methods. The NOVA generator creates gestures based on these profiles and heuristic rules, and outputs them in a linear script. In all, this work presents a complete work pipeline from collecting empirical data to obtaining an executable script and provides the necessary software, too.Die vorliegende Dissertation stellt einen Ansatz zur Generierung von Konversationsgesten für animierte Agenten aus annotatiertem Textinput vor. Zentrale Idee ist es, die Gestik menschlicher Individuen zu imitieren. Als empirisches Material dient eine Fernsehsendung, aus der Schlüsselparameter zur Generierung natürlicher und individueller Gesten extrahiert werden. Die Generierungsaufgabe wurde in drei Schritten mit eigens entwickelter Software gelöst. Das generische ANVIL-Annotationswerkzeug ermöglicht die Transkription von Gestik und Sprache in den empirischen Daten. Das NOVALIS-Modul berechnet aus den Annotationen individuelle Gestenprofile mit Hilfe statistischer Verfahren. Der NOVAGenerator erzeugt Gesten anhand dieser Profile und allgemeiner Heuristiken und gibt diese in Skriptform aus. Die Arbeit stellt somit einen vollständigen Arbeitspfad von empirischer Datenerhebung bis zum abspielfertigen Skript vor und liefert die entsprechenden Software-Werkzeuge dazu

Using qualitative methodology in ergonomics: theoretical background and practical examples

Qualitative methodology offers the opportunity for ergonomists to investigate work problems and research questions using context-sensitive tools for data collection and analysis. This is particularly useful in contexts with complex social and cultural dimensions, for example the high level of emotional and intimate personal interactions between staff and patients in the health care industry. Two aims have been achieved in this thesis. The first is to set out a clear process for using qualitative methodology in ergonomics by taking a middle ground position with respect to the background philosophy. A generic process for carrying out qualitative research is described and shown in detail in the two case studies. The ergonomists case study found that there was considerable interest in expanding the ergonomics tool box to include qualitative methodology. However concerns were raised about a perceived lack of knowledge with respect to the process for doing qualitative research. This needs to be addressed by including teaching qualitative methodology in ergonomics courses. The second aim is use qualitative methodology to identify characteristics of hospitals with respect to the practice of ergonomics. Three themes emerged: organisational issues (complexity and size); staff issues (multiplicity of professionals and gender); and patient issues (dirty and emotional work; patient expectations; and life, death and mistakes). These themes were also found in the practical case study on manual handling problems in occupational therapy. This suggests that knowledge of the characteristics of an industry can help the ergonomist to understand the context of the work problem or research question. A final dynamic model of ergonomics is proposed to bring together the internal dimensions of a person (representing physical, cognitive and spiritual levels) and the external dimensions of their interactions at a micro level (e. g. tasks) and at wider organisational and societal levels (macro). This model shows the importance of using qualitative methodology to achieve a more complete understanding of human interactions: the basis of the definition of ergonomics

Using qualitative methodology in ergonomics: theoretical background and practical examples

Qualitative methodology offers the opportunity for ergonomists to investigate work problems and research questions using context-sensitive tools for data collection and analysis. This is particularly useful in contexts with complex social and cultural dimensions, for example the high level of emotional and intimate personal interactions between staff and patients in the health care industry. Two aims have been achieved in this thesis. The first is to set out a clear process for using qualitative methodology in ergonomics by taking a middle ground position with respect to the background philosophy. A generic process for carrying out qualitative research is described and shown in detail in the two case studies. The ergonomists case study found that there was considerable interest in expanding the ergonomics tool box to include qualitative methodology. However concerns were raised about a perceived lack of knowledge with respect to the process for doing qualitative research. This needs to be addressed by including teaching qualitative methodology in ergonomics courses. The second aim is use qualitative methodology to identify characteristics of hospitals with respect to the practice of ergonomics. Three themes emerged: organisational issues (complexity and size); staff issues (multiplicity of professionals and gender); and patient issues (dirty and emotional work; patient expectations; and life, death and mistakes). These themes were also found in the practical case study on manual handling problems in occupational therapy. This suggests that knowledge of the characteristics of an industry can help the ergonomist to understand the context of the work problem or research question. A final dynamic model of ergonomics is proposed to bring together the internal dimensions of a person (representing physical, cognitive and spiritual levels) and the external dimensions of their interactions at a micro level (e. g. tasks) and at wider organisational and societal levels (macro). This model shows the importance of using qualitative methodology to achieve a more complete understanding of human interactions: the basis of the definition of ergonomics

Information enforcement in learning with graphics : improving syllogistic reasoning skills

This thesis is an investigation into the factors that contribute to good choices among graphical systems used in teaching, and the feasibility of implementing teaching software that uses this knowledge.The thesis describes a mathematical metric derived from a cognitive theory of human diagram processing. The theory characterises differences among representations by their ability to express information. The theory provides the factors and relationships needed to build the metric. It says that good representations are easily processed because they are more vivid, more tractable and less expressive, than poor representations.The metric is applied to abstract systems for teaching and learning syllogistic reasoning, TARSKI'S WORLD, EULER CIRCLES, VENN DIAGRAMS and CARROLL'S GAME OF LOGIC. A rank ordering reflects the value of each system predicted by the theory and the metric. The theory, the metric and the systems are then tested in empirical studies. Five studies involving sixty-eight learners, examined the benefit of software based on these abstract systems.Studies showed the theory correctly predicted learners' success with the circle systems and poorer performance with TARSKI'S WORLD. The metric showed small but clear differences in expressivity between the circle systems. Differences between results of the learners using the circle systems contradicted the predictions of the metric.Learners with mathematical training were better equipped and more successful at learning syllogistic reasoning with the systems. Performance of learners without mathematical training declined after using the software systems. Diagrams drawn by learners together with video footage collected during problem solving, led to a catalogue of errors, misconceptions and some helpful strategies for learning from graphical systems.A cognitive style test investigated the poor performance of non-mathematically trained learners. Learners with mathematics training showed serialist and versatile learning styles while learners without this training showed a holist learning style. This is consistent with the hypothesis that non-mathematically trained learners emphasise the use of semantic cues during learning and problem solving.A card-sorting task investigated learners' preferences for parts of the graphical lexicon used in the diagram systems. Preferences for the EULER lexicon increased difficulty in explaining the system's poor results in earlier studies. Video footage of learners using the systems in the final study illustrated useful learning strategies and improved performance with EULER while individual instruction was available.Further work describes a preliminary design for an adaptive syllogism tutor and other related work