D3.1 Instructional Designs for Real-time Feedback

The main objective of METALOGUE is to produce a multimodal dialogue system that is able to implement an interactive behaviour that seems natural to users and is flexible enough to exploit the full potential of multimodal interaction. The METALOGUE system will be arranged in the context of educational use-case scenarios, i.e. for training active citizens (Youth Parliament) and call centre employees. This deliverable describes the intended real-time feedback and reflection in-action support to support the training. Real-time feedback informs learners how they perform key skills and enables them to monitor their progress and thus reflect in-action. This deliverable examines the theoretical considerations of reflection in-action, what type of data is available and should be used, the timing and type of real-time feedback and, finally, concludes with an instructional design blueprint giving a global outline of a set of tasks with stepwise increasing complexity and the feedback proposed.The underlying research project is partly funded by the METALOGUE project. METALOGUE is a Seventh Framework Programme collaborative project funded by the European Commission, grant agreement number: 611073 (http://www.metalogue.eu)

Multi-user virtual environments for physical education and sport training

For effective learning and training, virtual environments may provide lifelike opportunities, and researchers are actively investigating their potential for educational purposes. Minimal research attention has been paid to the integration of multi-user virtual environments (MUVE) technology for teaching and practicing real sports. In this chapter, the authors reviewed the justifications, possibilities, challenges, and future directions of using MUVE systems. The authors addressed issues such as informal learning, design, engagement, collaboration, learning style, learning evaluation, motivation, and gender, followed by the identification of required elements for successful implementations. In the second part, the authors talked about exergames, the necessity of evaluation, and examples on exploring the behavior of players during playing. Finally, insights on the application of sports exergames in teaching, practicing, and encouraging real sports were discussed

Measuring the effectiveness of virtual training : a systematic review

The amount of research on virtual reality learning tools increases with time. Despite the diverse environments and theoretical foundations, enough data have been accumulated in recent years to provide a systematic review of the methods used. We pose ten questions concerning the methodological aspects of these studies. We performed a search in three databases according to the PRISMA guidelines and evaluated several characteristics, with particular emphasis on researchers' methodological decisions. We found an increase over time in the number of studies on the effectiveness of VR-based learning. We also identified shortcomings related to how the duration and number of training sessions are reported. We believe that these two factors could affect the effectiveness of VR-based training. Furthermore, when using the Kirkpatrick model, a significant imbalance can be observed in favor of outcomes from the ‘Reaction’ and ‘Learning’ levels compared to the ‘Behavior’ and ‘Results’ levels. The last of these was not used in any of the 330 reviewed studies. These results highlight the importance of research on the effectiveness of VR training. Taking into account the identified methodological shortcomings will allow for more significant research on this topic in the future

Multisensory learning in adaptive interactive systems

The main purpose of my work is to investigate multisensory perceptual learning and sensory integration in the design and development of adaptive user interfaces for educational purposes. To this aim, starting from renewed understanding from neuroscience and cognitive science on multisensory perceptual learning and sensory integration, I developed a theoretical computational model for designing multimodal learning technologies that take into account these results. Main theoretical foundations of my research are multisensory perceptual learning theories and the research on sensory processing and integration, embodied cognition theories, computational models of non-verbal and emotion communication in full-body movement, and human-computer interaction models. Finally, a computational model was applied in two case studies, based on two EU ICT-H2020 Projects, "weDRAW" and "TELMI", on which I worked during the PhD

Eye quietness and quiet eye in expert and novice golf performance: an electrooculographic analysis

Quiet eye (QE) is the final ocular fixation on the target of an action (e.g., the ball in golf putting). Camerabased eye-tracking studies have consistently found longer QE durations in experts than novices; however, mechanisms underlying QE are not known. To offer a new perspective we examined the feasibility of measuring the QE using electrooculography (EOG) and developed an index to assess ocular activity across time: eye quietness (EQ). Ten expert and ten novice golfers putted 60 balls to a 2.4 m distant hole. Horizontal EOG (2ms resolution) was recorded from two electrodes placed on the outer sides of the eyes. QE duration was measured using a EOG voltage threshold and comprised the sum of the pre-movement and post-movement initiation components. EQ was computed as the standard deviation of the EOG in 0.5 s bins from –4 to +2 s, relative to backswing initiation: lower values indicate less movement of the eyes, hence greater quietness. Finally, we measured club-ball address and swing durations. T-tests showed that total QE did not differ between groups (p = .31); however, experts had marginally shorter pre-movement QE (p = .08) and longer post-movement QE (p < .001) than novices. A group × time ANOVA revealed that experts had less EQ before backswing initiation and greater EQ after backswing initiation (p = .002). QE durations were inversely correlated with EQ from –1.5 to 1 s (rs = –.48 - –.90, ps = .03 - .001). Experts had longer swing durations than novices (p = .01) and, importantly, swing durations correlated positively with post-movement QE (r = .52, p = .02) and negatively with EQ from 0.5 to 1s (r = –.63, p = .003). This study demonstrates the feasibility of measuring ocular activity using EOG and validates EQ as an index of ocular activity. Its findings challenge the dominant perspective on QE and provide new evidence that expert-novice differences in ocular activity may reflect differences in the kinematics of how experts and novices execute skills

A Biosymtic (Biosymbiotic Robotic) Approach to Human Development and Evolution. The Echo of the Universe.

In the present work we demonstrate that the current Child-Computer Interaction paradigm is not potentiating human development to its fullest – it is associated with several physical and mental health problems and appears not to be maximizing children’s cognitive performance and cognitive development. In order to potentiate children’s physical and mental health (including cognitive performance and cognitive development) we have developed a new approach to human development and evolution. This approach proposes a particular synergy between the developing human body, computing machines and natural environments. It emphasizes that children should be encouraged to interact with challenging physical environments offering multiple possibilities for sensory stimulation and increasing physical and mental stress to the organism. We created and tested a new set of computing devices in order to operationalize our approach – Biosymtic (Biosymbiotic Robotic) devices: “Albert” and “Cratus”. In two initial studies we were able to observe that the main goal of our approach is being achieved. We observed that, interaction with the Biosymtic device “Albert”, in a natural environment, managed to trigger a different neurophysiological response (increases in sustained attention levels) and tended to optimize episodic memory performance in children, compared to interaction with a sedentary screen-based computing device, in an artificially controlled environment (indoors) - thus a promising solution to promote cognitive performance/development; and that interaction with the Biosymtic device “Cratus”, in a natural environment, instilled vigorous physical activity levels in children - thus a promising solution to promote physical and mental health

Scalable Teaching and Learning via Intelligent User Interfaces

The increasing demand for higher education and the educational budget cuts lead to large class sizes. Learning at scale is also the norm in Massive Open Online Courses (MOOCs). While it seems cost-effective, the massive scale of class challenges the adoption of proven pedagogical approaches and practices that work well in small classes, especially those that emphasize interactivity, active learning, and personalized learning. As a result, the standard teaching approach in today’s large classes is still lectured-based and teacher-centric, with limited active learning activities, and with relatively low teaching and learning effectiveness. This dissertation explores the usage of Intelligent User Interfaces (IUIs) to facilitate the efficient and effective adoption of the tried-and-true pedagogies at scale. The first system is MindMiner, an instructor-side data exploration and visualization system for peer review understanding. MindMiner helps instructors externalize and quantify their subjective domain knowledge, interactively make sense of student peer review data, and improve data exploration efficiency via distance metric learning. MindMiner also helps instructors generate customized feedback to students at scale. We then present BayesHeart, a probabilistic approach for implicit heart rate monitoring on smartphones. When integrated with MOOC mobile clients, BayesHeart can capture learners’ heart rates implicitly when they watch videos. Such information is the foundation of learner attention/affect modeling, which enables a ‘sensorless’ and scalable feedback channel from students to instructors. We then present CourseMIRROR, an intelligent mobile system integrated with Natural Language Processing (NLP) techniques that enables scalable reflection prompts in large classrooms. CourseMIRROR 1) automatically reminds and collects students’ in-situ written reflections after each lecture; 2) continuously monitors the quality of a student’s reflection at composition time and generates helpful feedback to scaffold reflection writing; 3) summarizes the reflections and presents the most significant ones to both instructors and students. Last, we present ToneWars, an educational game connecting Chinese as a Second Language (CSL) learners with native speakers via collaborative mobile gameplay. We present a scalable approach to enable authentic competition and skill comparison with native speakers by modeling their interaction patterns and language skills asynchronously. We also prove the effectiveness of such modeling in a longitudinal study

Mindfulness Program for Student Athletes

Student athletes face challenging situations during their time at college which can cause their experience to be a negative one if they are not fully supported. This can be due to several factors; most athletes face an immense amount of pressure to succeed in the classroom as well as in their sport. Also, many athletes are moving away from home for the first time and this can be a daunting feeling. They are expected to manage all of these challenges, and if mishandled, it can lead to an overwhelming amount of negative stress resulting in an unsuccessful time as a student athlete. Furthermore, the intensity and pressure of their sport can be detrimental to other areas of their life. Many of these young athletes do not possess the leadership skills necessary to manage these tasks and remain in a healthy state of mind. My recommendation is not to take away these challenges, but rather offer support systems which empower student athletes to be able to manage the struggles they encounter in university settings. This mindfulness program enables people to be present and aware of themselves, acknowledge what their body is saying to them, and take action in a calm focused way to move into a healthy state of mind. My program offers communication at the individual and group level, techniques to be present, manage stress, and motivate positive thoughts which create pathways for successful experiences. Mindfulness practices are new to sports and are not currently applied to college athletics; however, the stress and challenges associated with being a student athlete are ever present. Therefore, I argue that college programs/coaches need to provide support systems which help their students be successful. This would then become a platform whereby they could succeed athletically, academically, personally, and prepare them for life outside of college