Collaborative Gaze Channelling for Improved Cooperation During Robotic Assisted Surgery

The use of multiple robots for performing complex tasks is becoming a common practice for many robot applications. When different operators are involved, effective cooperation with anticipated manoeuvres is important for seamless, synergistic control of all the end-effectors. In this paper, the concept of Collaborative Gaze Channelling (CGC) is presented for improved control of surgical robots for a shared task. Through eye tracking, the fixations of each operator are monitored and presented in a shared surgical workspace. CGC permits remote or physically separated collaborators to share their intention by visualising the eye gaze of their counterparts, and thus recovers, to a certain extent, the information of mutual intent that we rely upon in a vis-à-vis working setting. In this study, the efficiency of surgical manipulation with and without CGC for controlling a pair of bimanual surgical robots is evaluated by analysing the level of coordination of two independent operators. Fitts' law is used to compare the quality of movement with or without CGC. A total of 40 subjects have been recruited for this study and the results show that the proposed CGC framework exhibits significant improvement (p<0.05) on all the motion indices used for quality assessment. This study demonstrates that visual guidance is an implicit yet effective way of communication during collaborative tasks for robotic surgery. Detailed experimental validation results demonstrate the potential clinical value of the proposed CGC framework. © 2012 Biomedical Engineering Society.link_to_subscribed_fulltex

Joint perception: gaze and social context

We found that the way people looked at images was influenced by their belief that others were looking too. If participants believed that an unseen other person was also looking at what they could see, it shifted the balance of their gaze between negative and positive images. The direction of this shift depended upon whether participants thought that later they would be compared against the other person or would be collaborating with them. Changes in the social context influenced both gaze and memory processes, and were not due just to participants' belief that they are looking at the same images, but also to the belief that they are doing the same task. We believe that the phenomenon of joint perception reveals the pervasive and subtle effect of social context upon cognitive and perceptual processes

Joint perception: gaze and beliefs about social context

The way that we look at images is influenced by social context. Previously we demonstrated this phenomenon of joint perception. If lone participants believed that an unseen other person was also looking at the images they saw, it shifted the balance of their gaze between negative and positive images. The direction of this shift depended upon whether participants thought that later they would be compared against the other person or would be collaborating with them. Here we examined whether the joint perception is caused by beliefs about shared experience (looking at the same images) or beliefs about joint action (being engaged in the same task with the images). We place our results in the context of the emerging field of joint action, and discuss their connection to notions of group emotion and situated cognition. Such findings reveal the persuasive and subtle effect of social context upon cognitive and perceptual processes

A New Medium for Remote Music Tuition

It is common to learn to play an orchestral musical instrument through regular one-to-one lessons with an experienced musician as a tutor. Students may work with the same tutor for many years, meeting regularly to receive real-time, iterative feedback on their performance. However, musicians travel regularly to audition, teach and perform and this can sometimes make it difficult to maintain regular contact. In addition, an experienced tutor for a specific instrument or musical style may not be available locally. General instrumental tuition may not be available at all in geographically distributed communities. One solution is to use technology such as videoconference to facilitate a remote lesson; however, this fundamentally changes the teaching interaction. For example, as a result of the change in communication medium, the availability of non-verbal cues and perception of relative spatiality is reduced. We describe a study using video-ethnography, qualitative video analysis and conversation analysis to make a fine-grained examination of student–tutor interaction during five co-present and one video-mediated woodwind lesson. Our findings are used to propose an alternative technological solution – an interactive digital score. Rather than the face-to-face configuration enforced by videoconference, interacting through a shared digital score, augmented by visual representation of the social cues found to be commonly used in co-present lessons, will better support naturalistic student–tutor interaction during the remote lesson experience. Our findings may also be applicable to other fields where knowledge and practice of a physical skill sometimes need to be taught remotely, such as surgery or dentistry

Measuring cognitive load and cognition: metrics for technology-enhanced learning

This critical and reflective literature review examines international research published over the last decade to summarise the different kinds of measures that have been used to explore cognitive load and critiques the strengths and limitations of those focussed on the development of direct empirical approaches. Over the last 40 years, cognitive load theory has become established as one of the most successful and influential theoretical explanations of cognitive processing during learning. Despite this success, attempts to obtain direct objective measures of the theory's central theoretical construct – cognitive load – have proved elusive. This obstacle represents the most significant outstanding challenge for successfully embedding the theoretical and experimental work on cognitive load in empirical data from authentic learning situations. Progress to date on the theoretical and practical approaches to cognitive load are discussed along with the influences of individual differences on cognitive load in order to assess the prospects for the development and application of direct empirical measures of cognitive load especially in technology-rich contexts

Tutor In-sight: Guiding and Visualizing Students Attention with Mixed Reality Avatar Presentation Tools

Remote conferencing systems are increasingly used to supplement or even replace in-person teaching. However, prevailing conferencing systems restrict the teacher’s representation to a webcam live-stream, hamper the teacher’s use of body-language, and result in students’ decreased sense of co-presence and participation. While Virtual Reality (VR) systems may increase student engagement, the teacher may not have the time or expertise to conduct the lecture in VR. To address this issue and bridge the requirements between students and teachers, we have developed Tutor In-sight, a Mixed Reality (MR) avatar augmented into the student’s workspace based on four design requirements derived from the existing literature, namely: integrated virtual with physical space, improved teacher’s co-presence through avatar, direct attention with auto-generated body language, and usable workfow for teachers. Two user studies were conducted from the perspectives of students and teachers to determine the advantages of Tutor In-sight in comparison to two existing conferencing systems, Zoom (video-based) and Mozilla Hubs (VR-based). The participants of both studies favoured Tutor In-sight. Among others, this main fnding indicates that Tutor Insight satisfed the needs of both teachers and students. In addition, the participants’ feedback was used to empirically determine the four main teacher requirements and the four main student requirements in order to improve the future design of MR educational tools

Work-In-Progress Technical Report: Designing A Two-User, Two-View TV Display

This work-in-progress paper previews how we can design interfaces and interactions for multi-view TVs, enabling users to transition between independent and shared activities, gain casual awareness of others’ activities, and collaborate more effectively. We first compare an Android-based multi-user TV against both multi-screen and multi-view TVs in a collaborative movie browsing task, to determine whether multiview can improve collaboration, and what level of awareness of each other’s activity users choose. Based on our findings, we iterate on our multi-view design in a second study, giving users the ability to transition between casual and focused modes of engagement, and dynamically set their engagement with other users’ activities. This research demonstrates that the shared focal point of the TV now has the capability to facilitate both collaborative and completely independent activity

Visualizing a Task Performer’s Gaze to Foster Observers’ Performance and Learning : a Systematic Literature Review on Eye Movement Modeling Examples

Eye movement modeling examples (EMMEs) are instructional videos (e.g., tutorials) that visualize another person’s gaze location while they demonstrate how to perform a task. This systematic literature review provides a detailed overview of studies on the effects of EMME to foster observers’ performance and learning and highlights their differences in EMME designs. Through a broad, systematic search on four relevant databases, we identified 72 EMME studies (78 experiments). First, we created an overview of the different study backgrounds. Studies most often taught tasks from the domains of sports/physical education, medicine, aviation, and STEM areas and had different rationales for displaying EMME. Next, we outlined how studies differed in terms of participant characteristics, task types, and the design of the EMME materials, which makes it hard to infer how these differences affect performance and learning. Third, we concluded that the vast majority of the experiments showed at least some positive effects of EMME during learning, on tests directly after learning, and tests after a delay. Finally, our results provide a first indication of which EMME characteristics may positively influence learning. Future research should start to more systematically examine the effects of specific EMME design choices for specific participant populations and task types