Supporting Memorization and Problem Solving with Spatial Information Presentations in Virtual Environments

While it has been suggested that immersive virtual environments could provide benefits for educational applications, few studies have formally evaluated how the enhanced perceptual displays of such systems might improve learning. Using simplified memorization and problem-solving tasks as representative approximations of more advanced types of learning, we are investigating the effects of providing supplemental spatial information on the performance of learning-based activities within virtual environments. We performed two experiments to investigate whether users can take advantage of a spatial information presentation to improve performance on cognitive processing activities. In both experiments, information was presented either directly in front of the participant or wrapped around the participant along the walls of a surround display. In our first experiment, we found that the spatial presentation caused better performance on a memorization and recall task. To investigate whether the advantages of spatial information presentation extend beyond memorization to higher level cognitive activities, our second experiment employed a puzzle-like task that required critical thinking using the presented information. The results indicate that no performance improvements or mental workload reductions were gained from the spatial presentation method compared to a non-spatial layout for our problem-solving task. The results of these two experiments suggest that supplemental spatial information can support performance improvements for cognitive processing and learning-based activities, but its effectiveness is dependent on the nature of the task and a meaningful use of space

TangiBoard: a toolkit to reduce the implementation burden of tangible user interfaces in education

The use of Tangible User Interfaces (TUI) as an educational technology has gained sustained interest over the years with common agreement on its innate ability to engage and intrigue students in active-learning pedagogies. Whilst encouraging results have been obtained in research, the widespread adoption of TUI architectures is still hindered by a myriad of implementation burdens imposed by current toolkits. To this end, this paper presents an innovative TUI toolkit: TangiBoard, which enables the deployment of an interactive TUI system using low-cost, and presently available educational technology. Apart from curtailing setup costs and technical expertise required for adopting TUI systems, the toolkit provides an application framework to facilitate system calibration and development integration with GUI applications. This is enabled by a robust computer vision application that tracks a contributed passive marker set providing a range of tangible interactions to TUI frameworks. The effectiveness of this toolkit was evaluated by computer systems developers with respect to alternate toolkits for TUI design. Open-source versions of the TangiBoard toolkit together with marker sets are provided online through research licens

Designing a marker set for vertical tangible user interfaces

Tangible User Interfaces (TUI)s extend the domain of reality-based human-computer interaction by providing users the ability to manipulate digital data using physical objects which embody representational significance. Whilst various advancements have been registered over the past years through the development and availability of TUI toolkits, these have mostly converged towards the deployment of tabletop TUI architectures. In this context, markers used in current toolkits can only be placed underneath the tangible objects to provide recognition. Albeit being effective in various literature studies, the limitations and challenges of deploying tabletop architectures have significantly hindered the proliferation of TUI technology due to the limited audience reach such systems can provide. Furthermore, available marker sets restrict the placement and use of tangible objects since if placed on top of the tangible object, the marker will interfere with the shape and texture of the object limiting the effect the TUI has on the end-user. To this end, this paper proposes the design and development of an innovative tangible marker set specifically designed towards the development of vertical TUIs. The proposed marker set design was optimized through a genetic algorithms to ensure robustness in scale invariance, the capability of being successfully detected with distances of up to 3.5 meters and a true occlusion resistance of up to 25%, where the marker is recognized and not tracked. Open-source versions of the marker set are provided through research license on www.geoffslab.com/tangiboard_marker_set

TangiBoard: a toolkit to reduce the implementation burden of tangible user interfaces in education

The use of Tangible User Interfaces (TUI) as an educational technology has gained sustained interest over the years with common agreement on its innate ability to engage and intrigue students in active-learning pedagogies. Whilst encouraging results have been obtained in research, the widespread adoption of TUI architectures is still hindered by a myriad of implementation burdens imposed by current toolkits. To this end, this paper presents an innovative TUI toolkit: TangiBoard, which enables the deployment of an interactive TUI system using low-cost, and presently available educational technology. Apart from curtailing setup costs and technical expertise required for adopting TUI systems, the toolkit provides an application framework to facilitate system calibration and development integration with GUI applications. This is enabled by a robust computer vision application that tracks a contributed passive marker set providing a range of tangible interactions to TUI frameworks. The effectiveness of this toolkit was evaluated by computer systems developers with respect to alternate toolkits for TUI design. Open-source versions of the TangiBoard toolkit together with marker sets are provided online through research licens

Tangible vs. Virtual Representations: when Tangibles Benefit the Training of Spatial Skills

Tangible user interfaces (TUIs) have been the focus of much attention in the HCI and learning communities because of their many potential benefits for learning. However, there have recently been debates about whether TUIs can actually increase learning outcomes and if so, under which conditions. In this article, we investigate the effect of object representation (physical vs. virtual) on learning in the domain of spatial skills. We ran a comparative study with 46 participants to measure the effects of the object representation on the ability to establish a link between 2D and 3D representations of an object. The participants were split into two conditions: in the first one, the 3D representation of the object was virtual; in the second one, it was tangible. Findings show that in both conditions the TUI led to a significant improvement of the spatial skills. The learning outcomes were not different between the two conditions, but the performance during the activities was significantly higher when using the tangible representation as opposed to the virtual one, and even more so in for difficult cases

Parental Spatial Input During Parent-Child Interactions: A Two-Dimensional versus a Three-Dimensional Learning Experience

Children’s spatial ability is predictive of their future achievement in many academic and occupational domains, including science, technology, engineering, and mathematics (STEM; e.g., Wai at el., 2009). During the early years, experiences such as hearing spatial language (e.g., Ferrara et al., 2011) and engaging in spatial activities with three-dimensional (3D) blocks or puzzles (e.g., Casey et al., 2008) are found to facilitate children’s spatial learning. Other than 3D toys, the use of two-dimensional (2D) touchscreen media (e.g., iPads®) by young children has been on the rise (e.g., Rideout, 2013). Technology has become part of children’s daily activities and a tool to promote language learning (e.g., Penuel et al., 2009). However, there is a dearth of research specifically investigating the nature of parent-child interactions and children’s spatial learning using digital mobile devices. Therefore, the present study examined the frequency and variation of parental linguistic input elicited during play using an iPad® (a 2D touchscreen device) and using 3D spatial toys. In addition to the types of spatial learning (3D versus 2D), factors such as parents’ spatial anxiety and attitudes towards math can also influence their spatial language production. Research suggests that one’s attitude or anxiety towards mathematics can influence the amount of numeracy talk in which individuals engage (e.g., Gunderson et al., 2013). However, no studies have examined the relationship between spatial anxiety and spatial talk. The present study examined whether the amount of parental spatial talk was influenced by their attitudes towards math, spatial anxiety. The present exploratory study has three objectives: (i) to examine the frequency and variation of parental spatial language during 3D spatial toys versus 2D iPad® visual-spatial applications interactions with their preschoolers, (ii) to investigate whether parental spatial input (i.e., language and activities) predicts children’s spatial knowledge, and (iii) to explore the role of parental spatial anxiety and attitude towards mathematics on their spatial language input. Thirty-four 3- to 5-year-old children and their parents participated in interaction with 3D and 2D spatial learning media at two home visit sessions. Math and spatial activities engaged by the dyads at home, parental level of spatial anxiety, and attitude towards math were assessed. Children were tested with the Woodcock Johnson III Tests (Woodcock et al., 2001) for spatial, math, language competencies, and working memory capacity. Their spatial abilities were also assessed via 3D Mega Blocks© Test of Spatial Assembly (TOSA; Verdine et al., 2014). The sessions were videotaped, transcribed, and coded for the frequency and variation of spatial talk produced by parent-child dyads. Results revealed that parents used more spatial talk with regards to spatial dimensions in 3D interaction and more orientations and transformations during 2D interaction, yet the total frequency and variation of parental spatial talk did not differ between 3D and 2D interaction. As parents engaged in a relatively infrequent spatial talk (6% in 3D talk and 5% in 2D talk), the frequency of parental spatial input was not predictive of preschoolers’ spatial language production, which led to a minimal effect on their spatial competence. Furthermore, parental levels of spatial anxiety and attitudes toward math were not related to the amount of parental spatial input produced during parent-child interactions. The present study underscores the importance of supporting parents with pointers on how to instill spatial talk and activities with their preschoolers. Implications on the use of 3D and 2D learning media are discussed

Investigation and development of a tangible technology framework for highly complex and abstract concepts

The ubiquitous integration of computer-supported learning tools within the educational domain has led educators to continuously seek effective technological platforms for teaching and learning. Overcoming the inherent limitations of traditional educational approaches, interactive and tangible computing platforms have consequently garnered increased interest in the pursuit of embedding active learning pedagogies within curricula. However, whilst Tangible User Interface (TUI) systems have been successfully developed to edutain children in various research contexts, TUI architectures have seen limited deployment towards more advanced educational pursuits. Thus, in contrast to current domain research, this study investigates the effectiveness and suitability of adopting TUI systems for enhancing the learning experience of abstract and complex computational science and technology-based concepts within higher educational institutions (HEI)s. Based on the proposal of a contextually apt TUI architecture, the research describes the design and development of eight distinct TUI frameworks embodying innovate interactive paradigms through tabletop peripherals, graphical design factors, and active tangible manipulatives. These computationally coupled design elements are evaluated through summative and formative experimental methodologies for their ability to aid in the effective teaching and learning of diverse threshold concepts experienced in computational science. In addition, through the design and adoption of a technology acceptance model for educational technology (TAM4Edu), the suitability of TUI frameworks in HEI education is empirically evaluated across a myriad of determinants for modelling students’ behavioural intention. In light of the statistically significant results obtained in both academic knowledge gain (μ = 25.8%) and student satisfaction (μ = 12.7%), the study outlines the affordances provided through TUI design for various constituents of active learning theories and modalities. Thus, based on an empirical and pedagogical analyses, a set of design guidelines is defined within this research to direct the effective development of TUI design elements for teaching and learning abstract threshold concepts in HEI adaptations

Augmented Reality to Facilitate a Conceptual Understanding of Statics in Vocational Education

At the core of the contribution of this dissertation there is an augmented reality (AR) environment, StaticAR, that supports the process of learning the fundamentals of statics in vocational classrooms, particularly in carpentry ones. Vocational apprentices are expected to develop an intuition of these topics rather than a formal comprehension. We have explored the potentials of the AR technology for this pedagogical challenge. Furthermore, we have investigated the role of physical objects in mixed-reality systems when they are implemented as tangible user interfaces (TUIs) or when they serve as a background for the augmentation in handheld AR. This thesis includes four studies. In the first study, we used eye-tracking methods to look for evidences of the benefits associated to TUIs in the learning context. We designed a 3D modelling task and compared users' performance when they completed it using a TUI or a GUI. The gaze measures that we analysed further confirmed the positive impact that TUIs can have on the learners' experience and enforced the empirical basis for their adoption in learning applications. The second study evaluated whether the physical interaction with models of carpentry structures could lead to a better understanding of statics principles. Apprentices engaged in a learning activity in which they could manipulate physical models that were mechanically augmented, allowing for exploring how structures react to external loads. The analysis of apprentices' performance and their gaze behaviors highlighted the absence of clear advantages in exploring statics through manipulation. This study also showed that the manipulation might prevent students from noticing aspects relevant for solving statics problems. From the second study we obtained guidelines to design StaticAR which implements the magic-lens metaphor: a tablet augments a small-scale structure with information about its structural behavior. The structure is only a background for the augmentation and its manipulation does not trigger any function, so in the third study we asked to what extent it was important to have it. We rephrased this question to whether users would look directly at the structure instead of seeing it only through a tablet. Our findings suggested that a shift of attention from the screen to the physical object (a structure in our case) might occur in order to sustain users' spatial orientation when they change positions. In addition, the properties of the gaze shift (e.g. duration) could depend on the features of the task (e.g. difficulty) and of the setup (e.g. stability of the augmentation). The focus of our last study was the digital representation of the forces that act in a loaded structure. From the second study we observed that the physical manipulation failed to help apprentices understanding the way the forces interact with each other. To overcome this issue, our solution was to combine an intuitive representation (springs) with a slightly more formal one (arrows) which would show both the nature of the forces and the interaction between them. In this study apprentices used the two representations to collaboratively solve statics problems. Even though apprentices had difficulties in interpreting the two representations, there were cases in which they gained a correct intuition of statics principles from them. In this thesis, besides describing the designed system and the studies, implications for future directions are discussed

Tangible Interfaces for Learning:Training Spatial Skills in Vocational Classrooms

There have been many claims that Tangible User Interfaces (TUIs) can have a positive impact on learning. Alleged benefits include increasing usability, improving engagement and collaboration of students, and providing a better perception of the task, especially spatial ones. However, there exists little empirical data to back up these claims. Moreover, for all their potential benefits for learning, TUIs are still scarcely used in schools. This thesis explores these two issues in the specific context of vocational education and training of carpenter apprentices. The learning objectives concern spatial skills and in particular, the mapping between 2D and 3D representations. We study (1) whether TUIs can support the training of spatial skills, and if so what features allow them to do so, and (2) what kinds of classroom pedagogical scenarios TUIs can support. We follow a design-based research approach and run empirical studies, mostly in classrooms. The contributions of this thesis touch on three research domains: 1. Spatial skills. Our results show that TUIs can help teach spatial skills to carpenter apprentices. The tangible nature of TUIs can help the learner relate multiple representations of an object, especially for difficult problems. It can also lower the barrier to entry into a learning domain for beginners. 2. Learning with TUIs. According to our results, TUIs can benefit learning, but the mere fact of using TUIs does not guarantee learning. Instead, special attention needs to be given to the design of the TUI. Small design variations, such as the physical correspondence between the tangible object and its virtual representations, or the type and timing of feedback given to the user, can have a significant impact on learning. 3. Classroom technologies and orchestration. We explore several classroom pedagogical scenarios that TUIs can support. The most promising one is to use a TUI as part of a hybrid classroom learning activity that includes both TUI and non TUI steps. Additionally, we devise two ways to promote the integration of TUIs in classroom. First, we introduce 5 design principles that reduce the classroom orchestration load. Second, we show how new web technologies can be used to deploy TUIs in schools at a lower cost