A review of augmented reality applications for ship bridges

We present a state-of-the art analysis of Augmented Reality (AR) applications for ship bridge operation. We compiled and reviewed what type of use cases were published, what type of maritime applications have been adapted to AR, how they were prototyped and evaluated and what type of technology was used. We also reviewed the user interaction mechanisms, information display and adaptation to maritime environmental conditions. Our analysis shows that although there are many examples of AR applications in ship bridges, there is still much work that needs to be done before these solutions can be suitably adapted to commercial settings. In addition, we argue there is a need to develop design requirements and regulations that can guide the safe development of AR

Performative Authoring: Nurturing Children’s Creativity and Creative Self-Efficacy through Digitally-Augmented Enactment-Based Storytelling

Psychological research, especially by Paul Torrance, has shown that the child’s engagement in creative activities tends to drop precipitously at around the 3rd- to 4th-grade period (8 to 11 years old). This phenomenon, called the ‘Fourth-Grade Slump’, occurs possibly because of an increase in social awareness and critical self-evaluation of competence in the child during this period. Increasing awareness of the complexity of the world’s problems, new paradigms of design focusing on the user, and advances in technology has led to rapid developments in the design and development of tools to support children’s creativity. Research in creativity support tools has generally focused on augmenting creative performance within specific tasks, using strategies such as facilitating access to information, or exposing the user to a space of possible combinations. Much less studied however, is how tools may help to develop positive attitudes towards creativity in individuals. This is important, especially in systems designed for children where the focus on the development of the person, during critical periods of growth such as the period of the Fourth-Grade Slump, may be said to be of equivalent or greater importance than the support of process or the generation of product. In the domain of storytelling or narrative construction, work in child development, educational pedagogy, social psychology, and performance studies have looked at how to tap into the power of children’s imagination during pretend play to nurture their storytelling abilities and their sense of self-efficacy or confidence. These interventions typically take the form of drama workshops or classroom roleplaying exercises. While results appear to provide good evidence that drama interventions and theater-based methods have some positive effects on children’s development of narratives, studies have shown mixed results in terms of the effects on children’s self-efficacy. I refer here to self-efficacy in the sense of a child’s perception of her creative abilities, in other words, her belief that she can produce creative outcomes. This creativity-oriented sense of self-efficacy has been called ‘creative self-efficacy’. This dissertation investigates how pretend play can be harnessed into the design of an interface to support children’s creativity in storytelling and their sense of creative self-efficacy. This overarching question was explored through four phases of research: Exploration, Design, Evaluation, and Integration. The Exploration phase consisted of two studies: a) a set of interviews with elementary school teachers, and b) an experimental study of how the interface or medium may affect children’s creative storytelling process; The Design phase consisted of two experimental studies, and design and development: a) the first study investigated how the physicality of props may support children’s enactment-based storytelling, and b) the second study explored the influence of the presentation of digital contextual/environmental cues on children’s enactment-based storytelling, c) design and development consisted of an exercise using the NEVO methodology to embody design knowledge gained from the Design phase into a concrete usable system, called DiME; The Evaluation phase consisted of two studies: a) the first was a pilot study that tested the usability of DiME and protocol of use with children, and b) the second was an experimental study across two school districts with different profiles investigating the effects of digitally-augmented enactment-based storytelling using DiME, on children’s creativity, story writing and creative self-efficacy; The Integration phase of the research consisted of a workshop with elementary school teachers, which initiated an exploration into how such a story authoring approach may be used in an elementary school curriculum and setting. The body of work that this dissertation presents elucidates (i) a physical enactment-based method for the authoring of stories by children, and (ii) how a digitally-augmented space may move beyond simple drama methods to positively influence the child’s creativity and imagination during storytelling, as well as her self-belief and motivation to engage in creation. The digitally-augmented enactment-based storytelling environment, that I term performative authoring, allows the child to collaboratively create a story through pretend play with a partner, while her enactments are reflected in real-time in the form of animated cartoon characters and objects on a large screen display through the use of motion tracking technologies. I have found that performative authoring has positive effects not only on the child’s creative self-efficacy, especially for the less extraverted children, but also on the richness of the child’s retelling or written narrative of her story. The significance of the results of the studies is with respect to the various domains and subareas represented (child-computer interaction, interactive storytelling, education and educational psychology, creativity and cognition). There is great potential to extend the concept of exploiting digitally-augmented enactment to support and scaffold higher-level cognition, beyond physical enactment. Extensions of this work include making use of more epistemic forms of enactment, instead of full-blown enactment, to support children’s creative story brainstorming, or to make use of digitally-augmented enactment to support other forms of higher thought apart from creativity and imagination. In the domain of storytelling or narrative construction, work in child development, educational pedagogy, social psychology, and performance studies have looked at how to tap into the power of children’s imagination during pretend play to nurture their storytelling abilities and their sense of self-efficacy or confidence. These interventions typically take the form of drama workshops or classroom roleplaying exercises. While results appear to provide good evidence that drama interventions and theater-based methods have some positive effects on children’s development of narratives, studies have shown mixed results in terms of the effects on children’s self-efficacy. I refer here to self-efficacy in the sense of a child’s perception of her creative abilities, in other words, her belief that she can produce creative outcomes. This creativity-oriented sense of self-efficacy has been called ‘creative self-efficacy’ (Beghetto, 2006, 2007). This dissertation investigates how pretend play can be harnessed into the design of an interface to support children’s creativity in storytelling and their sense of creative self-efficacy. This overarching question was explored through four phases of research: I. Exploration, II. Design, III. Evaluation, and IV. Integration. Phase 1 Exploration consisted of two studies: 1A) a set of interviews with elementary school teachers, and 1B) an experimental study of how the interface or medium may affect children’s creative storytelling process; Phase 2 Design consisted of two experimental studies, and design and development: 2A) the first study investigates how the physicality of props may support children’s enactment-based storytelling, and 2B) the second study explores the influence of the presentation of digital contextual/environmental cues on children’s enactment-based storytelling, 2C) design and development consisted of an exercise using the NEVO methodology to embody design knowledge gained from Phase 2 into a concrete usable system, called DiME; Phase 3 Evaluation consisted of two studies: 3A) the first was a pilot study that tested the usability of DiME and protocol of use with children, and 3B) the second was an experimental study across two school districts with different profiles investigating the effects of digitally-augmented enactment-based storytelling using DiME, on children’s creativity, story writing and creative self-efficacy; and Phase 4 Integration consisted of a workshop with elementary school teachers, which initiated an exploration into how such a story authoring approach may be used in an elementary school curriculum and setting. The body of work that this dissertation presents elucidates (i) a physical enactment-based method for the authoring of stories by children, and (ii) how a digitally-augmented space may move beyond simple drama methods to positively influence the child’s creativity and imagination during storytelling, as well as her self-belief and motivation to engage in creation. The digitally-augmented enactment-based storytelling environment, termed performative authoring in this document, allows the child to collaboratively create a story through pretend play with a partner, while her enactments are reflected in real-time in the form of animated cartoon characters and objects on a large screen display through the use of motion tracking technologies. I have found that performative authoring has positive effects not only on the child’s creative self- efficacy, especially for the less extraverted children, but also on the richness of the child’s retelling or written narrative of her story. This dissertation concludes by discussing the significance of the results of our studies with respect to the various domains and subareas represented (child-computer interaction, interactive storytelling, education and educational psychology, creativity and cognition) and extends the concept of exploiting digitally-augmented enactment to support and scaffold higher-level cognition, beyond physical enactment. Extensions of this work include making use of more epistemic forms of enactment, instead of full-blown enactment, to support children’s creative story brainstorming, or to make use of digitally-augmented enactment to support other forms of higher thought apart from creativity and imagination

Towards exploring future landscapes using augmented reality

With increasing pressure to better manage the environment many government and private organisations are studying the relationships between social, economic and environmental factors to determine how they can best be optimised for increased sustainability. The analysis of such relationships are undertaken using computer-based Integrated Catchment Models (ICM). These models are capable of generating multiple scenarios depicting alternative land use alternatives at a variety of temporal and spatial scales, which present (potentially) better Triple-Bottom Line (TBL) outcomes than the prevailing situation. Dissemination of this data is (for the most part) reliant on traditional, static map products however, the ability of such products to display the complexity and temporal aspects is limited and ultimately undervalues both the knowledge incorporated in the models and the capacity of stakeholders to disseminate the complexities through other means. Geovisualization provides tools and methods for disseminating large volumes of spatial (and associated non-spatial) data. Virtual Environments (VE) have been utilised for various aspects of landscape planning for more than a decade. While such systems are capable of visualizing large volumes of data at ever-increasing levels of realism, they restrict the users ability to accurately perceive the (virtual) space. Augmented Reality (AR) is a visualization technique which allows users freedom to explore a physical space and have that space augmented with additional, spatially referenced information. A review of existing mobile AR systems forms the basis of this research. A theoretical mobile outdoor AR system using Common-Of-The-Shelf (COTS) hardware and open-source software is developed. The specific requirements for visualizing land use scenarios in a mobile AR system were derived using a usability engineering approach known as Scenario-Based Design (SBD). This determined the elements required in the user interfaces resulting in the development of a low-fidelity, computer-based prototype. The prototype user interfaces were evaluated using participants from two targeted stakeholder groups undertaking hypothetical use scenarios. Feedback from participants was collected using the cognitive walk-through technique and supplemented by evaluator observations of participants physical actions. Results from this research suggest that the prototype user interfaces did provide the necessary functionality for interacting with land use scenarios. While there were some concerns about the potential implementation of "yet another" system, participants were able to envisage the benefits of visualizing land use scenario data in the physical environment

Keyframe Tagging: Unambiguous Content Delivery for Augmented Reality Environments

Context: When considering the use of Augmented Reality to provide navigation cues in a completely unknown environment, the content must be delivered into the environment with a repeatable level of accuracy such that the navigation cues can be understood and interpreted correctly by the user. Aims: This thesis aims to investigate whether a still image based reconstruction of an Augmented Reality environment can be used to develop a content delivery system that providers a repeatable level of accuracy for content placement. It will also investigate whether manipulation of the properties of a Spatial Marker object is sufficient to reduce object selection ambiguity in an Augmented Reality environment. Methods: A series of experiments were conducted to test the separate aspects of these aims. Participants were required to use the developed Keyframe Tagging tool to introduce virtual navigation markers into an Augmented Reality environment, and also to identify objects within an Augmented Reality environment that was signposted using different Virtual Spatial Markers. This tested the accuracy and repeatability of content placement of the approach, while also testing participants’ ability to reliably interpret virtual signposts within an Augmented Reality environment. Finally the Keyframe Tagging tool was tested by an expert user against a pre-existing solution to evaluate the time savings offered by this approach against the overall accuracy of content placement. Results: The average accuracy score for content placement across 20 participants was 64%, categorised as “Good” when compared with an expert benchmark result, while no tags were considered “incorrect” and only 8 from 200 tags were considered to have “Poor” accuracy, supporting the Keyframe Tagging approach. In terms of object identification from virtual cues, some of the predicted cognitive links between virtual marker property and target object did not surface, though participants reliably identified the correct objects across several trials. Conclusions: This thesis has demonstrated that accurate content delivery can be achieved through the use of a still image based reconstruction of an Augmented Reality environment. By using the Keyframe Tagging approach, content can be placed quickly and with a sufficient level of accuracy to demonstrate its utility in the scenarios outlined within this thesis. There are some observable limitations to the approach, which are discussed with the proposals for further work in this area

Holistic Approach for Authoring Immersive and Smart Environments for the Integration in Engineering Education

Die vierte industrielle Revolution und der rasante technologische Fortschritt stellen die etablierten Bildungsstrukturen und traditionellen Bildungspraktiken in Frage. Besonders in der Ingenieurausbildung erfordert das lebenslange Lernen, dass man sein Wissen und seine Fähigkeiten ständig verbessern muss, um auf dem Arbeitsmarkt wettbewerbsfähig zu sein. Es besteht die Notwendigkeit eines Paradigmenwechsels in der Bildung und Ausbildung hin zu neuen Technologien wie virtueller Realität und künstlicher Intelligenz. Die Einbeziehung dieser Technologien in ein Bildungsprogramm ist jedoch nicht so einfach wie die Investition in neue Geräte oder Software. Es müssen neue Bildungsprogramme geschaffen oder alte von Grund auf umgestaltet werden. Dabei handelt es sich um komplexe und umfangreiche Prozesse, die Entscheidungsfindung, Design und Entwicklung umfassen. Diese sind mit erheblichen Herausforderungen verbunden, die die Überwindung vieler Hindernisse erfordert. Diese Arbeit stellt eine Methodologie vor, die sich mit den Herausforderungen der Nutzung von Virtueller Realität und Künstlicher Intelligenz als Schlüsseltechnologien in der Ingenieurausbildung befasst. Die Methodologie hat zum Ziel, die Hauptakteure anzuleiten, um den Lernprozess zu verbessern, sowie neuartige und effiziente Lernerfahrungen zu ermöglichen. Da jedes Bildungsprogramm einzigartig ist, folgt die Methodik einem ganzheitlichen Ansatz, um die Erstellung maßgeschneiderter Kurse oder Ausbildungen zu unterstützen. Zu diesem Zweck werden die Wechselwirkung zwischen verschiedenen Aspekten berücksichtigt. Diese werden in den drei Ebenen - Bildung, Technologie und Management zusammengefasst. Die Methodik betont den Einfluss der Technologien auf die Unterrichtsgestaltung und die Managementprozesse. Sie liefert Methoden zur Entscheidungsfindung auf der Grundlage einer umfassenden pädagogischen, technologischen und wirtschaftlichen Analyse. Darüber hinaus unterstützt sie den Prozess der didaktischen Gestaltung durch eine umfassende Kategorisierung der Vor- und Nachteile immersiver Lernumgebungen und zeigt auf, welche ihrer Eigenschaften den Lernprozess verbessern können. Ein besonderer Schwerpunkt liegt auf der systematischen Gestaltung immersiver Systeme und der effizienten Erstellung immersiver Anwendungen unter Verwendung von Methoden aus dem Bereich der künstlichen Intelligenz. Es werden vier Anwendungsfälle mit verschiedenen Ausbildungsprogrammen vorgestellt, um die Methodik zu validieren. Jedes Bildungsprogramm hat seine eigenen Ziele und in Kombination decken sie die Validierung aller Ebenen der Methodik ab. Die Methodik wurde iterativ mit jedem Validierungsprojekt weiterentwickelt und verbessert. Die Ergebnisse zeigen, dass die Methodik zuverlässig und auf viele Szenarien sowie auf die meisten Bildungsstufen und Bereiche übertragbar ist. Durch die Anwendung der in dieser Arbeit vorgestellten Methoden können Interessengruppen immersiven Technologien effektiv und effizient in ihre Unterrichtspraxis integrieren. Darüber hinaus können sie auf der Grundlage der vorgeschlagenen Ansätze Aufwand, Zeit und Kosten für die Planung, Entwicklung und Wartung der immersiven Systeme sparen. Die Technologie verlagert die Rolle des Lehrenden in eine Moderatorrolle. Außerdem bekommen die Lehrkräfte die Möglichkeit die Lernenden individuell zu unterstützen und sich auf deren kognitive Fähigkeiten höherer Ordnung zu konzentrieren. Als Hauptergebnis erhalten die Lernenden eine angemessene, qualitativ hochwertige und zeitgemäße Ausbildung, die sie qualifizierter, erfolgreicher und zufriedener macht

Psychology applied : a fusion of abduction and ergonomics

The world has an increasing number of problems, many of which involve a human­ technological component, the solution of which requires strong methods. To explore problem solving methods, I proposed the development of an abductive-ergonomics framework. This framework must support the generation of theories that will support design. To achieve this, I discussed psychology's current method, the hypothetico­-deductive method, suggesting that it is not a general method. As an alternative I discussed abduction, which provides a strong general method. I also explored the aims, perspectives, and unit of analysis offered by ergonomics. In the last chapter I proposed that abduction provides the control task structure to the real world domain described by ergonomics. The fusion between abduction and ergonomics provides the basis for the real world problem solving framework proposed in this thesis

Designable Visual Markers for Mobile Human-Computer Interaction

Visual markers are graphic symbols designed to be easily recognised by machines. They are traditionally used to track goods, but there is increasing interest in their application to mobile human-computer interaction (HCI). By scanning a visual marker through a camera phone, users can retrieve localised information and access mobile services. In particular the dissertation examines the application of visual markers to physical tagging: practices of association of digital information with physical items. One missed opportunity in current visual marker systems is that the markers themselves cannot be visually designed; they are not expressive to humans, and thus fail to convey information before being scanned. To address this limitation, this dissertation introduces the idea of designable markers, visual markers that are both machine-readable and visually communicative to humans, and presents an investigation of the ways in which they can support mobile human-computer interaction. The application of designable visual markers to the creation of mobile interfaces is explored through a variety of methods: through formal usability experiments, through the creation and analysis of example designs, as well as through the qualitative analysis of two field trials. All three approaches were enabled by the engineering and development of d-touch, an actual recognition system that supports designable visual markers and by its integration in a variety of applications and experimental probes. D-touch is based on image topology, and its markers are defined in terms of constraints on the nesting of dark and light regions. The constraints imposed by d-touch are flexible enough to allow novice users to create markers which are visually expressive and at the same time machine readable. A user study demonstrates how such system enables people to design their own functional visual markers, determining their aesthetic qualities and what they visually communicate to others. A desktop application to support users in the creation of valid markers, the d-touch analyser, is presented and its usefulness is demonstrated through the same study. A formal usability experiment comparing five variations of marker-based interfaces on keypad and touch-screen phones shows that all of them allow users to reliably select targets within, on average, less than 4 seconds. Participants of the experiment reported a strong preference for interfaces that involve only marker scanning, compared to those that require a combination of marker scanning and key-presses or touch selections. Example designs of mobile interface generated by the author as well as others are presented to expose how the d-touch recognition system can be integrated in mobile applications. The examples illustrate a variety of ways in which markers can be used to augment printed materials such as cards, books and product packages, adding to them interactive capabilities. The examples show also different approaches to marker design, ranging from simple and recognisable iconic design, to symbols that integrate cues about the interactive functionality, to making them invisible by hiding them in existing graphics. Finally, the dissertation reports and analyses two field trials conducted to study what practices of physical tagging can emerge from, and be supported by, the use of markers. The trials were centred around the use of uWiki, a functional prototype based on d-touch, that allows users to associate digital content to markers printed on physical tags that can be affixed to objects or buildings. Observations show that a variety of practices emerge around the use of this technology, indicating that they provide a rich medium that has potential to attract the interest of real users. Though the results of this work are preliminary, they serve to demonstrate the range of potential for the future of such systems

Smart Technologies for Precision Assembly

This open access book constitutes the refereed post-conference proceedings of the 9th IFIP WG 5.5 International Precision Assembly Seminar, IPAS 2020, held virtually in December 2020. The 16 revised full papers and 10 revised short papers presented together with 1 keynote paper were carefully reviewed and selected from numerous submissions. The papers address topics such as assembly design and planning; assembly operations; assembly cells and systems; human centred assembly; and assistance methods in assembly