SELF-IMAGE MULTIMEDIA TECHNOLOGIES FOR FEEDFORWARD OBSERVATIONAL LEARNING

This dissertation investigates the development and use of self-images in augmented reality systems for learning and learning-based activities. This work focuses on self- modeling, a particular form of learning, actively employed in various settings for therapy or teaching. In particular, this work aims to develop novel multimedia systems to support the display and rendering of augmented self-images. It aims to use interactivity (via games) as a means of obtaining imagery for use in creating augmented self-images. Two multimedia systems are developed, discussed and analyzed. The proposed systems are validated in terms of their technical innovation and their clinical efficacy in delivering behavioral interventions for young children on the autism spectrum

Holographic reality: enhancing the artificial reality experience throuhg interactive 3D holography

Holography was made know by several science-fiction productions, however this technology dates back to the year 1940. Despite the considerable age of this discovery, this technology remains inaccessible to the average consumer. The main goal of this manuscript is to advance the state of the art in interactive holography, providing an accessible and low-cost solution. The final product intends to nudge the HCI com munity to explore potential applications, in particular to be aquatic centric and environmentally friendly. Two main user studies are performed, in order to determine the impact of the proposed solution by a sample audience. Provided user studies include a first prototype as a Tangible User Interface - TUI for Holographic Reality - HR Second study included the Holographic Mounted Display - HMD for proposed HR interface, further analyzing the interactive holographic experience without hand-held devices. Both of these studies were further compared with an Augmented Reality setting. Obtained results demonstrate a significantly higher score for the HMD approach. This suggests it is the better solution, most likely due to the added simplicity and immersiveness features it has. However the TUI study did score higher in several key parameters, and should be considered for future studies. Comparing with an AR experience, the HMD study scores slightly lower, but manages to surpass AR in several parameters. Several approaches were outlined and evaluated, depicting different methods for the creation of Interactive Holographic Reality experiences. In spite of the low maturity of holographic technology, it can be concluded it is comparable and can keep up to other more developed and mature artificial reality settings, further supporting the need for the existence of the Holographic Reality conceptA tecnologia holográfica tornou-se conhecida através da ficção científica, contudo esta tecnologia remonta até ao ano 1940. Apesar da considerável idade desta descoberta, esta tecnologia continua a não ser acessíveil para o consumidor. O objetivo deste manuscrito é avançar o estado de arte da Holografia Interactiva, e fornecer uma solução de baixo custo. O objetivo do produto final é persuadir a comunidade HCI para a exploração de aplicações desta tecnologia, em particular em contextos aquáticos e pró-ambientais. Dois estudos principais foram efetuados, de modo a determinar qual o impacto da solução pro posta numa amostra. Os estudos fornecidos incluem um protótipo inicial baseado numa Interface Tangível e Realidade Holográfica e um dispositivo tangível. O segundo estudo inclui uma interface baseada num dispositivo head-mounted e em Realidade Holográfica, de modo a analisar e avaliar a experiência interativa e holográfica. Ambos os estudos são comparados com uma experiência semelhante, em Realidade Aumentada. Os resultados obtidos demonstram que o estudo HMD recebeu uma avaliação significante mel hor, em comparação com a abordagem TUI. Isto sugere que uma abordagem "head-mounted" tende a ser melhor solução, muito provavelmente devido às vantagens que possui em relação à simplicidade e imersividade que oferece. Contudo, o estudo TUI recebeu pontuações mais altas em alguns parâmetros chave, e deve ser considerados para a implementação de futuros estudos. Comparando com uma experiência de realidade aumentada, o estudo HMD recebeu uma avaliação ligeiramente menor, mas por uma margem mínima, e ultrapassando a AR em alguns parâmetros. Várias abordagens foram deliniadas e avaliadas, com diferentes métodos para a criação de experiências de Realidade Holográfica. Apesar da pouca maturidade da tecnologia holográfica, podemos concluir que a mesma é comparável e consegue acompanhar outros tipos de realidade artificial, que são muito mais desenvolvidos, o que suporta a necessidade da existência do conceito de Realidade Holográfica

The Impossible Qualities Of Illusionary Spaces: Stop Motion Animation, Visual Effects And Metalepsis

This thesis examines stop motion animation, its role as a special effect and how the stop motion form impacts on narrative. In particular, it is concerned with the relationship between stop motion animation and the rhetorical concept of metalepsis, as well as the disruption and transgression of narrative spaces in fiction. The studio component of the work is an installation titled All The Nice Things Come From Here which uses an early film special effects technique, the Schüfftan process. The Schüfftan process is a form of in-camera compositing that uses mirrors to align two separate spaces to form the illusion of one cohesive space. The installation uses Newcastle’s light industrial landscape as a backdrop to create impossible miniature narrative spaces that can only be understood when the viewer is aligned to a station point forced by the placement of the mirrors. The theoretical portion of the thesis examines how this exploded view of an animated special effect can be used to explore ideas of narrative, narrative layers and the visual forms of stop motion animation. The thesis argues that object stop motion animation has aspects that are inherently metaleptic, as stop motion’s use of real objects doing impossible things creates its own subtle and impossible metaleptic spaces that simultaneously refer to both the world within the film and the world outside the film

Proceedings of the International Workshop “Re-Thinking Technology in Museums: towards a new understanding of people’s experience in museums"

Proceedings of the International Workshop “Re-Thinking Technology in Museums: towards a new understanding of people’s experience in museums

Viatopias: Exploring the experience of urban travel space

The title of this research is constructed from: `via' - route and töp(os) -a place. Viatopias are urban spaces of continual travel or flux that incorporate multiple forms of perception and inscriptions of meaning. My aim has been to define and describe the increasingly important fluid perceptual spaces that have developed between static nineteenth century destinations. Viatopias such as passageways, underground tunnels, train tracks, and the North Circular escape a sense of destination, operating as ever-changing experiences or events. The practice has sought to produce digital representations of these urban travel spaces that exist in constant flux, to communicate the experience of Viatopias. The research explores themes such as: The North Circular as a Deleuzian Route exploring driving as performance; Plica, Replica, Explica an unfolding of experience through digital media; The Making of Baroque Videos, using Baroque architectures of viewing; Mobilizing Perception treating human vision as an artifact; Mirrors For Un-Recognition disassembling nineteenth century controlled vision; Sound as an Urban Compass considering urban audio experience; Narrative Practice in New Media Space analysing contemporary approaches in digital media; and Convergent Languages, Digital Poiesis investigating the dislocation of representation in different digital languages. These conceptual frameworks developed in symbiosis with the practice. The visual practice presents a collection of digital videos that extend and complicate these concepts through experimental visual and audio techniques such as layering, repetition, anamorphic distortion, and mirroring to produce visual immersion and the fracturing of space. The concluding digital works incorporate video with audio and text resulting in integrated visual statements that attempt to stretch the viewer's perception, in the process offering a glimpse of a new experience within urban space

Creative interactions between visual technology and sound art in a live performance context

For introducing this thesis, it is important to place it in a context of: models of interactivity – systems, systemic work, mediation, participation, collaboration, interfaces, dance, drawing, video, sound, interaction, multimedia, Digital versus analogue, layers, physical time versus performance and film time, physical space versus virtual space, material (paper ink etc.) versus light projection and sound, wearable technology, and to briefly pinpoint artists who are working in relevant ways for the discussion arising. Within dance itself the use of new technology is also varied and has seen the creation of different contexts for Dance The main question is on how to achieve a creative solution for nonintrusive technology for dance expression and a dialogue between different parts like drawing, video, dance improvisation, interactive performative dialogues and sound. By nonintrusive technology one means technology that does not physically interfere with the ability a dancer has to move freely like cables, wires or heavy suits that restrict movement instead of promoting it. Finding a research framework that goes towards this idea is at the forefront of this research as well as testing a system that goes towards this idea of allowing the dancers to perform freely without physical constrictions or barriers. The main research question is therefore split into several smaller items that will be looked at within the body of this research like where did the interaction between Dance and Technology had its roots? This research focuses on finding a way to marry video technology, drawing, sound and live performance from the performer’s point of view, to enhance the performer´s ability to interact with the elements, for the performers freedom to move without technological constrictions. This project is about creating a situation that allows for observation of the performers, exploring and educating themselves through experimenting and experiencing technology brought together thinking about movement itself. It uses improvisation as a language tool to facilitate a dialogue between technology and live performance. ‘LFPF’ is proposing to develop a new approach to relationship between performer, stage and sound better still between performance and its special environment/context. It offers innovative possibilities without being intimidating. Once in place (once set up) it provides a new approach for artists to collaborate, it provides a common ground for Dance Performance and Fine/Sound Artists to merge their skills through experimentation with a live performance as the ultimate goal. ‘LFPF’ – Live Film Performance Facilitator tests and provides a new approach to the relationship between live performance, film and sound when combined in live performance situations. It enables the use of improvisation as performance language, giving birth to a constructive dialogue between video technology and live performance. ‘LFPF’ is a structured environment for exploration of movement through film concepts with the input given by drawings. It enables a live video to develop in front of the audience. It enables the performers to inhabit a truly responsive environment that changes depending on their reaction. It is a twoway dialogue between performers and spatial context powered by a second performer inputting drawings. It also frees the performers from directly having to manipulate any technology and deal with just the artistic result of it. It is therefore a human mediated technological environment. ‘LFPF’ was tested in a series of performances. There are various ways in which Video and Technology are nowadays integrated into Live Performance. It is only natural that the marriage between these elements is going to create an increasingly fluid language for the future. I feel it is essential to think about creating a performance language which supports the ideas of artists, therefore the technology at the service of the artists and never the other way around. That is the idea when I refer to technology in the introduction as a tool for developing creativity.Para realizar a apresentação desta tese será necessário, antes de mais, colocála no seu devido contexto: modelos de interactividade – sistemas, trabalho sistémico, mediação, participação, multimédia, representação digital versus analógica, camadas, tempo físico versus tempo de performance e vídeo, espaço físico versus espaço virtual, material (papel, tinta, etc.) versus projecção de luz e som, tecnologia vestível, e salientar resumidamente quais os artistas que se encontram a desenvolver trabalho relevante na presente área em discussão. Dentro da área da dança em si, já se assiste também à utilização de variadas novas tecnologias e à criação de diferentes contextos. A principal questão que aqui se coloca, é como alcançar uma solução criativa relativa às tecnologias não invasivas para a expressão na dança, e como estabelecer um diálogo entre as diferentes partes, como o desenho, o vídeo, a improvisação na dança, os diálogos interactivos e performativos e o som. Quando fazemos uso do termo “tecnologia não invasiva”, estamos a referirnos concretamente a qualquer tecnologia que não interfira com a integridade física de um dançarino e com a sua capacidade de se mover livremente, tais como cabos, fios ou roupas pesadas, que restrinjam os movimentos em vez de os fazerem fluir. Esta constitui a base do meu projecto: encontrar um quadro de investigação que vá ao encontro desta ideia, assim como encontrar um sistema de ensaio que permita ao dançarino actuar livremente, sem condicionantes ou barreiras físicas. O tema principal de investigação encontrase, como tal, dividido em vários pontos, que irão ser analisados individualmente ao longo da tese, como por exemplo: quando é que surgiu a interacção entre a dança e a tecnologia? O enfoque principal desta investigação é o de tentar casar a tecnologia de vídeo, desenho, som e performance ao vivo do ponto de vista do performer, de forma a melhorar a sua capacidade de interacção com os outros elementos, com total liberdade de movimentos e sem constrições tecnológicas. Este projecto pretende criar uma situação que permita a observação dos performers, explorando e educandose a si mesmos, através da experimentação e do uso de tecnologia aliadas ao movimento. A improvisação é utilizada como uma ferramenta de diálogo, de forma a facilitar a comunicação entre a tecnologia e a performance ao vivo. O ‘LFPF’ propõe desenvolver uma nova abordagem para a relação entre o performer, palco, som e ainda, entre a performance e o seu ambiente/contexto particular. Oferece possibilidades inovadoras sem se tornar intimidante. Uma vez colocado (devidamente configurado), proporciona novas possibilidades de colaboração entre os artistas e uma base comum para que o espectáculo de dança e os artistas plásticos/sonoros possam fundir artes, através da experimentação e do objectivo último: a performance ao vivo. O ‘LFPF’ (Live Film Performance Facilitator) testa e oferece uma nova abordagem à relação entre a performance ao vivo, filme e som, quando combinados em performances ao vivo. Permite a improvisação como linguagem de performance, dando origem a um diálogo constructivo entre a tecnologia de vídeo e a performance ao vivo. O ‘LFPF’ é um ambiente estruturado, destinado à exploração do movimento através da projecção de vídeo, cujo input é fornecido por desenhos. Desta forma é possível desenvolver um vídeo em directo para uma audiência, permitindo que os performers se movam num ambiente verdadeiramente interactivo, que se altera de acordo com a sua reacção. É, portanto, um diálogo nos dois sentidos, entre performers e contexto espacial, complementado por um segundo performer que nos vai apresentando desenhos. Esta dinâmica liberta também os performers de manusearem directamente qualquer tipo de tecnologia, lidando apenas com o respectivo resultado artístico. Tratase assim de um ambiente tecnológico mediado pela acção humana. O ‘LFPF’ já foi testado em vários espectáculos. Hoje em dia, as componentes de vídeo e tecnológica encontramse integradas de várias formas nas performances ao vivo. É, por isso, perfeitamente natural que o casamento entre estes elementos vá criar uma linguagem cada vez mais fluída no futuro. Creio que é fundamental pensar em criar uma linguagem de performance que harmonize as ideias dos artistas, no sentido de colocar a tecnologia ao seu dispor e não o inverso. É este o princípio base que defendo quando, ao longo da introdução, faço referência à tecnologia como uma ferramenta para desenvolver a criatividade

Towards System Agnostic Calibration of Optical See-Through Head-Mounted Displays for Augmented Reality

This dissertation examines the developments and progress of spatial calibration procedures for Optical See-Through (OST) Head-Mounted Display (HMD) devices for visual Augmented Reality (AR) applications. Rapid developments in commercial AR systems have created an explosion of OST device options for not only research and industrial purposes, but also the consumer market as well. This expansion in hardware availability is equally matched by a need for intuitive standardized calibration procedures that are not only easily completed by novice users, but which are also readily applicable across the largest range of hardware options. This demand for robust uniform calibration schemes is the driving motive behind the original contributions offered within this work. A review of prior surveys and canonical description for AR and OST display developments is provided before narrowing the contextual scope to the research questions evolving within the calibration domain. Both established and state of the art calibration techniques and their general implementations are explored, along with prior user study assessments and the prevailing evaluation metrics and practices employed within. The original contributions begin with a user study evaluation comparing and contrasting the accuracy and precision of an established manual calibration method against a state of the art semi-automatic technique. This is the first formal evaluation of any non-manual approach and provides insight into the current usability limitations of present techniques and the complexities of next generation methods yet to be solved. The second study investigates the viability of a user-centric approach to OST HMD calibration through novel adaptation of manual calibration to consumer level hardware. Additional contributions describe the development of a complete demonstration application incorporating user-centric methods, a novel strategy for visualizing both calibration results and registration error from the user’s perspective, as well as a robust intuitive presentation style for binocular manual calibration. The final study provides further investigation into the accuracy differences observed between user-centric and environment-centric methodologies. The dissertation concludes with a summarization of the contribution outcomes and their impact on existing AR systems and research endeavors, as well as a short look ahead into future extensions and paths that continued calibration research should explore