Scalable and Extensible Augmented Reality with Applications in Civil Infrastructure Systems.

In Civil Infrastructure System (CIS) applications, the requirement of blending synthetic and physical objects distinguishes Augmented Reality (AR) from other visualization technologies in three aspects: 1) it reinforces the connections between people and objects, and promotes engineers’ appreciation about their working context; 2) It allows engineers to perform field tasks with the awareness of both the physical and synthetic environment; 3) It offsets the significant cost of 3D Model Engineering by including the real world background. The research has successfully overcome several long-standing technical obstacles in AR and investigated technical approaches to address fundamental challenges that prevent the technology from being usefully deployed in CIS applications, such as the alignment of virtual objects with the real environment continuously across time and space; blending of virtual entities with their real background faithfully to create a sustained illusion of co- existence; integrating these methods to a scalable and extensible computing AR framework that is openly accessible to the teaching and research community, and can be readily reused and extended by other researchers and engineers. The research findings have been evaluated in several challenging CIS applications where the potential of having a significant economic and social impact is high. Examples of validation test beds implemented include an AR visual excavator-utility collision avoidance system that enables spotters to ”see” buried utilities hidden under the ground surface, thus helping prevent accidental utility strikes; an AR post-disaster reconnaissance framework that enables building inspectors to rapidly evaluate and quantify structural damage sustained by buildings in seismic events such as earthquakes or blasts; and a tabletop collaborative AR visualization framework that allows multiple users to observe and interact with visual simulations of engineering processes.PHDCivil EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/96145/1/dsuyang_1.pd

Tangible user interfaces : past, present and future directions

In the last two decades, Tangible User Interfaces (TUIs) have emerged as a new interface type that interlinks the digital and physical worlds. Drawing upon users' knowledge and skills of interaction with the real non-digital world, TUIs show a potential to enhance the way in which people interact with and leverage digital information. However, TUI research is still in its infancy and extensive research is required in or- der to fully understand the implications of tangible user interfaces, to develop technologies that further bridge the digital and the physical, and to guide TUI design with empirical knowledge. This paper examines the existing body of work on Tangible User In- terfaces. We start by sketching the history of tangible user interfaces, examining the intellectual origins of this field. We then present TUIs in a broader context, survey application domains, and review frame- works and taxonomies. We also discuss conceptual foundations of TUIs including perspectives from cognitive sciences, phycology, and philoso- phy. Methods and technologies for designing, building, and evaluating TUIs are also addressed. Finally, we discuss the strengths and limita- tions of TUIs and chart directions for future research

Investigation and development of a tangible technology framework for highly complex and abstract concepts

The ubiquitous integration of computer-supported learning tools within the educational domain has led educators to continuously seek effective technological platforms for teaching and learning. Overcoming the inherent limitations of traditional educational approaches, interactive and tangible computing platforms have consequently garnered increased interest in the pursuit of embedding active learning pedagogies within curricula. However, whilst Tangible User Interface (TUI) systems have been successfully developed to edutain children in various research contexts, TUI architectures have seen limited deployment towards more advanced educational pursuits. Thus, in contrast to current domain research, this study investigates the effectiveness and suitability of adopting TUI systems for enhancing the learning experience of abstract and complex computational science and technology-based concepts within higher educational institutions (HEI)s. Based on the proposal of a contextually apt TUI architecture, the research describes the design and development of eight distinct TUI frameworks embodying innovate interactive paradigms through tabletop peripherals, graphical design factors, and active tangible manipulatives. These computationally coupled design elements are evaluated through summative and formative experimental methodologies for their ability to aid in the effective teaching and learning of diverse threshold concepts experienced in computational science. In addition, through the design and adoption of a technology acceptance model for educational technology (TAM4Edu), the suitability of TUI frameworks in HEI education is empirically evaluated across a myriad of determinants for modelling students’ behavioural intention. In light of the statistically significant results obtained in both academic knowledge gain (μ = 25.8%) and student satisfaction (μ = 12.7%), the study outlines the affordances provided through TUI design for various constituents of active learning theories and modalities. Thus, based on an empirical and pedagogical analyses, a set of design guidelines is defined within this research to direct the effective development of TUI design elements for teaching and learning abstract threshold concepts in HEI adaptations

Visualization and interaction in a simulation system for flood emergencies

Dissertação apresentada na Faculdade de Ciências e Tecnologia da Universidade Nova de Lisboa para obtenção do grau de Mestre em Engenharia InformáticaThis thesis presents an interaction and visualization system for a river flood emergency simulation. It will also present a detailed study about forms of visual representation of critical elements in emergencies. All these elements are currently assembled in an application based on geographic information systems and agent simulation. Many of the goals in this thesis are interconnected with project Life-Saver. This project has the goal to develop an emergency response simulator, which needs a visualization and interaction system. The main goals of this thesis are, to create a visualization system for an emergency, to design an intuitive multimedia interface and to implement new forms of human-computer interaction. At the application level there is a representation of the simulation scenario with the multiple agent and their actions. Several studies were made to create an intuitive interface. New forms of multimedia interaction are studied and used such as interactive touch sensible boards and multi-touch panels. It is possible to load and retrieve geographic information on the scenario. The resulting architecture is used to visualize a simulation of an emergency flooding situation in a scenario where the Alqueva dam in Guadiana river fails

Supporting Collaborative Learning in Computer-Enhanced Environments

As computers have expanded into almost every aspect of our lives, the ever-present graphical user interface (GUI) has begun facing its limitations. Demanding its own share of attention, GUIs move some of the users\u27 focus away from the task, particularly when the task is 3D in nature or requires collaboration. Researchers are therefore exploring other means of human-computer interaction. Individually, some of these new techniques show promise, but it is the combination of multiple approaches into larger systems that will allow us to more fully replicate our natural behavior within a computing environment. As computers become more capable of understanding our varied natural behavior (speech, gesture, etc.), the less we need to adjust our behavior to conform to computers\u27 requirements. Such capabilities are particularly useful where children are involved, and make using computers in education all the more appealing. Herein are described two approaches and implementations of educational computer systems that work not by user manipulation of virtual objects, but rather, by user manipulation of physical objects within their environment. These systems demonstrate how new technologies can promote collaborative learning among students, thereby enhancing both the students\u27 knowledge and their ability to work together to achieve even greater learning. With these systems, the horizon of computer-facilitated collaborative learning has been expanded. Included among this expansion is identification of issues for general and special education students, and applications in a variety of domains, which have been suggested

tCAD: a 3D modeling application on a depth enhanced tabletop computer

Tabletop computers featuring multi-touch input and object tracking are a common platform for research on Tangible User Interfaces (also known as Tangible Interaction). However, such systems are confined to sensing activity on the tabletop surface, disregarding the rich and relatively unexplored interaction canvas above the tabletop. This dissertation contributes with tCAD, a 3D modeling tool combining fiducial marker tracking, finger tracking and depth sensing in a single system. This dissertation presents the technical details of how these features were integrated, attesting to its viability through the design, development and early evaluation of the tCAD application. A key aspect of this work is a description of the interaction techniques enabled by merging tracked objects with direct user input on and above a table surface.Universidade da Madeir

Review: Development and technical design of tangible user interfaces in wide-field areas of application

A tangible user interface or TUI connects physical objects and digital interfaces. It is more interactive and interesting for users than a classic graphic user interface. This article presents a descriptive overview of TUI's real-world applications sorted into ten main application areas-teaching of traditional subjects, medicine and psychology, programming, database development, music and arts, modeling of 3D objects, modeling in architecture, literature and storytelling, adjustable TUI solutions, and commercial TUI smart toys. The paper focuses on TUI's technical solutions and a description of technical constructions that influences the applicability of TUIs in the real world. Based on the review, the technical concept was divided into two main approaches: the sensory technical concept and technology based on a computer vision algorithm. The sensory technical concept is processed to use wireless technology, sensors, and feedback possibilities in TUI applications. The image processing approach is processed to a marker and markerless approach for object recognition, the use of cameras, and the use of computer vision platforms for TUI applications.Web of Science2113art. no. 425

Design e desenvolvimento de uma mesa tangível e objetos com tecnologia embarcada para uma experiência socioenativa

Orientador: Maria Cecília Calani BaranauskasDissertação (mestrado) - Universidade Estadual de Campinas, Instituto de ComputaçãoResumo: Os ambientes construídos a partir da tecnologia contemporânea (interativa, tangível e ubíqua) ainda desafiam os métodos de IHC tradicionais para projetar sistemas computacionais, exigindo novas maneiras de considerar o design de interação e sua avaliação. Ideias emergentes das ciências cognitivas enativas podem fornecer novas perspectivas para se criar novas maneiras de interagir com a tecnologia, considerando tanto o ambiente quanto a experiência humana como aspectos de embodiment. Tecnologias como tabletops tangíveis oferecem oportunidades para se explorar formas incorporadas de interação, colocando em primeiro plano o papel do corpo na interação e na aprendizagem. Além disso, o baixo custo, a quantidade e a capacidade de conexão à internet dos dispositivos oferecem oportunidades para tornar o sensoriamento físico acessível para incorporação no design de tabletops tangíveis e poder transformar ações físicas em efeitos físicos no ambiente. Nesta dissertação, investigamos a experiência (sócio)enativa de interação no contexto de sistemas computacionais ubíquos, projetando e desenvolvendo o TangiTime, uma instalação educacional desenhada como uma tabletop tangível e objetos com tecno- logia embarcada, para experimentar o conceito de "tempo profundo". Nesta dissertação apresentamos o processo de design e a implementação dos artefatos. Depois, apresentamos três diferentes contextos em que a instalação foi colocada em uso: a) um uso piloto para obter as primeiras impressões da experiência dos participantes com ela, b) uma exibição pública para obter um sentimento `real¿ das pessoas interagindo com a instalação e analisar os aspectos (sócio)enativos envolvidos, e c) um workshop em um museu exploratório científico para explorar o papel dos objetos com tecnologia embarcada para apoiar o engajamento e a aprendizagem dentro da exposição. Nossos resultados sugerem que a composição de tecnologia ubíqua potencializa a criação de uma experiência (sócio)enativa por meio do fomento de percepções multimodais e do engajamento dos participantes. Além disso, a interação com objetos com tecnologia embarcada cria novas maneiras de experimentar tabletops tangíveis em direção a uma experiência (sócio)enativaAbstract: The environments constructed from contemporary technology (interactive, tangible and ubiquitous) are still challenging the mainstream HCI methods for designing computational systems, demanding new ways of considering the interaction design and its evaluation. Emerging ideas of enactive cognitive sciences can provide new perspectives to create new ways of interacting with technology, considering both the environment and the human experience as aspects of embodiment. Technologies such as tangible tabletops offer opportunities to exploit embodied forms of interaction foregrounding the role of the body in interaction and learning. Furthermore, the low cost, quantity and internet capabilities of devices offer opportunities to make physical sensing accessible for incorporation into the design of tangible tabletops, transforming physical actions into physical effects in the environment. In this dissertation, we investigate the (socio)enactive experience of interaction in the context of the ubiquity of computational systems by designing and developing TangiTime, a tangible tabletop educational exhibit enhanced with embedded-technology objects for experiencing the `deep time¿ concept. In this dissertation work, we present the design process and the artifacts implementation. Moreover, we present three different contexts in which the exhibit was put into usage: a) a pilot use to get the first impressions of the user experience with it, b) a public exhibition to get a `real¿ feeling of people interacting with the installation and to analyze the (socio)enactive aspects involved, and c) a workshop within a science exploratory museum to explore the role of the embedded-technology objects to support engagement and learning in the exhibit. Our results suggest that the composition of ubiquitous technologies potentializes the creation of a (socio)enactive experience by fostering multimodal perceptions and the engagement of the participants. Moreover, interaction with embedded-technology objects creates new ways of experiencing tangible tabletops toward a (socio)enactive experienceMestradoCiência da ComputaçãoMestra em Ciência da ComputaçãoCAPE

A Mixed Reality Approach to 3D Interactive Prototyping for Participatory Design of Ambient Intelligence

Ambient Intelligence (AmI in short) is a multi-disciplinary approach aimed at enriching physical environments with a network of distributed devices in order to support humans in achieving their everyday goals. However, in current research and development, AmI is still largely considered within the engineering domain bearing undeveloped relationship with architecture. The fact that architecture design substantially aims to address the requirements of supporting people in carrying out their everyday life activities, tasks and practices with spatial strategies. These are common to the AmI’s objectives and purposes, and we aim at considering the possibilities or even necessities of investigating the potential design approach accessible to an architectural context. For end users, AmI is a new type of service. Designing and evaluating the AmI experience before resources are spent on designing the processes and technology needed to eventually run the service can save large amounts of time and money. Therefore, it is essential to create an environment in which designers can involve real people in trying out the service design proposals as early as possible in the design process. Existing cases related to stakeholder engaged design of AmI have primarily focused on engineering implementation and generally only present final outcome to stakeholders for user evaluation. Researchers have been able to build AmI prototypes for design communication. However, most of these prototypes are typically built without the involvement of stakeholders and architects in their conceptual design stage. Using concepts solely designed by engineers may not be user centric and even contain safety risks. The key research question of this thesis is: “How can Ambient Intelligence be designed through a participatory process that involves stakeholders and prospective users?" The thesis consists of the following five components: 1) Identification of a novel participatory design process for modelling AmI scenarios; 2) Identification of the requirements to support prototyping of AmI design, resulting in a conceptual framework that both "lowers the floor" (i.e. making it easier for designers to build the AmI prototypes) and "raises the ceiling" (i.e. increasing the ability of stakeholders and end users to participate in the design process deeply); i 3) Prototyping an experimental Mixed Reality Modelling (MRM in short) platform to facilitate the participatory design of AmI that supports the requirements, design process, and scenarios prototyping; 4) Case study of applying MRM platform to participatory design of a Smart Laser Cutting Workshop(LCW in short) which used to evaluate the proposed MRM based AmI design approach. The result of the research shows that the MRM based participatory design approach is able to support the design of AmI effectively