Ubiquitous computing and natural interfaces for environmental information

Dissertação apresentada na Faculdade de Ciências e Tecnologia da Universidade Nova de Lisboa para obtenção do Grau de Mestre em Engenharia do Ambiente, perfil Gestão e Sistemas AmbientaisThe next computing revolution‘s objective is to embed every street, building, room and object with computational power. Ubiquitous computing (ubicomp) will allow every object to receive and transmit information, sense its surroundings and act accordingly, be located from anywhere in the world, connect every person. Everyone will have the possibility to access information, despite their age, computer knowledge, literacy or physical impairment. It will impact the world in a profound way, empowering mankind, improving the environment, but will also create new challenges that our society, economy, health and global environment will have to overcome. Negative impacts have to be identified and dealt with in advance. Despite these concerns, environmental studies have been mostly absent from discussions on the new paradigm. This thesis seeks to examine ubiquitous computing, its technological emergence, raise awareness towards future impacts and explore the design of new interfaces and rich interaction modes. Environmental information is approached as an area which may greatly benefit from ubicomp as a way to gather, treat and disseminate it, simultaneously complying with the Aarhus convention. In an educational context, new media are poised to revolutionize the way we perceive, learn and interact with environmental information. cUbiq is presented as a natural interface to access that information

Designing multi-sensory displays for abstract data

The rapid increase in available information has lead to many attempts to automatically locate patterns in large, abstract, multi-attributed information spaces. These techniques are often called data mining and have met with varying degrees of success. An alternative approach to automatic pattern detection is to keep the user in the exploration loop by developing displays for perceptual data mining. This approach allows a domain expert to search the data for useful relationships and can be effective when automated rules are hard to define. However, designing models of the abstract data and defining appropriate displays are critical tasks in building a useful system. Designing displays of abstract data is especially difficult when multi-sensory interaction is considered. New technology, such as Virtual Environments, enables such multi-sensory interaction. For example, interfaces can be designed that immerse the user in a 3D space and provide visual, auditory and haptic (tactile) feedback. It has been a goal of Virtual Environments to use multi-sensory interaction in an attempt to increase the human-to-computer bandwidth. This approach may assist the user to understand large information spaces and find patterns in them. However, while the motivation is simple enough, actually designing appropriate mappings between the abstract information and the human sensory channels is quite difficult. Designing intuitive multi-sensory displays of abstract data is complex and needs to carefully consider human perceptual capabilities, yet we interact with the real world everyday in a multi-sensory way. Metaphors can describe mappings between the natural world and an abstract information space. This thesis develops a division of the multi-sensory design space called the MS-Taxonomy. The MS-Taxonomy provides a concept map of the design space based on temporal, spatial and direct metaphors. The detailed concepts within the taxonomy allow for discussion of low level design issues. Furthermore the concepts abstract to higher levels, allowing general design issues to be compared and discussed across the different senses. The MS-Taxonomy provides a categorisation of multi-sensory design options. However, to design effective multi-sensory displays requires more than a thorough understanding of design options. It is also useful to have guidelines to follow, and a process to describe the design steps. This thesis uses the structure of the MS-Taxonomy to develop the MS-Guidelines and the MS-Process. The MS-Guidelines capture design recommendations and the problems associated with different design choices. The MS-Process integrates the MS-Guidelines into a methodology for developing and evaluating multi-sensory displays. A detailed case study is used to validate the MS-Taxonomy, the MS-Guidelines and the MS-Process. The case study explores the design of multi-sensory displays within a domain where users wish to explore abstract data for patterns. This area is called Technical Analysis and involves the interpretation of patterns in stock market data. Following the MS-Process and using the MS-Guidelines some new multi-sensory displays are designed for pattern detection in stock market data. The outcome from the case study includes some novel haptic-visual and auditory-visual designs that are prototyped and evaluated

Processo e desconstrução: estruturas recursivas entre cozinha e design

From the triangulation kitchen, design and process this study searches evidence for bridging process between the fields of kitchen and design following Buchanan’s theory of rethinking placements over categories by way of signs, things, actions and thoughts. Kitchen and design are thus understood as liberal arts disciplines seeking to privilege a placement-based approach to projectual practice. Deconstruction was instrumental for exploring and testing the presence of recursive structures —elements carrying the essential information that generates similar patterns visible on different fields— intersecting the two fields, aiming to prove if and whence kitchen may contribute to expand the knowledge of design. Pallasmaa’s conception of an architecture of the senses, for whom the role of the body is understood as the locus of perception, thought and consciousness, helped explore and convoke the space of kitchen visited by artists and designers throughout recent history, as a means to establish relations between theories, processes, and projectual methodologies in kitchen and design. The reading of the space finds its translation through diverse processes applied by these creators leading to an understanding of a kitchen milieu: process as context. The empirical work with the research sample practices allowed for a discourse within, between and beyond individual samples to reveal the dialogical abilities of the applied processes. From the interpretation of the empirical work it is suggested that kitchen multiplies design (k x d). It implies that the context of kitchen multiplies the space of the discipline of design, becoming, in Buchanan’s term, a “quasi-subject matter of design thinking”. If so, kitchen as other placements may offer, or are open to receive and edify, an expanded view of the discipline of design. Considering findings of the three main typologies (education, research, process) it suggests transversality and integrates fundamental knowledge dimensions as the capability for negotiation between different actors/disciplines. This is made visible through the practices that comprise the research sample, namely curiosity, context, scale, desire, care. This study recommends further research into the transformative potential and imaginary of the kitchen/canteen in liberal arts education.A partir da triangulação cozinha, design e processo, este estudo investiga a evidência para o processo de ligação entre as áreas de cozinha e de design, de acordo com a teoria de Buchanan de repensar mais os posicionamentos que as categorias mediante sinais, coisas, ações e pensamentos. Assim, cozinha e design são entendidos como disciplinas das artes liberais, procurando privilegiar uma abordagem baseada no posicionamento em relação à prática projectual. A desconstrução foi fundamental para explorar e testar a presença de estruturas recursivas —elementos que comportam a informação essencial que gera padrões semelhantes em áreas diferentes— entrecruzando as duas áreas, visando provar se e a partir de onde a cozinha pode contribuir para expandir o conhecimento sobre o design. A concepção de Pallasmaa de uma arquitetura dos sentidos, para quem o papel do corpo é entendido como o locus da percepção, do pensamento e da consciência, ajudou a explorar e a convocar o espaço da cozinha, visitado por artistas e designers ao longo da história recente, como um meio de estabelecer relações entre teorias, processos e metodologias projectuais na cozinha e no design. A leitura do espaço encontra a sua tradução através de diversos processos aplicados por estes criadores, levando a um entendimento de um milieu de cozinha: o processo como contexto. O trabalho empírico com as práticas da amostra de investigação permitiu um discurso dentro, entre e para além das amostras individuais para revelar as capacidades dialógicas dos processos aplicados. A partir da interpretação do trabalho empírico, sugere-se que a cozinha multiplica o design (k x d). Isto implica que o contexto da cozinha multiplica o espaço da disciplina de design, tornando-se, na terminologia de Buchanan, uma “matéria de quase-sujeito de design thinking”. Se assim for, a cozinha, tal como outros posicionamentos, pode oferecer ou estar aberta a receber e a edificar uma visão alargada da disciplina de design. Considerando os resultados das três principais tipologias (educação, investigação, processo), sugere transversalidade e integra dimensões de conhecimento fundamentais como a capacidade de negociação entre diferentes actores/disciplinas. Isto torna-se visível através das práticas que incluem amostra de investigação, nomeadamente curiosidade, contexto, escala, desejo, cuidado. Este estudo recomenda desenvolver investigação sobre o potencial e o imaginário transformadores da cozinha/cantina na educação das artes liberais.Programa Doutoral em Desig

Using Haptic Virtual Reality to Increase Learning Gains and Construct Knowledge of Unobservable Phenomena

This project is designed to be a compilation of ten haptic virtual reality labs using the software zSpace. The labs will follow the NYS Living Environment Standards as well as the Next Generation Science Standards for living environment as well as physical/general science topics for middle school students. The project will be a list of available laboratories along with their appropriate fit into the curriculum and a description of how they fit New York State curriculum standards for the appropriate discipline. The goal of these laboratory assignments is to increase learning gains in students by allowing them to experience scientific phenomena that can often be unrelatable and unobservable

Artifact-Based Rendering: Harnessing Natural and Traditional Visual Media for More Expressive and Engaging 3D Visualizations

We introduce Artifact-Based Rendering (ABR), a framework of tools, algorithms, and processes that makes it possible to produce real, data-driven 3D scientific visualizations with a visual language derived entirely from colors, lines, textures, and forms created using traditional physical media or found in nature. A theory and process for ABR is presented to address three current needs: (i) designing better visualizations by making it possible for non-programmers to rapidly design and critique many alternative data-to-visual mappings; (ii) expanding the visual vocabulary used in scientific visualizations to depict increasingly complex multivariate data; (iii) bringing a more engaging, natural, and human-relatable handcrafted aesthetic to data visualization. New tools and algorithms to support ABR include front-end applets for constructing artifact-based colormaps, optimizing 3D scanned meshes for use in data visualization, and synthesizing textures from artifacts. These are complemented by an interactive rendering engine with custom algorithms and interfaces that demonstrate multiple new visual styles for depicting point, line, surface, and volume data. A within-the-research-team design study provides early evidence of the shift in visualization design processes that ABR is believed to enable when compared to traditional scientific visualization systems. Qualitative user feedback on applications to climate science and brain imaging support the utility of ABR for scientific discovery and public communication.Comment: Published in IEEE VIS 2019, 9 pages of content with 2 pages of references, 12 figure

Literacy for digital futures : Mind, body, text

The unprecedented rate of global, technological, and societal change calls for a radical, new understanding of literacy. This book offers a nuanced framework for making sense of literacy by addressing knowledge as contextualised, embodied, multimodal, and digitally mediated. In today’s world of technological breakthroughs, social shifts, and rapid changes to the educational landscape, literacy can no longer be understood through established curriculum and static text structures. To prepare teachers, scholars, and researchers for the digital future, the book is organised around three themes – Mind and Materiality; Body and Senses; and Texts and Digital Semiotics – to shape readers’ understanding of literacy. Opening up new interdisciplinary themes, Mills, Unsworth, and Scholes confront emerging issues for next-generation digital literacy practices. The volume helps new and established researchers rethink dynamic changes in the materiality of texts and their implications for the mind and body, and features recommendations for educational and professional practice

Enhanced Virtuality: Increasing the Usability and Productivity of Virtual Environments

Mit stetig steigender Bildschirmauflösung, genauerem Tracking und fallenden Preisen stehen Virtual Reality (VR) Systeme kurz davor sich erfolgreich am Markt zu etablieren. Verschiedene Werkzeuge helfen Entwicklern bei der Erstellung komplexer Interaktionen mit mehreren Benutzern innerhalb adaptiver virtueller Umgebungen. Allerdings entstehen mit der Verbreitung der VR-Systeme auch zusätzliche Herausforderungen: Diverse Eingabegeräte mit ungewohnten Formen und Tastenlayouts verhindern eine intuitive Interaktion. Darüber hinaus zwingt der eingeschränkte Funktionsumfang bestehender Software die Nutzer dazu, auf herkömmliche PC- oder Touch-basierte Systeme zurückzugreifen. Außerdem birgt die Zusammenarbeit mit anderen Anwendern am gleichen Standort Herausforderungen hinsichtlich der Kalibrierung unterschiedlicher Trackingsysteme und der Kollisionsvermeidung. Beim entfernten Zusammenarbeiten wird die Interaktion durch Latenzzeiten und Verbindungsverluste zusätzlich beeinflusst. Schließlich haben die Benutzer unterschiedliche Anforderungen an die Visualisierung von Inhalten, z.B. Größe, Ausrichtung, Farbe oder Kontrast, innerhalb der virtuellen Welten. Eine strikte Nachbildung von realen Umgebungen in VR verschenkt Potential und wird es nicht ermöglichen, die individuellen Bedürfnisse der Benutzer zu berücksichtigen. Um diese Probleme anzugehen, werden in der vorliegenden Arbeit Lösungen in den Bereichen Eingabe, Zusammenarbeit und Erweiterung von virtuellen Welten und Benutzern vorgestellt, die darauf abzielen, die Benutzerfreundlichkeit und Produktivität von VR zu erhöhen. Zunächst werden PC-basierte Hardware und Software in die virtuelle Welt übertragen, um die Vertrautheit und den Funktionsumfang bestehender Anwendungen in VR zu erhalten. Virtuelle Stellvertreter von physischen Geräten, z.B. Tastatur und Tablet, und ein VR-Modus für Anwendungen ermöglichen es dem Benutzer reale Fähigkeiten in die virtuelle Welt zu übertragen. Des Weiteren wird ein Algorithmus vorgestellt, der die Kalibrierung mehrerer ko-lokaler VR-Geräte mit hoher Genauigkeit und geringen Hardwareanforderungen und geringem Aufwand ermöglicht. Da VR-Headsets die reale Umgebung der Benutzer ausblenden, wird die Relevanz einer Ganzkörper-Avatar-Visualisierung für die Kollisionsvermeidung und das entfernte Zusammenarbeiten nachgewiesen. Darüber hinaus werden personalisierte räumliche oder zeitliche Modifikationen vorgestellt, die es erlauben, die Benutzerfreundlichkeit, Arbeitsleistung und soziale Präsenz von Benutzern zu erhöhen. Diskrepanzen zwischen den virtuellen Welten, die durch persönliche Anpassungen entstehen, werden durch Methoden der Avatar-Umlenkung (engl. redirection) kompensiert. Abschließend werden einige der Methoden und Erkenntnisse in eine beispielhafte Anwendung integriert, um deren praktische Anwendbarkeit zu verdeutlichen. Die vorliegende Arbeit zeigt, dass virtuelle Umgebungen auf realen Fähigkeiten und Erfahrungen aufbauen können, um eine vertraute und einfache Interaktion und Zusammenarbeit von Benutzern zu gewährleisten. Darüber hinaus ermöglichen individuelle Erweiterungen des virtuellen Inhalts und der Avatare Einschränkungen der realen Welt zu überwinden und das Erlebnis von VR-Umgebungen zu steigern