Multimodal Content Delivery for Geo-services

This thesis describes a body of work carried out over several research projects in the area of multimodal interaction for location-based services. Research in this area has progressed from using simulated mobile environments to demonstrate the visual modality, to the ubiquitous delivery of rich media using multimodal interfaces (geo- services). To effectively deliver these services, research focused on innovative solutions to real-world problems in a number of disciplines including geo-location, mobile spatial interaction, location-based services, rich media interfaces and auditory user interfaces. My original contributions to knowledge are made in the areas of multimodal interaction underpinned by advances in geo-location technology and supported by the proliferation of mobile device technology into modern life. Accurate positioning is a known problem for location-based services, contributions in the area of mobile positioning demonstrate a hybrid positioning technology for mobile devices that uses terrestrial beacons to trilaterate position. Information overload is an active concern for location-based applications that struggle to manage large amounts of data, contributions in the area of egocentric visibility that filter data based on field-of-view demonstrate novel forms of multimodal input. One of the more pertinent characteristics of these applications is the delivery or output modality employed (auditory, visual or tactile). Further contributions in the area of multimodal content delivery are made, where multiple modalities are used to deliver information using graphical user interfaces, tactile interfaces and more notably auditory user interfaces. It is demonstrated how a combination of these interfaces can be used to synergistically deliver context sensitive rich media to users - in a responsive way - based on usage scenarios that consider the affordance of the device, the geographical position and bearing of the device and also the location of the device

A systematic review of augmented reality applications in maintenance

Augmented Reality (AR) technologies for supporting maintenance operations have been an academic research topic for around 50 years now. In the last decade, major progresses have been made and the AR technology is getting closer to being implemented in industry. In this paper, the advantages and disadvantages of AR have been explored and quantified in terms of Key Performance Indicators (KPI) for industrial maintenance. Unfortunately, some technical issues still prevent AR from being suitable for industrial applications. This paper aims to show, through the results of a systematic literature review, the current state of the art of AR in maintenance and the most relevant technical limitations. The analysis included filtering from a large number of publications to 30 primary studies published between 1997 and 2017. The results indicate a high fragmentation among hardware, software and AR solutions which lead to a high complexity for selecting and developing AR systems. The results of the study show the areas where AR technology still lacks maturity. Future research directions are also proposed encompassing hardware, tracking and user-AR interaction in industrial maintenance is proposed

An aesthetics of touch: investigating the language of design relating to form

How well can designers communicate qualities of touch? This paper presents evidence that they have some capability to do so, much of which appears to have been learned, but at present make limited use of such language. Interviews with graduate designer-makers suggest that they are aware of and value the importance of touch and materiality in their work, but lack a vocabulary to fully relate to their detailed explanations of other aspects such as their intent or selection of materials. We believe that more attention should be paid to the verbal dialogue that happens in the design process, particularly as other researchers show that even making-based learning also has a strong verbal element to it. However, verbal language alone does not appear to be adequate for a comprehensive language of touch. Graduate designers-makers’ descriptive practices combined non-verbal manipulation within verbal accounts. We thus argue that haptic vocabularies do not simply describe material qualities, but rather are situated competences that physically demonstrate the presence of haptic qualities. Such competencies are more important than groups of verbal vocabularies in isolation. Design support for developing and extending haptic competences must take this wide range of considerations into account to comprehensively improve designers’ capabilities

Extended Skin: Designing Interactive Content for Ubiquitous Computing Materials

Current research is inspired by the impact of digital media on disciplinary division. Sim- ultaneously, recognizes the difficulty of engineering (applied science) to consider the humani- ties as fundamental contributors in the process of making. Steaming from a design perspective, the intersection between art (design) and science, questions if these relations can open per- spectives on the matter of designing within a U.C. context, and fundamentally, introduces the question on how this can be done Furthermore, the motivation for this research arises from considering that innovation in technology is happening in the fields typically identified as engineering. And, despite this, the in-corporation of these inventions in life, considering some discussed exceptions, has not typ- ically been present in the concerns of design action and methods. Therefore, the challenge of current research is to contribute to the realm of ubiquitous computing, routed by design, to some degree aiming to contribute to the field. A deeper analysis into the subject of U.C., there is the realization that there is minority presence of the humanities in the discussion of U.C. (Dourish and Bell, 2011). Technological disruption offers continuous inspiration for design innovation within U.C. Furthermore, the inquiry labeled as “material turn” contextualizes a dialogue between nano- technology and traditional materials. Nanotechnology is applied to project development, while considering a human centred design approach. This focus is present throughout this disserta- tion. The research proposal describes SuberSkin, as a responsive surface that works as a screen. The exploration of aesthetical effects is focused on visual properties – using high con- trast between natural cork colors, dark and light brown. The proposal is highly experimental, and ultimately, aims to explore potential routes on cork research, linked to that of U.C. Thus, recreating and transforming this material into an intelligent surface. In sum, this thesis discusses displacement of disciplines suggested as having a positive impact in interdisciplinary thought and for future design. Therefore a methodology, "research through techne" is presented that illustrates this intention.A presente pesquisa é inspirada pelo impacto exercido pelos media digitais na divisão disciplinar. Simultaneamente, reconhece a dificuldade da engenharia (ciência aplicada) em considerar as humanidades como contribuintes fundamentais no processo de fazer. Partindo de uma perspectiva de design e da interseção entre arte (design) e ciência, questiona-se se essas relações poderão abrir perspectivas na criação no âmbito da Computação Ubíqua. Fun- damentalmente, introduz a questão de como poderá ser feito. A motivação para esta pesquisa decorre de considerar que a inovação tecnológica acontece nas áreas normalmente identificadas como engenharia. E, apesar disso, a incor- poração dessas invenções na vida, considerando as exceções discutidas, normalmente não está presente nas preocupações, ação e métodos de design. Portanto, o desafio da pesquisa é con- tribuir para o domínio da Computação Ubíqua, orientada pelo design. Uma análise mais pro- funda sobre o tema da Computação Ubiqua, constata que há na sua discussão uma presença minoritária das humanidades (Dourish e Bell, 2011). A disrupção tecnológica oferece inspiração contínua para inovação de design, e o mesmo se aplica no âmbito da Computação Ubíqua. Além disso, a pesquisa intitulada como “material turn” contextualiza um diálogo entre a nanotecnologia e os materiais tradicionais. A nanotecnologia é aplicada ao desenvolvimento de projetos, considerando uma abordagem de design centrada no ser humano. Este foco está presente ao longo desta dissertação. O projecto de pesquisa descreve SuberSkin, uma superfície responsiva. A exploração centra-se nos efeitos estéticos da cortiça, recorrendo a um contraste entre as suas cores natu- rais: castanho escuro e claro. A proposta é experimental e, em última análise, visa explorar potenciais linhas de investigação ligando a cortiça à Computação Ubíqua. E assim, recriar e transformar este material numa superfície inteligente. Em suma, esta tese discute o deslocamento disciplinar como tendo um impacto posi- tivo no pensamento interdisciplinar e no futuro da prática do design. Consequentemente, apresenta uma metodologia, "investigação através da techne" que a exemplifica

Augmenting the field experience: a student-led comparison of techniques and technologies

In this study we report on our experiences of creating and running a student fieldtrip exercise which allowed students to compare a range of approaches to the design of technologies for augmenting landscape scenes. The main study site is around Keswick in the English Lake District, Cumbria, UK, an attractive upland environment popular with tourists and walkers. The aim of the exercise for the students was to assess the effectiveness of various forms of geographic information in augmenting real landscape scenes, as mediated through a range of techniques and technologies. These techniques were: computer-generated acetate overlays showing annotated wireframe views from certain key points; a custom-designed application running on a PDA; a mediascape running on the mScape software on a GPS-enabled mobile phone; Google Earth on a tablet PC; and a head-mounted in-field Virtual Reality system. Each group of students had all five techniques available to them, and were tasked with comparing them in the context of creating a visitor guide to the area centred on the field centre. Here we summarise their findings and reflect upon some of the broader research questions emerging from the project

Toward an affect-sensitive multimodal human-computer interaction

The ability to recognize affective states of a person... This paper argues that next-generation human-computer interaction (HCI) designs need to include the essence of emotional intelligence -- the ability to recognize a user's affective states -- in order to become more human-like, more effective, and more efficient. Affective arousal modulates all nonverbal communicative cues (facial expressions, body movements, and vocal and physiological reactions). In a face-to-face interaction, humans detect and interpret those interactive signals of their communicator with little or no effort. Yet design and development of an automated system that accomplishes these tasks is rather difficult. This paper surveys the past work in solving these problems by a computer and provides a set of recommendations for developing the first part of an intelligent multimodal HCI -- an automatic personalized analyzer of a user's nonverbal affective feedback

Introduction to location-based mobile learning

[About the book] The report follows on from a 2-day workshop funded by the STELLAR Network of Excellence as part of their 2009 Alpine Rendez-Vous workshop series and is edited by Elizabeth Brown with a foreword from Mike Sharples. Contributors have provided examples of innovative and exciting research projects and practical applications for mobile learning in a location-sensitive setting, including the sharing of good practice and the key findings that have resulted from this work. There is also a debate about whether location-based and contextual learning results in shallower learning strategies and a section detailing the future challenges for location-based learning