User Interface Abstraction for enabling TV set based Inclusive Access to the Information Society

199 p.The television (TV) set is present in most homes worldwide, and is the most used Information and Communication Technology (ICT). Despite its large implantation in the market, the interactive services consumption on TV set is limited. This thesis focuses on overcoming the following limiting factors: (i) limited Human Computer Interaction and (ii) lack of considering user’s real life context in the digital television (dTV) service integration strategy. Making interactive services accessible to TV set’s large user base, and especially to the most vulnerable ones, is understood as the path to integrate the mankind with the information society. This thesis explores the use of user interface abstraction technologies to reach the introduced goals. The main contributions of this thesis are: (i) an approach to enable the universally accessible remote control of the TV set, (ii) an approach for the provision of universally accessible interactive services through TV sets, and (iii) an approach for the provision of universally accessible services in the TV user’s real life context. We have implemented the contributing approaches for different use cases, and we have evaluated them with real users, achieving good results

FPGA-based architectures for acoustic beamforming with microphone arrays : trends, challenges and research opportunities

Over the past decades, many systems composed of arrays of microphones have been developed to satisfy the quality demanded by acoustic applications. Such microphone arrays are sound acquisition systems composed of multiple microphones used to sample the sound field with spatial diversity. The relatively recent adoption of Field-Programmable Gate Arrays (FPGAs) to manage the audio data samples and to perform the signal processing operations such as filtering or beamforming has lead to customizable architectures able to satisfy the most demanding computational, power or performance acoustic applications. The presented work provides an overview of the current FPGA-based architectures and how FPGAs are exploited for different acoustic applications. Current trends on the use of this technology, pending challenges and open research opportunities on the use of FPGAs for acoustic applications using microphone arrays are presented and discussed

Web Engineering for Workflow-based Applications: Models, Systems and Methodologies

This dissertation presents novel solutions for the construction of Workflow-based Web applications: The Web Engineering DSL Framework, a stakeholder-oriented Web Engineering methodology based on Domain-Specific Languages; the Workflow DSL for the efficient engineering of Web-based Workflows with strong stakeholder involvement; the Dialog DSL for the usability-oriented development of advanced Web-based dialogs; the Web Engineering Reuse Sphere enabling holistic, stakeholder-oriented reuse

Theoretical Approaches to Musical Creativity: The Ubimus Perspective

We tackle the theoretical implications of an emergent field of study, ubiquitous music (ubimus). Striving to place musical research within the wider field of creativity studies, this paper covers theoretical background and critical discussions of three music creativity models - a model proposed by Mannis (2014), the In-group, Out-group model (MDF - Ferraz and Keller 2014) and the ecologically grounded perspective (Keller and Lazzarini 2017). Data and on-site observations of creative activities provide the raw materials to test musical theories. Recent ubiquitous music research features multiple experimental initiatives, involving children (Pereira et al. in press), adolescents (Lima et al. 2012) or the elderly, and targeting trained musicians (Ferreira et al. 2015), novices (Miletto et al. 2011) and cognitively challenged adults. We chose three ubimus studies to exemplify empirical research that brings to light issues that have not been considered by some of the current theoretical approaches to musical creativity. Abordamos as implicações teóricas de um campo de estudos emergente – a música ubíqua (ubimus). Buscando situar a pesquisa musical dentro dos estudos sobre criatividade, este artigo abrange o substrato teórico e as discussões críticas de três modelos de criatividade musical – um modelo proposto por Mannis (2014), o Modelo Dentro-Fora (MDF - Ferraz e Keller 2014) e a perspectiva ecológica (Keller e Lazzarini 2017). Os dados e as observações in loco de atividades criativas forneceram materiais brutos para testarmos teorias musicais. Pesquisas recentes em música ubíqua concentram-se em múltiplas iniciativas experimentais, envolvendo crianças (Pereira et al. no prelo), adolescentes (Lima et al. 2012) ou idosos, e focando em músicos profissionais (Ferreira et al. 2015), iniciantes (Miletto et al. 2011) e adultos com deficiências cognitivas. Escolhemos três estudos em música ubíqua para exemplificar as pesquisas empíricas que trazem à luz problemas que ainda não foram tratados desde as perspectivas teóricas da criatividade musical

Self-adaptation of multimodal systems

Tese de mestrado em Engenharia Informática (Sistemas de Informação), apresentada à Universidade de Lisboa, através da Faculdade de Ciências, 2011Este documento centra-se nos Sistemas Multimodais Adaptativos e suas aplicações no melhoramento da acessibilidade das interfaces. Pessoas idosas ou com algum tipo de deficiência, seja associado ou não à idade, são um grupo de alto risco sujeito à exclusão social. Muitas vezes o acesso a oportunidades ou serviços oferecidos pela sociedade é limitado ou mesmo inacessível para indivíduos com estas características. No caso específico da comunicação pessoa-máquina, um exemplo de falta de acessibilidade resulta das modalidades utilizadas para interagir ou apresentar informação. Se for usada apenas informação visual, alguém com dificuldades visuais não conseguirá interagir ou perceber a informação apresentada, logo é excluída. Como solução para estes problemas, neste documento apresenta-se GUIDE, um projecto europeu que pretende desenvolver uma framework que permite integrar eficientemente características de acessibilidade nas aplicações. Este projecto foca-se nas plataformas e serviços emergentes para TV e pretende transformar as Set-Top Box’s em sistemas multimodais adaptativos. São apresentados os objectivos do GUIDE, o papel da Faculdade de Ciências neste projecto e as contribuições desta tese. No segundo capítulo deste documento é apresentada a noção de Sistemas Multimodais Adaptativos, as vantagens associadas a este tipo de sistemas, arquitectura e descrição dos vários componentes. Um sistema multimodal ´e um sistema que disponibiliza várias modalidades para interagir (voz, gestos, etc), sendo possível a utilização de apenas uma ou várias modalidades em simultâneo. Estes sistemas não só oferecem ao utilizador uma comunicação mais natural como permitem alternativas tanto na maneira de interagir como de apresentar a informação a pessoas que, devido `as suas características, de outra maneira não conseguiriam. A outra característica destes sistemas é a adaptação, que pode ser descrita como um método para aumentar a usabilidade de uma aplicação em termos de eficiência, eficácia e facilidade de uso. Este método traz benefícios para a interação¸ ao do utilizador em diversas situações e pode ser aplicado em diferentes níveis de automação apresentados neste documento, sendo o mais interessante a auto-adaptação. Geralmente, a arquitectura de um sistema deste tipo ´e composta por reconhecedores para as modalidades de input, sintetizadores para as modalidades de output e entre eles, um conjunto de componentes responsável pela integração multimodal do sistema. Esses componentes são: um módulo de fusão (para os inputs) e outro de cisão (para outputs), um gestor de diálogo (Dialogue Manager) e um gestor de contexto. A descrição e função de cada componente são detalhadas também no capítulo 2. Esta tese dá especial destaque ao Dialogue Manager, cuja responsabilidade é coordenar a actividade dos vários componentes do sistema, manter a representação do estado actual do diálogo, actualizar o contexto do diálogo com base na informação interpretada na comunicação e decidir que conteúdo deve ser apresentado e quando, deixando a forma como é apresentado para o módulo de cisão. Das várias abordagens existentes para a implementação do Dialogue Manager, a que vem de encontro aos requisitos do projecto e está descrita neste documento chama-se “Frame-based approach”. Esta implementação aplica a analogia do preenchimento de um formulário, ou seja, interpreta o diálogo de um certo estado da aplicação como a necessidade de se verem realizadas certas condições para efectuar uma acção. Cada estado da aplicação pode conter vários formulários e o preenchimento de cada um deles resulta numa accão, que normalmente leva a estados diferentes da aplicação. Contudo,também é possível haver accões que são independentes da aplicação, como por exemplo pedir ao GUIDE para aumentar o volume. Estes formulários são instanciados automaticamente pelo Dialogue Manager. Pelo contrário, os formulários relativos à aplicação têm de ser inferidos a partir de uma representação da interface da aplicação Para realizar um projecto deste tipo é preciso conhecer melhor os futuros utilizadores. Por isso, foram realizados dois estudos feitos com utilizadores-alvo que pretendiam entender como os idosos reagiriam a novas maneiras de interagir com a sua televisão bem como as suas preferências em relação à apresentação da informação e dos elementos interactivos. As descobertas feitas são relatadas nesta tese e serviram não só para ajudar no desenvolvimento do Dialogue Manager mas também todos os outros componentes envolvidos. Outro dos principais objectivos destes estudos foi perceber que diferentes agrupamentos se pode criar para caracterizar e agrupar futuros utilizadores. Para esse fim foi implementado um protótipo que integra diferentes dispositivos para interação (ex: WiiMote, Kinect) e que permite a criação de interfaces multimodais recorrendo à descrição dos elementos que desejamos ver no ecrã num ficheiro XML. Os sistemas adaptativos dependem de informação guardada em modelos como base para aplicar a adaptação. Existem diferentes modelos tendo em conta os diferentes tipos de informação que se quer manter: modelo de utilizador, modelo da tarefa, modelo de domínio, modelo de diálogo, modelo de ambiente, etc. Desses modelos destaca-se o modelo de utilizador pois é o mais importante no campo dos sistemas adaptativos. O modelo de utilizador é a representação do conhecimento e preferências que o sistema “acredita” ser o que o utilizador possui. Toda esta informação é importante para melhorar a interação e portanto deve ser mantida e actualizada. Neste documento são também descritas as técnicas para a aquisição¸ ao desse conhecimento, existindo dois processos diferentes para o fazer: explicitamente e implicitamente. Observando o utilizador e o seu compor tamento é possível obter as informações implicitamente. Perguntando directamente ou permitindo ao utilizador alterar o modelo de utilizador a aquisição de informação está a ser feita explicitamente. Neste documento é apresentado um protótipo que pretende servir como um tutorial de modo a ensinar ao utilizador como interagir com o sistema mas também obter informações sobre ele de maneira a poder atribuí-lo a um dos agrupamentos encontrados nos estudos feitos anteriormente. Este protótipo obtém essas informações pedindo ao utilizador para efectuar certas tarefas e respondendo a algumas perguntas. Como foi referido anteriormente, o Dialogue Manager bem como outros componentes lidam com as aplicações através de uma representação da interface de utilizador. Neste documento é feito uma comparação entre algumas linguagens que têm este objectivo e é escolhido então o formato que vai ser usado neste projecto. Para extrair automaticamente essa representação, no capítulo 6 é apresentado uma ferramenta que o faz a partir de aplicações Web, ou seja implementadas em HTML e Javascript. Esta ferramenta analisa a estrutura dos elementos apresentados através da análise do código HTML bem como de algumas propriedades WAI-ARIA, desenhadas com o propósito da acessibilidade. De seguida, percorre e analisa não só o código HTML com também o CSS, de modo a descrever as propriedades visuais dos elementos que vão ser apresentados no ecrã. Finalmente, todo o trabalho resultante desta tese continuará a ser melhorado e avaliado até ao final da duração deste projecto que contará ainda com 1 ano e 3 meses. Durante esse período será realizada a integração dos vários componentes e será feita a migração das versões em PC para Set-top-box. No fim ´e esperado que todos os objectivos a que este projecto se propôs sejam cumpridos e que pelo menos facilite a vida daqueles que são afectados pela falta de acessibilidade nas tecnologias existentes.This thesis focuses on Adaptive Multimodal Systems and their applications on improving user interface accessibility. Disabled and/or elderly people are a group at high risk of social exclusion. The access to the opportunities offered by society is obviously limited if these cannot be reached by persons with impairments or restricted mobility. A more subtle way of exclusion results from the sensory modalities in which they are presented. Therefore, if the presentation of information has only one modality it will exclude people with impairments in that particular sensory modality. As a solution to these problems, this document presents GUIDE, an European funded project which intends to develop a software framework which allows developers to efficiently integrate accessibility features into their applications. To perform adaptation the system must know the users, their characteristics and preferences. Thus, a prototype was implemented to assist in user trials. These trials had the goal to understand users’ interaction patterns as well as to group users with common characteristics. In order to match a user with its cluster, it was implemented a prototype named User Initialisation Application (UIA) that besides of tutoring the user on how to interact with the system, it asks the user to perform some tasks and answer some questions. When finished the UIA is able to decide which group the user identifies with. This thesis takes special focus on Dialogue Manager as it is the core component of the system architecture. It coordinates the activity of several subcomponents in a dialogue system and its main goal is to maintain a representation of the current state of the ongoing dialogue. This document presents the design of the Dialogue Manager that will run in the GUIDE framework. Additionally, in order to the Dialogue Manager and the other components understand the applications’ user interface it was implemented a tool that extracts a User Interface Markup Language (UIML) from a Web based application

Analysis and design of individual information systems to support health behavior change

As a wide-ranging socio-technical transformation, the digitalization has significantly influenced the world, bringing opportunities and challenges to our lives. Despite numerous benefits like the possibility to stay connected with people around the world, the increasing dispersion and use of digital technologies and media (DTM) pose risks to individuals’ well-being and health. Rising demands emerging from the digital world have been linked to digital stress, that is, stress directly or indirectly resulting from DTM (Ayyagari et al. 2011; Ragu-Nathan et al. 2008; Tarafdar et al. 2019; Weil and Rosen 1997), potentially intensifying individuals’ overall exposure to stress. Individuals experiencing this adverse consequence of digitalization are at elevated risk of developing severe mental health impairments (Alhassan et al. 2018; Haidt and Allen 2020; Scott et al. 2017), which is why various scholars emphasize that research should place a stronger focus on analyzing and shaping the role of the individual in a digital world, pursuing instrumental as well as humanistic objectives (Ameen et al. 2021; Baskerville 2011b). Information Systems (IS) research has long placed emphasis on the use of information and communication technology (ICT) in organizations, viewing an information system as the socio-technical system that emerges from individuals’ interaction with DTM in organizations. However, socio-technical information systems, as the essence of the IS discipline (Lee 2004; Sarker et al. 2019), are also present in different social contexts from private life. Acknowledging the increasing private use of DTM, such as smartphones and social networks, IS scholars have recently intensified their efforts to understand the human factor of IS (Avison and Fitzgerald 1991; Turel et al. 2021). A framework recently proposed by Matt et al. (2019) suggests three research angles: analyzing individuals’ behavior associated with their DTM use, analyzing what consequences arise from their DTM use behavior, and designing new technologies that promote positive or mitigate negative effects of individuals’ DTM use. Various recent studies suggest that individuals’ behavior seems to be an important lever influencing the outcomes of their DTM use (Salo et al. 2017; Salo et al. 2020; Weinstein et al. 2016). Therefore, this dissertation aims to contribute to IS research targeting the facilitation of a healthy DTM use behavior. It explores the use behavior, consequences, and design of DTM for individuals' use with the objective to deliver humanistic value by increasing individuals' health through supporting a behavior change related to their DTM use. The dissertation combines behavioral science and design science perspectives and applies pluralistic methodological approaches from qualitative (e.g., interviews, prototyping) and quantitative research (e.g., survey research, field studies), including mixed-methods approaches mixing both. Following the framework from Matt et al. (2019), the dissertation takes three perspectives therein: analyzing individuals’ behavior, analyzing individuals’ responses to consequences of DTM use, and designing information systems assisting DTM users. First, the dissertation presents new descriptive knowledge on individuals’ behavior related to their use of DTM. Specifically, it investigates how individuals behave when interacting with DTM, why they behave the way they do, and how their behavior can be influenced. Today, a variety of digital workplace technologies offer employees different ways of pursuing their goals or performing their tasks (Köffer 2015). As a result, individuals exhibit different behaviors when interacting with these technologies. The dissertation analyzes what interactional roles DTM users can take at the digital workplace and what may influence their behavior. It uses a mixed-methods approach and combines a quantitative study building on trace data from a popular digital workplace suite and qualitative interviews with users of this digital workplace suite. The empirical analysis yields eight user roles that advance the understanding of users’ behavior at the digital workplace and first insights into what factors may influence this behavior. A second study adds another perspective and investigates how habitual behavior can be changed by means of DTM design elements. Real-time feedback has been discussed as a promising way to do so (Schibuola et al. 2016; Weinmann et al. 2016). In a field experiment, employees working at the digital workplace are provided with an external display that presents real-time feedback on their office’s indoor environmental quality. The experiment examines if and to what extent the feedback influences their ventilation behavior to understand the effect of feedback as a means of influencing individuals’ behavior. The results suggest that real-time feedback can effectively alter individuals’ behavior, yet the feedback’s effectiveness reduces over time, possibly as a result of habituation to the feedback. Second, the dissertation presents new descriptive and prescriptive knowledge on individuals’ ways to mitigate adverse consequences arising from the digitalization of individuals. A frequently discussed consequence that digitalization has on individuals is digital stress. Although research efforts strive to determine what measures individuals can take to effectively cope with digital stress (Salo et al. 2017; Salo et al. 2020; Weinert 2018), further understanding of individuals’ coping behavior is needed (Weinert 2018). A group at high risk of suffering from the adverse effects of digital stress is adolescents because they grow up using DTM daily and are still developing their identity, acquiring mental strength, and adopting essential social skills. To facilitate a healthy DTM use, the dissertation explores what strategies adolescents use to cope with the demands of their DTM use. Combining a qualitative and a quantitative study, it presents 30 coping responses used by adolescents, develops five factors underlying adolescents’ activation of coping responses, and identifies gender- and age-related differences in their coping behavior. Third, the dissertation presents new prescriptive knowledge on the design of individual information systems supporting individuals in understanding and mitigating their perceived stress. Facilitated by the sensing capabilities of modern mobile devices, it explores the design and development of mobile systems that assess stress and support individuals in coping with stress by initiating a change of stress-related behavior. Since there is currently limited understanding of how to develop such systems, this dissertation explores various facets of their design and development. As a first step, it presents the development of a prototype aiming for life-integrated stress assessment, that is, the mobile sensor-based assessment of an individual’s stress without interfering with their daily routines. Data collected with the prototype yields a stress model relating sensor data to individuals’ perception of stress. To deliver a more generalized perspective on mobile stress assessment, the dissertation further presents a literature- and experience-based design theory comprising a design blueprint, design requirements, design principles, design features, and a discussion of potentially required trade-offs. Mobile stress assessment may be used for the development of mobile coping assistants. Aiming to assist individuals in effectively coping with stress and preventing future stress, a mobile coping assistant should recommend adequate coping strategies to the stressed individual in real-time or execute targeted actions within a defined scope of action automatically. While the implementation of a mobile coping assistant is yet up to future research, the dissertation presents an abstract design and algorithm for selecting appropriate coping strategies. To sum up, this dissertation contributes new knowledge on the digitalization of individuals to the IS knowledge bases, expanding both descriptive and prescriptive knowledge. Through the combination of diverse methodological approaches, it delivers knowledge on individuals’ behavior when using DTM, on the mitigation of consequences that may arise from individuals’ use of DTM, and on the design of individual information systems with the goal of facilitating a behavior change, specifically, regarding individuals’ coping with stress. Overall, the research contained in this dissertation may promote the development of digital assistants that support individuals’ in adopting a healthy DTM use behavior and thereby contribute to shaping a socio-technical environment that creates more benefit than harm for all individuals

Task-based Adaptation of Graphical Content in Smart Visual Interfaces

To be effective visual representations must be adapted to their respective context of use, especially in so-called Smart Visual Interfaces striving to present specifically those information required for the task at hand. This thesis proposes a generic approach that facilitate the automatic generation of task-specific visual representations from suitable task descriptions. It is discussed how the approach is applied to four principal content types raster images, 2D vector and 3D graphics as well as data visualizations, and how existing display techniques can be integrated into the approach.Effektive visuelle Repräsentationen müssen an den jeweiligen Nutzungskontext angepasst sein, insbesondere in sog. Smart Visual Interfaces, welche anstreben, möglichst genau für die aktuelle Aufgabe benötigte Informationen anzubieten. Diese Arbeit entwirft einen generischen Ansatz zur automatischen Erzeugung aufgabenspezifischer Darstellungen anhand geeigneter Aufgabenbeschreibungen. Es wird gezeigt, wie dieser Ansatz auf vier grundlegende Inhaltstypen Rasterbilder, 2D-Vektor- und 3D-Grafik sowie Datenvisualisierungen anwendbar ist, und wie existierende Darstellungstechniken integrierbar sind

Human-Computer Interaction

In this book the reader will find a collection of 31 papers presenting different facets of Human Computer Interaction, the result of research projects and experiments as well as new approaches to design user interfaces. The book is organized according to the following main topics in a sequential order: new interaction paradigms, multimodality, usability studies on several interaction mechanisms, human factors, universal design and development methodologies and tools

Web engineering for workflow-based applications: models, systems and methodologies

Workflow-based Web applications present a central pillar of companies\u27 endeavors towards increased business process efficiency and flexibility. Considering their particular characteristics, this book presents innovative approaches for their efficient, completely model-driven construction with particular emphasis on effective stakeholder involvement, usability-oriented dialog design and cross-methodological reuse