PRESTK : situation-aware presentation of messages and infotainment content for drivers

The amount of in-car information systems has dramatically increased over the last few years. These potentially mutually independent information systems presenting information to the driver increase the risk of driver distraction. In a first step, orchestrating these information systems using techniques from scheduling and presentation planning avoid conflicts when competing for scarce resources such as screen space. In a second step, the cognitive capacity of the driver as another scarce resource has to be considered. For the first step, an algorithm fulfilling the requirements of this situation is presented and evaluated. For the second step, I define the concept of System Situation Awareness (SSA) as an extension of Endsley’s Situation Awareness (SA) model. I claim that not only the driver needs to know what is happening in his environment, but also the system, e.g., the car. In order to achieve SSA, two paths of research have to be followed: (1) Assessment of cognitive load of the driver in an unobtrusive way. I propose to estimate this value using a model based on environmental data. (2) Developing model of cognitive complexity induced by messages presented by the system. Three experiments support the claims I make in my conceptual contribution to this field. A prototypical implementation of the situation-aware presentation management toolkit PRESTK is presented and shown in two demonstrators.In den letzten Jahren hat die Menge der informationsanzeigenden Systeme im Auto drastisch zugenommen. Da sie potenziell unabhängig voneinander ablaufen, erhöhen sie die Gefahr, die Aufmerksamkeit des Fahrers abzulenken. Konflikte entstehen, wenn zwei oder mehr Systeme zeitgleich auf limitierte Ressourcen wie z. B. den Bildschirmplatz zugreifen. Ein erster Schritt, diese Konflikte zu vermeiden, ist die Orchestrierung dieser Systeme mittels Techniken aus dem Bereich Scheduling und Präsentationsplanung. In einem zweiten Schritt sollte die kognitive Kapazität des Fahrers als ebenfalls limitierte Ressource berücksichtigt werden. Der Algorithmus, den ich zu Schritt 1 vorstelle und evaluiere, erfüllt alle diese Anforderungen. Zu Schritt 2 definiere ich das Konzept System Situation Awareness (SSA), basierend auf Endsley’s Konzept der Situation Awareness (SA). Dadurch wird erreicht, dass nicht nur der Fahrer sich seiner Umgebung bewusst ist, sondern auch das System (d.h. das Auto). Zu diesem Zweck m¨ussen zwei Bereiche untersucht werden: (1) Die kognitive Belastbarkeit des Fahrers unaufdringlich ermitteln. Dazu schlage ich ein Modell vor, das auf Umgebungsinformationen basiert. (2) Ein weiteres Modell soll die Komplexität der präsentierten Informationen bestimmen. Drei Experimente stützen die Behauptungen in meinem konzeptuellen Beitrag. Ein Prototyp des situationsbewussten Präsentationsmanagement-Toolkits PresTK wird vorgestellt und in zwei Demonstratoren gezeigt

Improving elderly access to audiovisual and social media, using a multimodal human-computer interface

With the growth of Internet and especially, the proliferation of social media services, an opportunity has emerged for greater social and technological integration of the elderly. However, the adoption of new technologies by this segment of the population is not always straightforward mainly due to the physical and cognitive difficulties that are typically associated with ageing. Thus, for elderly to take advantage of new technologies and services that can help improve their quality of life, barriers must be broken by designing solutions with those needs in mind from the start. The aim of this work is to verify whether Multimodal Human-Computer Interaction (MHCI) systems designed with Universal Accessibility principles, taking into account elderly specific requirements, facilitate the adoption and access to popular Social Media Services (SMSs) and Audiovisual Communication Services, thus potentially contributing to the elderly social and technological integration. A user study was initially conducted in order to learn about the limitations and requirements of elderly people with existing HCI, concerning particularly SMSs and Audiovisual Communication Services, such as Facebook or Windows Live Messenger (WLM). The results of the study, basically a set of new MHCI requirements, were used to inform further development and enhancement of a multimodal prototype previously proposed for mobility-impaired individuals, now targeting the elderly. The prototype allows connecting users with their social networks through a text, audio and video communication service and integrates with SMSs, using natural interaction modalities, like speech, touch and gesture. After the development stage a usability evaluation study was conducted. The study reveals that such multimodal solution could simplify accessibility to the supported services, through the provision of simpler to use interfaces, by adopting natural interaction modalities and by being more satisfying to use by the elderly population, than most of the current graphical user interfaces for those same services, such as Facebook.Com o crescimento da Internet e, especialmente, das redes sociais surge a oportunidade para uma maior integração social e tecnológica dos idosos. No entanto, a adoção de novas tecnologias por essa população nem sempre é simples, principalmente devido às dificuldades físicas e cognitivas que estão associadas com o envelhecimento. Assim, e para que os idosos possam tirar proveito das novas tecnologias e serviços que podem ajudar a melhorar sua qualidade de vida, essas barreiras devem ser ultrapassadas desenhando soluções de raiz com essas necessidades em mente. O objetivo deste trabalho é verificar se interfaces humano-computador multimodais desenhadas com base em princípios de Acessibilidade Universal, tendo em conta requisitos específicos da população idosa, proporcionam um acesso simplificado a serviços de média social e serviços de comunicação audiovisuais, potencialmente contribuindo para a integração social e tecnológica desta população. Um estudo com utilizadores foi inicialmente conduzido a fim de apurar as necessidades especiais desses utilizadores com soluções de software, mais especificamente serviços de média social e serviços de conferência, como o Facebook ou o Windows Live Messenger. Os resultados do estudo foram utilizados para planear o desenvolvimento de um protótipo multimodal proposto anteriormente para utilizadores com mobilidade reduzida. Este permite ligar utilizadores às suas redes sociais através de um serviço de conferência por texto, áudio e vídeo, e um serviço integrado de média social, usando modalidades de interação natural, como o toque, fala e gestos. Após a fase de desenvolvimento foi realizado um estudo de usabilidade. Esse estudo revelou que este tipo de soluções pode simplificar a acessibilidade aos serviços considerados, dado ter interfaces mais simples, por adotar modalidades de interação mais naturais e por ser mais gratificante do que a maioria das interfaces gráficas atuais para os mesmos serviços, como por exemplo o Facebook

A speaker classification framework for non-intrusive user modeling : speech-based personalization of in-car services

Speaker Classification, i.e. the automatic detection of certain characteristics of a person based on his or her voice, has a variety of applications in modern computer technology and artificial intelligence: As a non-intrusive source for user modeling, it can be employed for personalization of human-machine interfaces in numerous domains. This dissertation presents a principled approach to the design of a novel Speaker Classification system for automatic age and gender recognition which meets these demands. Based on literature studies, methods and concepts dealing with the underlying pattern recognition task are developed. The final system consists of an incremental GMM-SVM supervector architecture with several optimizations. An extensive data-driven experiment series explores the parameter space and serves as evaluation of the component. Further experiments investigate the language-independence of the approach. As an essential part of this thesis, a framework is developed that implements all tasks associated with the design and evaluation of Speaker Classification in an integrated development environment that is able to generate efficient runtime modules for multiple platforms. Applications from the automotive field and other domains demonstrate the practical benefit of the technology for personalization, e.g. by increasing local danger warning lead time for elderly drivers.Die Sprecherklassifikation, also die automatische Erkennung bestimmter Merkmale einer Person anhand ihrer Stimme, besitzt eine Vielzahl von Anwendungsmöglichkeiten in der modernen Computertechnik und Künstlichen Intelligenz: Als nicht-intrusive Wissensquelle für die Benutzermodellierung kann sie zur Personalisierung in vielen Bereichen eingesetzt werden. In dieser Dissertation wird ein fundierter Ansatz zum Entwurf eines neuartigen Sprecherklassifikationssystems zur automatischen Bestimmung von Alter und Geschlecht vorgestellt, welches diese Anforderungen erfüllt. Ausgehend von Literaturstudien werden Konzepte und Methoden zur Behandlung des zugrunde liegenden Mustererkennungsproblems entwickelt, welche zu einer inkrementell arbeitenden GMM-SVM-Supervector-Architektur mit diversen Optimierungen führen. Eine umfassende datengetriebene Experimentalreihe dient der Erforschung des Parameterraumes und zur Evaluierung der Komponente. Weitere Studien untersuchen die Sprachunabhängigkeit des Ansatzes. Als wesentlicher Bestandteil der Arbeit wird ein Framework entwickelt, das alle im Zusammenhang mit Entwurf und Evaluierung von Sprecherklassifikation anfallenden Aufgaben in einer integrierten Entwicklungsumgebung implementiert, welche effiziente Laufzeitmodule für verschiedene Plattformen erzeugen kann. Anwendungen aus dem Automobilbereich und weiteren Domänen demonstrieren den praktischen Nutzen der Technologie zur Personalisierung, z.B. indem die Vorlaufzeit von lokalen Gefahrenwarnungen für ältere Fahrer erhöht wird

Proceedings of the 2018 Canadian Society for Mechanical Engineering (CSME) International Congress

Published proceedings of the 2018 Canadian Society for Mechanical Engineering (CSME) International Congress, hosted by York University, 27-30 May 2018

The always best positioned paradigm for mobile indoor applications

In this dissertation, methods for personal positioning in outdoor and indoor environments are investigated. The Always Best Positioned paradigm, which has the goal of providing a preferably consistent self-positioning, will be defined. Furthermore, the localization toolkit LOCATO will be presented, which allows to easily realize positioning systems that follow the paradigm. New algorithms were developed, which particularly address the robustness of positioning systems with respect to the Always Best Positioned paradigm. With the help of this toolkit, three example positioning-systems were implemented, each designed for different applications and requirements: a low-cost system, which can be used in conjunction with user-adaptive public displays, a so-called opportunistic system, which enables positioning with room-level accuracy in any building that provides a WiFi infrastructure, and a high-accuracy system for instrumented environments, which works with active RFID tags and infrared beacons. Furthermore, a new and unique evaluation-method for positioning systems is presented, which uses step-accurate natural walking-traces as ground truth. Finally, six location based services will be presented, which were realized either with the tools provided by LOCATO or with one of the example positioning-systems.In dieser Doktorarbeit werden Methoden zur Personenpositionierung im Innen- und Außenbereich von Gebäuden untersucht. Es wird das ,,Always Best Positioned” Paradigma definiert, welches eine möglichst lückenlose Selbstpositionierung zum Ziel hat. Weiterhin wird die Lokalisierungsplattform LOCATO vorgestellt, welche eine einfache Umsetzung von Positionierungssystemen ermöglicht. Hierzu wurden neue Algorithmen entwickelt, welche gezielt die Robustheit von Positionierungssystemen unter Berücksichtigung des ,,Always Best Positioned” Paradigmas angehen. Mit Hilfe dieser Plattform wurden drei Beispiel Positionierungssysteme entwickelt, welche unterschiedliche Einsatzgebiete berücksichtigen: Ein kostengünstiges System, das im Zusammenhang mit benutzeradaptiven öffentlichen Bildschirmen benutzt werden kann; ein sogenanntes opportunistisches Positionierungssystem, welches eine raumgenaue Positionierung in allen Gebäuden mit WLAN-Infrastruktur ermöglicht, sowie ein metergenaues Positionierungssystem, welches mit Hilfe einer Instrumentierung aus aktiven RFID-Tags und Infrarot-Baken arbeitet. Weiterhin wird erstmalig eine Positionierungsevaluation vorgestellt, welche schrittgenaue, natürliche Bewegungspfade als Referenzsystem einsetzt. Im Abschluss werden 6 lokationsbasierte Dienste vorgestellt, welche entweder mit Hilfe von LOCATO oder mit Hilfe einer der drei Beispiel-Positionierungssysteme entwickelt wurden

Just-in-time Information Interfaces: A new Paradigm for Information Discovery and Exploration

We live in a time of increasing information overload. Described as “a byproduct of the lack of maturity of the information age” (Spira & Goldes, 2007), information overload can be painful, and harm our concentration - the resulting choice overload impacts out decision-making abilities. Given the problem of information overload, and the unsatisfying nature of human-information interaction using traditional browsing or keyword-based search, this research investigates how the design of just-in-time information services can improve the user experience of goal-driven interactions with information. This thesis explores the design of just-in-time information services through the iterative development of two strands of high-level prototypes (FMI and KnowDis). I custombuilt both prototype systems for the respective evaluations, which have then been conducted as part of a series of lab-based eye-tracking studies (FMI) as well as two field studies (KnowDis). The lab-based eye-tracking studies were conducted with 100 participants measuring task performance, user satisfaction, and gaze behaviour. The lab studies found that the FMI prototype design did improve the performance aspect of the user experience for all participants and improved the usability aspect of the user experience for novice participants. However, the FMI prototype design seemed to be less effective and usable for expert participants. Two field studies were conducted as part of a two-year research collaboration, which lasted a total of 10 weeks and involved approximately 70 knowledge workers overall from across the globe. As part of those field studies, 46 semi-structured interviews were also conducted. The field studies found that the KnowDis prototype design did improve the user experience for participants overall by making work-related information search more efficient. However, while the KnowDis prototype design was useful for some knowledge workers and even integrated seamlessly into their day-to-day work, it appeared to be less useful and even distracting to others