Assistive Systems for the Visually Impaired Based on Image Processing

In this chapter, we proposed three assistive systems for visually impaired individuals based on image processing: Kinect cane system, Kinect goggle system, and light checking system. The Kinect cane system can detect obstacles of various sizes and also recognize objects such as seats. A visually impaired user is notified of the results of detection and recognition by means of vibration feedback. The Kinect goggle system is another type of wearable system, and can make user’s hands free. The light checking system is implemented as an application for a smartphone, and can tell a visually impaired user the ON/OFF states of room lights and elevator button lights. The experimental results demonstrate that the proposed systems are effective in helping visually impaired individuals in everyday environments

Embedding mobile learning into everyday life settings

The increasing ubiquity of smartphones has changed the way we interact with information and acquire new knowledge. The prevalence of personal mobile devices in our everyday lives creates new opportunities for learning that exceed the narrow boundaries of a school’s classroom and provide the foundations for lifelong learning. Learning can now happen whenever and wherever we are; whether on the sofa at home, on the bus during our commute, or on a break at work. However, the flexibility offered by mobile learning also creates its challenges. Being able to learn anytime and anywhere does not necessarily result in learning uptake. Without the school environment’s controlled schedule and teacher guidance, the learners must actively initiate learning activities, keep up repetition schedules, and cope with learning in interruption-prone everyday environments. Both interruptions and infrequent repetition can harm the learning process and long-term memory retention. We argue that current mobile learning applications insufficiently support users in coping with these challenges. In this thesis, we explore how we can utilize the ubiquity of mobile devices to ensure frequent engagement with the content, focusing primarily on language learning and supporting users in dealing with learning breaks and interruptions. Following a user-centered design approach, we first analyzed mobile learning behavior in everyday settings. Based on our findings, we proposed concepts and designs, developed research prototypes, and evaluated them in laboratory and field evaluations with a specific focus on user experience. To better understand users’ learning behavior with mobile devices, we first characterized their interaction with mobile learning apps through a detailed survey and a diary study. Both methods confirmed the enormous diversity in usage situations and preferences. We observed that learning often happens unplanned, infrequently, among the company of friends or family, or while simultaneously performing secondary tasks such as watching TV or eating. The studies further uncovered a significant prevalence of interruptions in everyday settings that affected users’ learning behavior, often leading to suspension and termination of the learning activities. We derived design implications to support learning in diverse situations, particularly aimed at mitigating the adverse effects of multitasking and interruptions. The proposed strategies should help designers and developers create mobile learning applications that adapt to the opportunities and challenges of learning in everyday mobile settings. We explored four main challenges, emphasizing that (1) we need to consider that Learning in Everyday Settings is Diverse and Interruption-prone, (2) learning performance is affected by Irregular and Infrequent Practice Behavior, (3) we need to move From Static to Personalized Learning, and (4) that Interruptions and Long Learning Breaks can Negatively Affect Performance. To tackle these challenges, we propose to embed learning into everyday smartphone interactions, which could foster frequent engagement with – and implicitly personalize – learning content (according to users’ interests and skills). Further, we investigate how memory cues could be applied to support task resumption after interruptions in mobile learning. To confirm that our idea of embedding learning into everyday interactions can increase exposure, we developed an application integrating learning tasks into the smartphone authentication process. Since unlocking the smartphone is a frequently performed action without any other purpose, our subjects appreciated the idea of utilizing this process to perform quick and simple learning interactions. Evidence from a comparative user study showed that embedding learning tasks into the unlocking mechanism led to significantly more interactions with the learning content without impairing the learning quality. We further explored a method for embedding language comprehension assessment into users’ digital reading and listening activities. By applying physiological measurements as implicit input, we reliably detected unknown words during laboratory evaluations. Identifying such knowledge gaps could be used for the provision of in-situ support and to inform the generation of personalized language learning content tailored to users’ interests and proficiency levels. To investigate memory cueing as a concept to support task resumption after interruptions, we complemented a theoretical literature analysis of existing applications with two research probes implementing and evaluating promising design concepts. We showed that displaying memory cues when the user resumes the learning activity after an interruption improves their subjective user experience. A subsequent study presented an outlook on the generalizability of memory cues beyond the narrow use case of language learning. We observed that the helpfulness of memory cues for reflecting on prior learning is highly dependent on the design of the cues, particularly the granularity of the presented information. We consider interactive cues for specific memory reactivation (e.g., through multiple-choice questions) a promising scaffolding concept for connecting individual micro-learning sessions when learning in everyday settings. The tools and applications described in this thesis are a starting point for designing applications that support learning in everyday settings. We broaden the understanding of learning behavior and highlight the impact of interruptions in our busy everyday lives. While this thesis focuses mainly on language learning, the concepts and methods have the potential to be generalized to other domains, such as STEM learning. We reflect on the limitations of the presented concepts and outline future research perspectives that utilize the ubiquity of mobile devices to design mobile learning interactions for everyday settings.Die Allgegenwärtigkeit von Smartphones verändert die Art und Weise wie wir mit Informationen umgehen und Wissen erwerben. Die weite Verbreitung von mobilen Endgeräten in unserem täglichen Leben führt zu neuen Möglichkeiten des Lernens, welche über die engen Grenzen eines Klassenraumes hinausreichen und das Fundament für lebenslanges Lernen schaffen. Lernen kann nun zu jeder Zeit und an jedem Ort stattfinden: auf dem Sofa Zuhause, im Bus während des Pendelns oder in der Pause auf der Arbeit. Die Flexibilität des mobilen Lernens geht jedoch zeitgleich mit Herausforderungen einher. Ohne den kontrollierten Ablaufplan und die Unterstützung der Lehrpersonen im schulischen Umfeld sind die Lernenden selbst dafür verantwortlich, aktiv Lernsitzungen zu initiieren, Wiederholungszyklen einzuhalten und Lektionen in unterbrechungsanfälligen Alltagssituationen zu meistern. Sowohl Unterbrechungen als auch unregelmäßige Wiederholung von Inhalten können den Lernprozess behindern und der Langzeitspeicherung der Informationen schaden. Wir behaupten, dass aktuelle mobile Lernanwendungen die Nutzer*innen nur unzureichend in diesen Herausforderungen unterstützen. In dieser Arbeit erforschen wir, wie wir uns die Allgegenwärtigkeit mobiler Endgeräte zunutze machen können, um zu erreichen, dass Nutzer*innen regelmäßig mit den Lerninhalten interagieren. Wir fokussieren uns darauf, sie im Umgang mit Unterbrechungen und Lernpausen zu unterstützen. In einem nutzerzentrierten Designprozess analysieren wir zunächst das Lernverhalten auf mobilen Endgeräten in alltäglichen Situationen. Basierend auf den Erkenntnissen schlagen wir Konzepte und Designs vor, entwickeln Forschungsprototypen und werten diese in Labor- und Feldstudien mit Fokus auf User Experience (wörtl. “Nutzererfahrung”) aus. Um das Lernverhalten von Nutzer*innen mit mobilen Endgeräten besser zu verstehen, versuchen wir zuerst die Interaktionen mit mobilen Lernanwendungen durch eine detaillierte Umfrage und eine Tagebuchstudie zu charakterisieren. Beide Methoden bestätigen eine enorme Vielfalt von Nutzungssituationen und -präferenzen. Wir beobachten, dass Lernen oft ungeplant, unregelmäßig, im Beisein von Freunden oder Familie, oder während der Ausübung anderer Tätigkeiten, beispielsweise Fernsehen oder Essen, stattfindet. Die Studien decken zudem Unterbrechungen in Alltagssituationen auf, welche das Lernverhalten der Nutzer*innen beeinflussen und oft zum Aussetzen oder Beenden der Lernaktivität führen. Wir leiten Implikationen ab, um Lernen in vielfältigen Situationen zu unterstützen und besonders die negativen Einflüsse von Multitasking und Unterbrechungen abzuschwächen. Die vorgeschlagenen Strategien sollen Designer*innen und Entwickler*innen helfen, mobile Lernanwendungen zu erstellen, welche sich den Möglichkeiten und Herausforderungen von Lernen in Alltagssituationen anpassen. Wir haben vier zentrale Herausforderungen identifiziert: (1) Lernen in Alltagssituationen ist divers und anfällig für Unterbrechungen; (2) Die Lerneffizienz wird durch unregelmäßiges Wiederholungsverhalten beeinflusst; (3) Wir müssen von statischem zu personalisiertem Lernen übergehen; (4) Unterbrechungen und lange Lernpausen können dem Lernen schaden. Um diese Herausforderungen anzugehen, schlagen wir vor, Lernen in alltägliche Smartphoneinteraktionen einzubetten. Dies führt zu einer vermehrten Beschäftigung mit Lerninhalten und könnte zu einer impliziten Personalisierung von diesen anhand der Interessen und Fähigkeiten der Nutzer*innen beitragen. Zudem untersuchen wir, wie Memory Cues (wörtl. “Gedächtnishinweise”) genutzt werden können, um das Fortsetzen von Aufgaben nach Unterbrechungen im mobilen Lernen zu erleichtern. Um zu zeigen, dass unsere Idee des Einbettens von Lernaufgaben in alltägliche Interaktionen wirklich die Beschäftigung mit diesen erhöht, haben wir eine Anwendung entwickelt, welche Lernaufgaben in den Entsperrprozess von Smartphones integriert. Da die Authentifizierung auf dem Mobilgerät eine häufig durchgeführte Aktion ist, welche keinen weiteren Mehrwert bietet, begrüßten unsere Studienteilnehmenden die Idee, den Prozess für die Durchführung kurzer und einfacher Lerninteraktionen zu nutzen. Ergebnisse aus einer vergleichenden Nutzerstudie haben gezeigt, dass die Einbettung von Aufgaben in den Entsperrprozess zu signifikant mehr Interaktionen mit den Lerninhalten führt, ohne dass die Lernqualität beeinträchtigt wird. Wir haben außerdem eine Methode untersucht, welche die Messung von Sprachverständnis in die digitalen Lese- und Höraktivitäten der Nutzer*innen einbettet. Mittels physiologischer Messungen als implizite Eingabe können wir in Laborstudien zuverlässig unbekannte Wörter erkennen. Die Aufdeckung solcher Wissenslücken kann genutzt werden, um in-situ Untestützung bereitzustellen und um personalisierte Lerninhalte zu generieren, welche auf die Interessen und das Wissensniveau der Nutzer*innen zugeschnitten sind. Um Memory Cues als Konzept für die Unterstützung der Aufgabenfortsetzung nach Unterbrechungen zu untersuchen, haben wir eine theoretische Literaturanalyse von bestehenden Anwendungen um zwei Forschungsarbeiten erweitert, welche vielversprechende Designkonzepte umsetzen und evaluieren. Wir haben gezeigt, dass die Präsentation von Memory Cues die subjektive User Experience verbessert, wenn der Nutzer die Lernaktivität nach einer Unterbrechung fortsetzt. Eine Folgestudie stellt einen Ausblick auf die Generalisierbarkeit von Memory Cues dar, welcher über den Tellerrand des Anwendungsfalls Sprachenlernen hinausschaut. Wir haben beobachtet, dass der Nutzen von Memory Cues für das Reflektieren über gelernte Inhalte stark von dem Design der Cues abhängt, insbesondere von der Granularität der präsentierten Informationen. Wir schätzen interaktive Cues zur spezifischen Gedächtnisaktivierung (z.B. durch Mehrfachauswahlfragen) als einen vielversprechenden Unterstützungsansatz ein, welcher individuelle Mikrolerneinheiten im Alltag verknüpfen könnte. Die Werkzeuge und Anwendungen, die in dieser Arbeit beschrieben werden, sind ein Startpunkt für das Design von Anwendungen, welche das Lernen in Alltagssituationen unterstützen. Wir erweitern das Verständnis, welches wir von Lernverhalten im geschäftigen Alltagsleben haben und heben den Einfluss von Unterbrechungen in diesem hervor. Während sich diese Arbeit hauptsächlich auf das Lernen von Sprachen fokussiert, haben die vorgestellten Konzepte und Methoden das Potential auf andere Bereiche übertragen zu werden, beispielsweise das Lernen von MINT Themen. Wir reflektieren über die Grenzen der präsentierten Konzepte und skizzieren Perspektiven für zukünftige Forschungsarbeiten, welche sich die Allgegenwärtigkeit von mobilen Endgeräten zur Gestaltung von Lernanwendungen für den Alltag zunutze machen

Novas soluções para a interação entre operador e passageiro nos transportes públicos

Environmental awareness has been increasing over the years. Since the transportation sector is one of the great contributors of greenhouse gas (GHG) emissions, it is necessary to motivate people to leave their car at home and use public transport. However, service providers must meet passengers' needs in a more personalized way and improve the quality of the service they offer. This project aims to provide a tool that positively impacts the required shift towards sustainability, and it relies on the exchange of passenger information to achieve this goal. The challenge proposed by the company, OPT, is based on the creation of new solutions for the interaction between operator and passenger through the Smartphone. Given that OPT is headquartered in Porto, its area of intervention par excellence is focused on the Metropolitan Area of Porto (MAP). The project focused on the acquisition of passenger data from this same area to characterize it in terms of urban mobility. The study has the MAP as a starting point however, the goal is to create a solution that is scalable to any city. The project began by characterizing the current situation regarding passenger information in the metropolitan area of Porto and a passenger survey was conducted to understand their perception towards information in public transports. Despite the available app offer, there are opportunities for improvement in the context of the integration of essential app functionalities and user experience. In addition, a bilateral interaction between the two stakeholders is needed in order to generate relevant information from user feedback. In this way, a new mobile application concept for public transport was designed to co-create value and improve the quality of service and allow seamless mobility across the travel network. Taking into account the commercial potential of this project, the strategies of innovation and business are based on the optimization of resources and value for the clientAo longo dos anos, a consciencialização das populações relativamente ao meio ambiente tem vindo a aumentar. O setor dos transportes, nomeadamente o de passageiros, tem um grande impacto na emissão de gases poluentes e, por isso, a utilização do veículo próprio como meio de transporte de eleição tem de ser reduzida. Para isto, é necessário que o transporte público se torne apelativo e sirva as necessidades das pessoas de forma personalizada e com qualidade. Este projeto é desenvolvido de acordo com a premissa de melhoria da qualidade dos transportes públicos e baseia-se na troca de informação para atingir este objetivo. O desafio lançado pela OPT assenta na criação de novas soluções para a interação entre operador e passageiro. Tendo em conta que a OPT está sediada no Porto, a sua área de intervenção por excelência foca-se na Área Metropolitana do Porto (AMP) e, por isso, o projeto focou-se na aquisição de dados desta mesma área para a caracterizar em termos de mobilidade. O estudo tem a AMP como ponto de partida, no entanto, o objetivo é criar uma solução que seja escalável para qualquer cidade. Para analisar o estado da informação ao público na área metropolitana do Porto, procedeu-se ao levantamento da oferta e à realização de um inquérito aos passageiros. Apesar de existir alguma oferta relativamente a aplicações móveis que disponibilizam informação, existem oportunidades de melhoria no contexto da integração de funcionalidades essenciais e user experience. Além disso, a evolução indica uma interação entre os intervenientes de forma a que se possa gerar informação relevante a partir do feedback do utilizador. Desta forma, foi concetualizado um novo modelo de aplicação móvel para os transportes públicos. O objetivo é baseado na co-criação de valor para melhorar a qualidade do serviço e permitir uma mobilidade contínua, motivando assim para a utilização dos transportes coletivos. Tendo em conta o potencial comercial deste projeto, reflete-se sobre as estratégias de inovação e negócio tendo por base a otimização dos recursos e o valor para o clienteMestrado em Engenharia e Gestão Industria

Regulating Habit-Forming Technology

Tech developers, like slot machine designers, strive to maximize the user’s “time on device.” They do so by designing habit-forming products— products that draw consciously on the same behavioral design strategies that the casino industry pioneered. The predictable result is that most tech users spend more time on device than they would like, about five hours of phone time a day, while a substantial minority develop life-changing behavioral problems similar to problem gambling. Other countries have begun to regulate habit-forming tech, and American jurisdictions may soon follow suit. Several state legislatures today are considering bills to regulate “loot boxes,” a highly addictive slot-machine- like mechanic that is common in online video games. The Federal Trade Commission has also announced an investigation into the practice. As public concern mounts, it is surprisingly easy to envision consumer regulation extending beyond video games to other types of apps. Just as tobacco regulations might prohibit brightly colored packaging and fruity flavors, a social media regulation might limit the use of red notification badges or “streaks” that reward users for daily use. It is unclear how much of this regulation could survive First Amendment scrutiny; software, unlike other consumer products, is widely understood as a form of protected “expression.” But it is also unclear whether well-drawn laws to combat compulsive technology use would seriously threaten First Amendment values. At a very low cost to the expressive interests of tech companies, these laws may well enhance the quality and efficacy of online speech by mitigating distraction and promoting deliberation

Blush: An Exploration of Empathetic Cultivation Through Technological Devices

Technology has helped us keep in touch. Letters, telegrams, telephones, email, video chatting, social-networking — all ways in which we have communicated throughout the years. Each innovation has allowed us to lead independent lives from our loved ones and friends, while trying not to lose touch with each other. It’s freed us, but at the same time we’ve become chained it to. We’ve lost our ability to converse and empathize with each other directly — we’ve become more comfortable in front of a screen. Unfortunately for us, digital devices in their current state are terrible forms of communication. It lacks the vital components that are needed to develop empathy for others, which helps us to fully understand other individuals and allow us to become more compassionate towards them. Screens have lead to a world of anonymity and sterile conversations — which in turn has lead to misunderstanding and potential stresses on relationships. This research used lean user-experience design strategies in order to develop a social experience that allows users to visualize and communicate their emotions to one another — allowing them to become more empathetic towards each and act according to other’s emotional states; not just solely by verbal information in which they receive from the sender. Research consisted of surveying the core target audience which was based upon online surveys, paper surveys, in-person interviews, and usability testing. Core design components consisted of user-interface design, product design, information design, and interaction design principles. The overall goal of this conceptual research and experience design was to explore how design could create new uses of current technological devices for social good and improved digital communication — allowing us not to further distance ourselves from each other, but to allow technology to bring us closer together and to help us understand each other more intimately

Supporting Diabetes Self-Management with Ubiquitous Computing Technologies: A User-Centered Inquiry

Ubiquitous computing technologies offer opportunities to improve treatments for chronic health conditions. Type 1 diabetes is a compelling use-case for such approaches, given its severity, and need for individuals to make frequent care decisions, informed by complex data. However, current apps, typically based on effortful reflection on collected data, generally show poor adoption, lack vital cognitive and emotional support, and are poorly tailored to users’ actual diabetes decision making processes. This thesis investigates how diabetes apps can be improved from a user-centered perspective. An initial questionnaire-based study investigated how well existing diabetes apps meet user needs. Perceived benefits, limitations, and reasons for low adoption rates were identified. A talk-aloud study of detailed user interactions with diabetes logging apps was conducted to characterize the benefits and limitations of diverse UI elements for T1 diabetes management, and to more precisely identify wider problems with current interaction designs. This led to positing a refined version of Mamykina et al.’s model for diabetes self-management, to account for observed practices, whereby the previously accepted habitual and sensemaking cognitive states are augmented by a posited ‘fluid contextual reasoning’ (FCR) mode, which allows multiple contextual factors to be balanced for dynamic course correction when navigating complex situations, using previously learned knowledge. To investigate user perceptions of the levels and kinds of monitoring anticipated in next generation diabetes decision support systems, a 4-week technology probe, in which participants used multiple networked devices and external data aggregation, was used to frame requirements for user-centered development of such future systems. Integrating all of the above work, an iterative design process was undertaken to create DUETS, a card-based system to facilitate reflection by designers, users, and other stakeholders on diabetes support management systems. The resulting tool and method were then implemented and evaluated through structured sessions with stakeholder focus groups

Accessibility of Health Data Representations for Older Adults: Challenges and Opportunities for Design

Health data of consumer off-the-shelf wearable devices is often conveyed to users through visual data representations and analyses. However, this is not always accessible to people with disabilities or older people due to low vision, cognitive impairments or literacy issues. Due to trade-offs between aesthetics predominance or information overload, real-time user feedback may not be conveyed easily from sensor devices through visual cues like graphs and texts. These difficulties may hinder critical data understanding. Additional auditory and tactile feedback can also provide immediate and accessible cues from these wearable devices, but it is necessary to understand existing data representation limitations initially. To avoid higher cognitive and visual overload, auditory and haptic cues can be designed to complement, replace or reinforce visual cues. In this paper, we outline the challenges in existing data representation and the necessary evidence to enhance the accessibility of health information from personal sensing devices used to monitor health parameters such as blood pressure, sleep, activity, heart rate and more. By creating innovative and inclusive user feedback, users will likely want to engage and interact with new devices and their own data

Kodinportti Mobile user interface usability research and redesign

Abstract. Usability is described as follows “The extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use”. Usability plays a crucial role in the success of interactive products and services. With good usability can be achieved an important competitive advantage in an ever-growing market if it is involved in software products development life cycle. User experience (UX) is based on people’s expectations and the following emotions and it involves extensively everything that happens outside the screen. UX design aims to emphasize features that are bringing pleasure in addition to practicality. Usability evaluation is easy and very cost-effective way to analyze user interface and find possible usability problems. In general, the empirical methods like heuristic evaluation and usability testing are the most used and principal means to evaluate user interfaces. This thesis focuses on exploring the usability of Kodinportti Mobile application. The motivation for the study is to improve its user interface (UI) more user-friendly using usability research methods that include heuristic evaluation, usability testing, and user experience research. The practical work in the study is responsible by the University of Oulu usability testing course student group and the group also reports the results of research. The goal of the study is to redesign the app’s UI. Kodinportti Mobile is designed to meet the needs of the residents of the housing association and serves as a supplement to the electronic bulletin board for the residents. Mobile application UI design applies the same basic guidelines as any other UI design process and in mobile application UI design, it is often recommended to keep it simple as possible and cut out as much as possible. The results of usability research revealed several flaws in the apps usability and user experience. In the final phase of the thesis, the concept plan of new UI was designed based on the research results. There was also a perception during the process that it would be recommendable that designers are involved in all stages of the process. That would improve the process and minimize the possibility of misunderstandings during the process, which will certainly have a positive impact on the result

Mapping Accessible Paths in the City Using Collective Intelligence

New information and communication technologies (ICTs) have an increasingly stronger role in people\u27s lives, especially after the commoditization of smartphones. They affect many aspects of everyday life, including urban mobility. Some applications, including Waze, benefit from the collective intelligence (CI) of the crowds to gather the information they need to provide users with good advice on the routes to follow. But they are mainly focused on roads and streets, giving little information on the quality of sidewalks, which are essential to pedestrians, people on wheelchairs and blind people. With the intention to improve the mobility of citizens with special needs, we developed the prototype of an application that allows users themselves to update accessibility maps, tagging obstacles and also indicating the existence of resources that contribute to improve the mobility of people with special needs in urban spaces. Tests in a controlled environment helped to debug the application’s functionalities, before members of the intended target group of users were finally exposed to it. Results are promising, as users were able to include relevant data by themselves and seem motivated to keep doing so, due a sense of utility, social facilitation or simply due to altruism, as anticipated by the CI literature. One unexpected outcome was that impaired users are more excited about the potential the application has to give visibility to the challenges they face than with the actual improvement it can bring to their mobility