Smart Notifications Based on Priority and Context

Current Operating Systems (OSs) of devices such as desktop computers, laptops, mobile phones, and tablets, provide applications with capabilities to serve information to users via builtin notification mechanisms. If the information presented by a notification is not useful or timely, the user’s current task is needlessly disrupted. Moreover, the user is likely to dismiss an inopportune notification quickly, thus reducing user engagement. The techniques of this disclosure enable smart delivery of notifications such that notifications are delivered to the user at an opportune time. On-device neural networks are utilized to make the determination of the opportune time. With user permission, the content of a generated notification is processed to determine whether it is to be shown immediately, by interrupting the user, or whether the delivery is to be deferred until an opportune time

Deep Learning for Mobile Multimedia: A Survey

Deep Learning (DL) has become a crucial technology for multimedia computing. It offers a powerful instrument to automatically produce high-level abstractions of complex multimedia data, which can be exploited in a number of applications, including object detection and recognition, speech-to- text, media retrieval, multimodal data analysis, and so on. The availability of affordable large-scale parallel processing architectures, and the sharing of effective open-source codes implementing the basic learning algorithms, caused a rapid diffusion of DL methodologies, bringing a number of new technologies and applications that outperform, in most cases, traditional machine learning technologies. In recent years, the possibility of implementing DL technologies on mobile devices has attracted significant attention. Thanks to this technology, portable devices may become smart objects capable of learning and acting. The path toward these exciting future scenarios, however, entangles a number of important research challenges. DL architectures and algorithms are hardly adapted to the storage and computation resources of a mobile device. Therefore, there is a need for new generations of mobile processors and chipsets, small footprint learning and inference algorithms, new models of collaborative and distributed processing, and a number of other fundamental building blocks. This survey reports the state of the art in this exciting research area, looking back to the evolution of neural networks, and arriving to the most recent results in terms of methodologies, technologies, and applications for mobile environments

Fatigue detection system to aid in remote work

During the Covid-19 pandemic there was a noticeable surge in the amount of remote workers. In the aftermath of the pandemic working from home still remains a reality for many workers with noticeable impacts on the mental health of people. With the increased stress caused by current situation and the harder time establishing boundaries there was an increase in the overall stress and fatigue in workers, leading to burnouts. Fatigue detection systems are used in several areas, mainly in the automotive industry as a mean to decrease the number of accidents. This research started by approaching the Artificial Intelligence (AI) area and its domains, followed by a study of the current techniques used in order to predict fatigue. With the main ones utilising eye state, facial landmarks, electrocardiogram or heart rate. After a research into existing Fatigue detection systems was done in order to identify the strengths of solutions currently in the market, whether in the automotive industry or other applications. This thesis proposes the creation of a system able to detect fatigue in a user as well as warn him when fatigue levels increase. This system incorporates a webcam analysing the users face and performing eye state detection in order to calculate the percentage of the time the eyes are closed (PERCLOS). Heart rate data was also analysed and a model was developed in order to incorporate this data, the percentage of time the eyes are closed, the program the user has open and time of day in order to predict the level of fatigue. By combining these two different techniques this system can be more effective and more accurate in giving predictions of the level of fatigue. The review of literature showed that the conjunction of these two techniques in predicting fatigue is novelty. The developed system also contains integration with smartwatch technology in order to both harness heart rate data as well as communicate with the user via pop up notifications to inform him when fatigue levels get too high. The conclusion of this work is that eye state detection using Artificial Intelligence can achieve a high accuracy and be a reliable tool in identifying fatigue in an user. The combination of Heart Rate and PERCLOS allows the system to have a higher accuracy as well as not being completely reliant on one sensor. The creation of a fatigue prediction model was hindered by the lack of existent data in order to train a model, a problem that could be fixed with the adoption of the system in a broader scope.Durante a pandemia de Covid-19, houve um aumento notável na quantidade de trabalhadores remotos. No rescaldo da pandemia, trabalhar a partir de casa continua a ser uma realidade para muitos trabalhadores, com impactos visíveis na saúde mental das pessoas. Com o aumento do stresse causado pela situação atual e a dificuldade de estabelecer limites, houve um aumento do stresse geral e da fadiga dos trabalhadores, levando ao esgotamento. Os sistemas de detecção de fadiga são utilizados em diversas áreas, principalmente na indústria automobilística como forma de diminuir o número de acidentes. Este estudo começou por abordar a área de Inteligência Artificial (IA) e os seus domínios, seguida de um estudo das técnicas atuais utilizadas para prever a fadiga. Com os principais utilizando o estado dos olhos, pontos de referência faciais, eletrocardiograma ou frequência cardíaca. Depois foi feita uma pesquisa sobre os sistemas de detecção de fadiga existentes de forma a identificar os pontos fortes das soluções actualmente no mercado, quer seja na indústria automóvel ou outras aplicações. Esta dissertação propõe a criação de um sistema capaz de detectar fadiga num utilizador, bem como alertar quando os níveis de fadiga aumentam. Este sistema incorpora uma webcam que analisa a face do utilizador e realiza a detecção do estado dos olhos para calcular a percentagem de tempo em que os olhos estão fechados (PERCLOS). Os dados de frequência cardíaca também foram analisados e um modelo foi desenvolvido para incorporar estes dados, a percentagem de tempo que os olhos ficam fechados, o programa que o utilizador tem aberto e a hora do dia para prever o nível de fadiga. Ao combinar essas duas técnicas diferentes, este sistema pode ser mais eficaz e mais preciso em fornecer previsões do nível de fadiga. A revisão da literatura mostrou que a conjunção dessas duas técnicas na previsão da fadiga é novidade. O sistema desenvolvido também contém integração com a tecnologia smartwatch para aproveitar os dados da frequência cardíaca e comunicar com o utilizador por meio de notificações pop-up para informá-lo quando os níveis de fadiga se encontrarem altos. A conclusão deste trabalho é que a detecção do estado ocular usando Inteligência Artificial pode alcançar uma alta precisão e ser uma ferramenta confiável na identificação de fadiga num utilizador. A combinação da frequência cardíaca e PERCLOS permite que o sistema tenha maior precisão, além de não depender completamente de um unico sensor. A criação de um modelo de previsão de fadiga foi dificultada pela falta de dados existentes para treinar um modelo, problema que poderia ser colmatado com a adoção do sistema numa população maior

Automated Video and Audio based Stress Detection using Deep Learning Techniques

In today's world, stress has become an undoubtedly severe problem that affects people's health. Stress can modify a person's behavior, ideas, and feelings in addition to having an impact on mental health. Unchecked stress can contribute to chronic illnesses including high blood pressure, diabetes, and obesity. Early stress detection promotes a healthy lifestyle in society. This work demonstrates a deep learning-based method for identifying stress from facial expressions and speech signals.An image dataset formed by collecting images from the web is used to construct and train the model Convolution Neural Network (CNN), which then divides the images into two categories: stressed and normal. Recurrent Neural Network (RNN), which is used to categorize speech signals into stressed and normal categories based on the features extracted by the MFCC (Mel Frequency Cepstral Coefficient), is thought to perform better on sequential data since it maintains the past results to determine the final output

Developing an Autonomous Mobile Robotic Device for Monitoring and Assisting Older People

A progressive increase of the elderly population in the world has required technological solutions capable of improving the life prospects of people suffering from senile dementias such as Alzheimer's. Socially Assistive Robotics (SAR) in the research field of elderly care is a solution that can ensure, through observation and monitoring of behaviors, their safety and improve their physical and cognitive health. A social robot can autonomously and tirelessly monitor a person daily by providing assistive tasks such as remembering to take medication and suggesting activities to keep the assisted active both physically and cognitively. However, many projects in this area have not considered the preferences, needs, personality, and cognitive profiles of older people. Moreover, other projects have developed specific robotic applications making it difficult to reuse and adapt them on other hardware devices and for other different functional contexts. This thesis presents the development of a scalable, modular, multi-tenant robotic application and its testing in real-world environments. This work is part of the UPA4SAR project ``User-centered Profiling and Adaptation for Socially Assistive Robotics''. The UPA4SAR project aimed to develop a low-cost robotic application for faster deployment among the elderly population. The architecture of the proposed robotic system is modular, robust, and scalable due to the development of functionality in microservices with event-based communication. To improve robot acceptance the functionalities, enjoyed through microservices, adapt the robot's behaviors based on the preferences and personality of the assisted person. A key part of the assistance is the monitoring of activities that are recognized through deep neural network models proposed in this work. The final experimentation of the project carried out in the homes of elderly volunteers was performed with complete autonomy of the robotic system. Daily care plans customized to the person's needs and preferences were executed. These included notification tasks to remember when to take medication, tasks to check if basic nutrition activities were accomplished, entertainment and companionship tasks with games, videos, music for cognitive and physical stimulation of the patient

End-to-End Multiview Gesture Recognition for Autonomous Car Parking System

The use of hand gestures can be the most intuitive human-machine interaction medium. The early approaches for hand gesture recognition used device-based methods. These methods use mechanical or optical sensors attached to a glove or markers, which hinders the natural human-machine communication. On the other hand, vision-based methods are not restrictive and allow for a more spontaneous communication without the need of an intermediary between human and machine. Therefore, vision gesture recognition has been a popular area of research for the past thirty years. Hand gesture recognition finds its application in many areas, particularly the automotive industry where advanced automotive human-machine interface (HMI) designers are using gesture recognition to improve driver and vehicle safety. However, technology advances go beyond active/passive safety and into convenience and comfort. In this context, one of America’s big three automakers has partnered with the Centre of Pattern Analysis and Machine Intelligence (CPAMI) at the University of Waterloo to investigate expanding their product segment through machine learning to provide an increased driver convenience and comfort with the particular application of hand gesture recognition for autonomous car parking. In this thesis, we leverage the state-of-the-art deep learning and optimization techniques to develop a vision-based multiview dynamic hand gesture recognizer for self-parking system. We propose a 3DCNN gesture model architecture that we train on a publicly available hand gesture database. We apply transfer learning methods to fine-tune the pre-trained gesture model on a custom-made data, which significantly improved the proposed system performance in real world environment. We adapt the architecture of the end-to-end solution to expand the state of the art video classifier from a single image as input (fed by monocular camera) to a multiview 360 feed, offered by a six cameras module. Finally, we optimize the proposed solution to work on a limited resources embedded platform (Nvidia Jetson TX2) that is used by automakers for vehicle-based features, without sacrificing the accuracy robustness and real time functionality of the system

A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms

In this paper a review is presented of the research on eye gaze estimation techniques and applications, that has progressed in diverse ways over the past two decades. Several generic eye gaze use-cases are identified: desktop, TV, head-mounted, automotive and handheld devices. Analysis of the literature leads to the identification of several platform specific factors that influence gaze tracking accuracy. A key outcome from this review is the realization of a need to develop standardized methodologies for performance evaluation of gaze tracking systems and achieve consistency in their specification and comparative evaluation. To address this need, the concept of a methodological framework for practical evaluation of different gaze tracking systems is proposed.Comment: 25 pages, 13 figures, Accepted for publication in IEEE Access in July 201

Beyond mobile apps: a survey of technologies for mental well-being

Mental health problems are on the rise globally and strain national health systems worldwide. Mental disorders are closely associated with fear of stigma, structural barriers such as financial burden, and lack of available services and resources which often prohibit the delivery of frequent clinical advice and monitoring. Technologies for mental well-being exhibit a range of attractive properties, which facilitate the delivery of state-of-the-art clinical monitoring. This review article provides an overview of traditional techniques followed by their technological alternatives, sensing devices, behaviour changing tools, and feedback interfaces. The challenges presented by these technologies are then discussed with data collection, privacy, and battery life being some of the key issues which need to be carefully considered for the successful deployment of mental health toolkits. Finally, the opportunities this growing research area presents are discussed including the use of portable tangible interfaces combining sensing and feedback technologies. Capitalising on the data these ubiquitous devices can record, state of the art machine learning algorithms can lead to the development of robust clinical decision support tools towards diagnosis and improvement of mental well-being delivery in real-time

Addressing the Data Acquisition Paradigm in the Early Detection of Pediatric Foot Deformities

The analysis of plantar pressure through podometry has allowed analyzing and detecting different types of disorders and treatments in child patients. Early detection of an inadequate distribution of the patient’s weight can prevent serious injuries to the knees and lower spine. In this paper, an embedded system capable of detecting the presence of normal, flat, or arched footprints using resistive pressure sensors was proposed. For this purpose, both hardware- and software-related criteria were studied for an improved data acquisition through signal coupling and filtering processes. Subsequently, learning algorithms allowed us to estimate the type of footprint biomechanics in preschool and school children volunteers. As a result, the proposed algorithm achieved an overall classification accuracy of 97.2%. A flat feet share of 60% was encountered in a sample of 1000 preschool children. Similarly, flat feet were observed in 52% of a sample of 600 school children