An end-to-end review of gaze estimation and its interactive applications on handheld mobile devices

In recent years we have witnessed an increasing number of interactive systems on handheld mobile devices which utilise gaze as a single or complementary interaction modality. This trend is driven by the enhanced computational power of these devices, higher resolution and capacity of their cameras, and improved gaze estimation accuracy obtained from advanced machine learning techniques, especially in deep learning. As the literature is fast progressing, there is a pressing need to review the state of the art, delineate the boundary, and identify the key research challenges and opportunities in gaze estimation and interaction. This paper aims to serve this purpose by presenting an end-to-end holistic view in this area, from gaze capturing sensors, to gaze estimation workflows, to deep learning techniques, and to gaze interactive applications.PostprintPeer reviewe

AI Modeling Approaches for Detecting, Characterizing, and Predicting Brief Daily Behaviors such as Toothbrushing using Wrist Trackers.

Continuous advancements in wrist-worn sensors have opened up exciting possibilities for real-time monitoring of individuals\u27 daily behaviors, with the aim of promoting healthier, more organized, and efficient lives. Understanding the duration of specific daily behaviors has become of interest to individuals seeking to optimize their lifestyles. However, there is still a research gap when it comes to monitoring short-duration behaviors that have a significant impact on health using wrist-worn inertial sensors in natural environments. These behaviors often involve repetitive micro-events that last only a few seconds or even microseconds, making their detection and analysis challenging. Furthermore, these micro-events are often surrounded by non-repetitive boundary events, further complicating the identification process. Effective detection and timely intervention during these short-duration behaviors are crucial for designing personalized interventions that can positively impact individuals\u27 lifestyles. To address these challenges, this dissertation introduces three models: mORAL, mTeeth, and Brushing Prompt. These models leverage wrist-worn inertial sensors to accurately infer short-duration behaviors, identify repetitive micro-behaviors, and provide timely interventions related to oral hygiene. The dissertation\u27s contributions extend beyond the development of these models. Firstly, precise and detailed labels for each brief and micro-repetitive behavior are acquired to train and validate the models effectively. This involved meticulous marking of the exact start and end times of each event, including any intervening pauses, at a second-level granularity. A comprehensive scientific research study was conducted to collect such data from participants in their free-living natural environments. Secondly, a solution is proposed to address the issue of sensor placement variability. Given the different positions of the sensor within a wristband and variations in wristband placement on the wrist, the model needs to determine the relative configuration of the inertial sensor accurately. Accurately determining the relative positioning of the inertial sensor with respect to the wrist is crucial for the model to determine the orientation of the hand. Additionally, time synchronization errors between sensor data and associated video, despite both being collected on the same smartphone, are addressed through the development of an algorithm that tightly synchronizes the two data sources without relying on an explicit anchor event. Furthermore, an event-based approach is introduced to identify candidate segments of data for applying machine learning models, outperforming the traditional fixed window-based approach. These candidate segments enable reliable detection of brief daily behaviors in a computationally efficient manner suitable for real-time. The dissertation also presents a computationally lightweight method for identifying anchor events using wrist-worn inertial sensors. Anchor events play a vital role in assigning unambiguous labels in a fixed-length window-based approach to data segmentation and effectively demarcating transitions between micro-repetitive events. Significant features are extracted, and explainable machine learning models are developed to ensure reliable detection of brief daily and micro-repetitive behaviors. Lastly, the dissertation addresses the crucial factor of the opportune moment for intervention during brief daily behaviors using wrist-worn inertial sensors. By leveraging these sensors, users can receive timely and personalized interventions to enhance their performance and improve their lifestyles. Overall, this dissertation makes substantial contributions to the field of real-time monitoring of short-duration behaviors. It tackles various technical challenges, provides innovative solutions, and demonstrates the potential for wrist-worn sensors to facilitate effective interventions and promote healthier behaviors. By advancing our understanding of these behaviors and optimizing intervention strategies, this research has the potential to significantly impact individuals\u27 well-being and contribute to the development of personalized health solutions

Privacy-Protecting Techniques for Behavioral Data: A Survey

Our behavior (the way we talk, walk, or think) is unique and can be used as a biometric trait. It also correlates with sensitive attributes like emotions. Hence, techniques to protect individuals privacy against unwanted inferences are required. To consolidate knowledge in this area, we systematically reviewed applicable anonymization techniques. We taxonomize and compare existing solutions regarding privacy goals, conceptual operation, advantages, and limitations. Our analysis shows that some behavioral traits (e.g., voice) have received much attention, while others (e.g., eye-gaze, brainwaves) are mostly neglected. We also find that the evaluation methodology of behavioral anonymization techniques can be further improved

NASA space station automation: AI-based technology review

Research and Development projects in automation for the Space Station are discussed. Artificial Intelligence (AI) based automation technologies are planned to enhance crew safety through reduced need for EVA, increase crew productivity through the reduction of routine operations, increase space station autonomy, and augment space station capability through the use of teleoperation and robotics. AI technology will also be developed for the servicing of satellites at the Space Station, system monitoring and diagnosis, space manufacturing, and the assembly of large space structures

Design Principles of Mobile Information Systems in the Digital Transformation of the Workplace - Utilization of Smartwatch-based Information Systems in the Corporate Context

During the last decades, smartwatches emerged as an innovative and promising technology and hit the consumer market due to the accessibility of affordable devices and predominant acceptance caused by the considerable similarity to common wristwatches. With the unique characteristics of permanent availability, unobtrusiveness, and hands-free operation, they can provide additional value in the corporate context. Thus, this thesis analyzes use cases for smartwatches in companies, elaborates on the design of smartwatch-based information systems, and covers the usability of smartwatch applications during the development of smartwatch-based information systems. It is composed of three research complexes. The first research complex focuses on the digital assistance of (mobile) employees who have to execute manual work and have been excluded so far from the benefits of the digitalization since they cannot operate hand-held devices. The objective is to design smartwatch-based information systems to support workflows in the corporate context, facilitate the daily work of numerous employees, and make processes more efficient for companies. During a design science research approach, smartwatch-based software artifacts are designed and evaluated in use cases of production, support, security service, as well as logistics, and a nascent design theory is proposed to complement theory according to mobile information system research. The evaluation shows that, on the one hand, smartwatches have enormous potential to assist employees with a fast and ubiquitous exchange of information, instant notifications, collaboration, and workflow guidance while they can be operated incidentally during manual work. On the other hand, the design of smartwatch-based information systems is a crucial factor for successful long-term deployment in companies, and especially limitations according to the small form-factor, general conditions, acceptance of the employees, and legal regulations have to be addressed appropriately. The second research complex addresses smartwatch-based information systems at the office workplace. This broadens and complements the view on the utilization of smartwatches in the corporate context in addition to the mobile context described in the first research complex. Though smartwatches are devices constructed for mobile use, the utilization in low mobile or stationary scenarios also has benefits due they exhibit the characteristic of a wearable computer and are directly connected to the employee’s body. Various sensors can perceive employee-, environment- and therefore context-related information and demand the employees’ attention with proactive notifications that are accompanied by a vibration. Thus, a smartwatch-based and gamified information system for health promotion at the office workplace is designed and evaluated. Research complex three provides a closer look at the topic of usability concerning applications running on smartwatches since it is a crucial factor during the development cycle. As a supporting element for the studies within the first and second research complex, a framework for the usability analysis of smartwatch applications is developed. For research, this thesis contributes a systemization of the state-of-the-art of smartwatch utilization in the corporate context, enabling and inhibiting influence factors of the smartwatch adoption in companies, and design principles as well as a nascent design theory for smartwatch-based information systems to support mobile employees executing manual work. For practice, this thesis contributes possible use cases for smartwatches in companies, assistance in decision-making for the introduction of smartwatch-based information systems in the corporate context with the Smartwatch Applicability Framework, situated implementations of a smartwatch-based information system for typical use cases, design recommendations for smartwatch-based information systems, an implementation of a smartwatch-based information system for the support of mobile employees executing manual work, and a usability-framework for smartwatches to automatically access usability of existing applications providing suggestions for usability improvement

Behavioral Model For Live Detection of Apps Based Attack

Smartphones with the platforms of applications are gaining extensive attention and popularity. The enormous use of different applications has paved the way to numerous security threats. The threats are in the form of attacks such as permission control attacks, phishing attacks, spyware attacks, botnets, malware attacks, privacy leakage attacks. Moreover, other vulnerabilities include invalid authorization of apps, compromise on the confidentiality of data, invalid access control. In this paper, an application-based attack modeling and attack detection is proposed. Due to A novel attack vulnerability is identified based on the app execution on the smartphone. The attack modeling involves an end-user vulnerable application to initiate an attack. The vulnerable application is installed at the background end on the smartphone with hidden visibility from the end-user. Thereby, accessing the confidential information. The detection model involves the proposed technique of an Application-based Behavioral Model Analysis (ABMA) scheme to address the attack model. The model incorporates application-based comparative parameter analysis to perform the process of intrusion detection. The ABMA is estimated by using the parameters of power, battery level, and the data usage. Based on the source internet accessibility, the analysis is performed using three different configurations as, WiFi, mobile data, and the combination of the two. The simulation results verify and demonstrates the effectiveness of the proposed model

Proceedings of the 2nd IUI Workshop on Interacting with Smart Objects

These are the Proceedings of the 2nd IUI Workshop on Interacting with Smart Objects. Objects that we use in our everyday life are expanding their restricted interaction capabilities and provide functionalities that go far beyond their original functionality. They feature computing capabilities and are thus able to capture information, process and store it and interact with their environments, turning them into smart objects

Development of a simulation tool for measurements and analysis of simulated and real data to identify ADLs and behavioral trends through statistics techniques and ML algorithms

openCon una popolazione di anziani in crescita, il numero di soggetti a rischio di patologia è in rapido aumento. Molti gruppi di ricerca stanno studiando soluzioni pervasive per monitorare continuamente e discretamente i soggetti fragili nelle loro case, riducendo i costi sanitari e supportando la diagnosi medica. Comportamenti anomali durante l'esecuzione di attività di vita quotidiana (ADL) o variazioni sulle tendenze comportamentali sono di grande importanza.With a growing population of elderly people, the number of subjects at risk of pathology is rapidly increasing. Many research groups are studying pervasive solutions to continuously and unobtrusively monitor fragile subjects in their homes, reducing health-care costs and supporting the medical diagnosis. Anomalous behaviors while performing activities of daily living (ADLs) or variations on behavioral trends are of great importance. To measure ADLs a significant number of parameters need to be considering affecting the measurement such as sensors and environment characteristics or sensors disposition. To face the impossibility to study in the real context the best configuration of sensors able to minimize costs and maximize accuracy, simulation tools are being developed as powerful means. This thesis presents several contributions on this topic. In the following research work, a study of a measurement chain aimed to measure ADLs and represented by PIRs sensors and ML algorithm is conducted and a simulation tool in form of Web Application has been developed to generate datasets and to simulate how the measurement chain reacts varying the configuration of the sensors. Starting from eWare project results, the simulation tool has been thought to provide support for technicians, developers and installers being able to speed up analysis and monitoring times, to allow rapid identification of changes in behavioral trends, to guarantee system performance monitoring and to study the best configuration of the sensors network for a given environment. The UNIVPM Home Care Web App offers the chance to create ad hoc datasets related to ADLs and to conduct analysis thanks to statistical algorithms applied on data. To measure ADLs, machine learning algorithms have been implemented in the tool. Five different tasks have been identified. To test the validity of the developed instrument six case studies divided into two categories have been considered. To the first category belong those studies related to: 1) discover the best configuration of the sensors keeping environmental characteristics and user behavior as constants; 2) define the most performant ML algorithms. The second category aims to proof the stability of the algorithm implemented and its collapse condition by varying user habits. Noise perturbation on data has been applied to all case studies. Results show the validity of the generated datasets. By maximizing the sensors network is it possible to minimize the ML error to 0.8%. Due to cost is a key factor in this scenario, the fourth case studied considered has shown that minimizing the configuration of the sensors it is possible to reduce drastically the cost with a more than reasonable value for the ML error around 11.8%. Results in ADLs measurement can be considered more than satisfactory.INGEGNERIA INDUSTRIALEopenPirozzi, Michel

A review of smart homes in healthcare

The technology of Smart Homes (SH), as an instance of ambient assisted living technologies, is designed to assist the homes’ residents accomplishing their daily-living activities and thus having a better quality of life while preserving their privacy. A SH system is usually equipped with a collection of inter-related software and hardware components to monitor the living space by capturing the behaviour of the resident and understanding his activities. By doing so the system can inform about risky situations and take actions on behalf of the resident to his satisfaction. The present survey will address technologies and analysis methods and bring examples of the state of the art research studies in order to provide background for the research community. In particular, the survey will expose infrastructure technologies such as sensors and communication platforms along with artificial intelligence techniques used for modeling and recognizing activities. A brief overview of approaches used to develop Human–Computer interfaces for SH systems is given. The survey also highlights the challenges and research trends in this area