729 research outputs found
Eyewear Computing \u2013 Augmenting the Human with Head-Mounted Wearable Assistants
The seminar was composed of workshops and tutorials on head-mounted eye tracking, egocentric
vision, optics, and head-mounted displays. The seminar welcomed 30 academic and industry
researchers from Europe, the US, and Asia with a diverse background, including wearable and
ubiquitous computing, computer vision, developmental psychology, optics, and human-computer
interaction. In contrast to several previous Dagstuhl seminars, we used an ignite talk format to
reduce the time of talks to one half-day and to leave the rest of the week for hands-on sessions,
group work, general discussions, and socialising. The key results of this seminar are 1) the
identification of key research challenges and summaries of breakout groups on multimodal eyewear
computing, egocentric vision, security and privacy issues, skill augmentation and task guidance,
eyewear computing for gaming, as well as prototyping of VR applications, 2) a list of datasets and
research tools for eyewear computing, 3) three small-scale datasets recorded during the seminar, 4)
an article in ACM Interactions entitled \u201cEyewear Computers for Human-Computer Interaction\u201d,
as well as 5) two follow-up workshops on \u201cEgocentric Perception, Interaction, and Computing\u201d
at the European Conference on Computer Vision (ECCV) as well as \u201cEyewear Computing\u201d at
the ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp)
PrivacEye: Privacy-Preserving Head-Mounted Eye Tracking Using Egocentric Scene Image and Eye Movement Features
Eyewear devices, such as augmented reality displays, increasingly integrate
eye tracking but the first-person camera required to map a user's gaze to the
visual scene can pose a significant threat to user and bystander privacy. We
present PrivacEye, a method to detect privacy-sensitive everyday situations and
automatically enable and disable the eye tracker's first-person camera using a
mechanical shutter. To close the shutter in privacy-sensitive situations, the
method uses a deep representation of the first-person video combined with rich
features that encode users' eye movements. To open the shutter without visual
input, PrivacEye detects changes in users' eye movements alone to gauge changes
in the "privacy level" of the current situation. We evaluate our method on a
first-person video dataset recorded in daily life situations of 17
participants, annotated by themselves for privacy sensitivity, and show that
our method is effective in preserving privacy in this challenging setting.Comment: 10 pages, 6 figures, supplementary materia
Computer Vision Algorithms for Mobile Camera Applications
Wearable and mobile sensors have found widespread use in recent years due to their ever-decreasing cost, ease of deployment and use, and ability to provide continuous monitoring as opposed to sensors installed at fixed locations. Since many smart phones are now equipped with a variety of sensors, including accelerometer, gyroscope, magnetometer, microphone and camera, it has become more feasible to develop algorithms for activity monitoring, guidance and navigation of unmanned vehicles, autonomous driving and driver assistance, by using data from one or more of these sensors. In this thesis, we focus on multiple mobile camera applications, and present lightweight algorithms suitable for embedded mobile platforms. The mobile camera scenarios presented in the thesis are: (i) activity detection and step counting from wearable cameras, (ii) door detection for indoor navigation of unmanned vehicles, and (iii) traffic sign detection from vehicle-mounted cameras.
First, we present a fall detection and activity classification system developed for embedded smart camera platform CITRIC. In our system, the camera platform is worn by the subject, as opposed to static sensors installed at fixed locations in certain rooms, and, therefore, monitoring is not limited to confined areas, and extends to wherever the subject may travel including indoors and outdoors. Next, we present a real-time smart phone-based fall detection system, wherein we implement camera and accelerometer based fall-detection on Samsung Galaxy S™ 4. We fuse these two sensor modalities to have a more robust fall detection system. Then, we introduce a fall detection algorithm with autonomous thresholding using relative-entropy within the class of Ali-Silvey distance measures. As another wearable camera application, we present a footstep counting algorithm using a smart phone camera. This algorithm provides more accurate step-count compared to using only accelerometer data in smart phones and smart watches at various body locations.
As a second mobile camera scenario, we study autonomous indoor navigation of unmanned vehicles. A novel approach is proposed to autonomously detect and verify doorway openings by using the Google Project Tango™ platform.
The third mobile camera scenario involves vehicle-mounted cameras. More specifically, we focus on traffic sign detection from lower-resolution and noisy videos captured from vehicle-mounted cameras. We present a new method for accurate traffic sign detection, incorporating Aggregate Channel Features and Chain Code Histograms, with the goal of providing much faster training and testing, and comparable or better performance, with respect to deep neural network approaches, without requiring specialized processors. Proposed computer vision algorithms provide promising results for various useful applications despite the limited energy and processing capabilities of mobile devices
XAIR: A Framework of Explainable AI in Augmented Reality
Explainable AI (XAI) has established itself as an important component of
AI-driven interactive systems. With Augmented Reality (AR) becoming more
integrated in daily lives, the role of XAI also becomes essential in AR because
end-users will frequently interact with intelligent services. However, it is
unclear how to design effective XAI experiences for AR. We propose XAIR, a
design framework that addresses "when", "what", and "how" to provide
explanations of AI output in AR. The framework was based on a
multi-disciplinary literature review of XAI and HCI research, a large-scale
survey probing 500+ end-users' preferences for AR-based explanations, and three
workshops with 12 experts collecting their insights about XAI design in AR.
XAIR's utility and effectiveness was verified via a study with 10 designers and
another study with 12 end-users. XAIR can provide guidelines for designers,
inspiring them to identify new design opportunities and achieve effective XAI
designs in AR.Comment: Proceedings of the 2023 CHI Conference on Human Factors in Computing
System
Forecasting User Attention During Everyday Mobile Interactions Using Device-Integrated and Wearable Sensors
Visual attention is highly fragmented during mobile interactions, but the
erratic nature of attention shifts currently limits attentive user interfaces
to adapting after the fact, i.e. after shifts have already happened. We instead
study attention forecasting -- the challenging task of predicting users' gaze
behaviour (overt visual attention) in the near future. We present a novel
long-term dataset of everyday mobile phone interactions, continuously recorded
from 20 participants engaged in common activities on a university campus over
4.5 hours each (more than 90 hours in total). We propose a proof-of-concept
method that uses device-integrated sensors and body-worn cameras to encode rich
information on device usage and users' visual scene. We demonstrate that our
method can forecast bidirectional attention shifts and predict whether the
primary attentional focus is on the handheld mobile device. We study the impact
of different feature sets on performance and discuss the significant potential
but also remaining challenges of forecasting user attention during mobile
interactions.Comment: 13 pages, 9 figure
- …