Wearable Eye Tracking for Multisensor Physical Activity Recognition

This paper explores the use of wearable eye-tracking to detect physical activities and location information during assembly and construction tasks involving small groups of up to four people. Large physical activities, like carrying heavy items and walking, are analysed alongside more precise, hand-tool activities, like using a drill, or a screwdriver. In a first analysis, gazeinvariant features from the eye-tracker are classified (using Naive Bayes) alongside features obtained from wrist-worn accelerometers and microphones. An evaluation is presented using data from an 8-person dataset containing over 600 physical activity events, performed under real-world (noisy) conditions. Despite the challenges of working with complex, and sometimes unreliable, data we show that event-based precision and recall of 0.66 and 0.81 respectively can be achieved by combining all three sensing modalities (using experiment independent training, and temporal smoothing). In a further analysis, we apply state-ofthe-art computer vision methods like object recognition, scene recognition, and face detection, to generate features from the eye-trackers’ egocentric videos. Activity recognition trained on the output of an object recognition model (e.g., VGG16 trained on ImageNet) could predict Precise activities with an (overall average) f-measure of 0.45. Location of participants was similarly obtained using visual scene recognition, with average precision and recall of 0.58 and 0.56

Collecting complex activity data sets in highly rich networked sensor environments

We deployed 72 sensors of 10 modalities in 15 wireless and wired networked sensor systems in the environment, in objects, and on the body to create a sensor-rich environment for the machine recognition of human activities. We acquired data from 12 subjects performing morning activities, yielding over 25 hours of sensor data. We report the number of activity occurrences observed during post-processing, and estimate that over 13000 and 14000 object and environment interactions occurred. We describe the networked sensor setup and the methodology for data acquisition, synchronization and curation. We report on the challenges and outline lessons learned and best practice for similar large scale deployments of heterogeneous networked sensor systems. We evaluate data acquisition quality for on-body and object integrated wireless sensors; there is less than 2.5% packet loss after tuning. We outline our use of the dataset to develop new sensor network self-organization principles and machine learning techniques for activity recognition in opportunistic sensor configurations. Eventually this dataset will be made public

OPPORTUNITY: Towards opportunistic activity and context recognition systems

Opportunistic sensing allows to efficiently collect information about the physical world and the persons behaving in it. This may mainstream human context and activity recognition in wearable and pervasive computing by removing requirements for a specific deployed infrastructure. In this paper we introduce the newly started European research project OPPORTUNITY within which we develop mobile opportunistic activity and context recognition systems. We outline the project’s objective, the approach we follow along opportunistic sensing, data processing and interpretation, and autonomous adaptation and evolution to environmental and user changes, and we outline preliminary results

Magnetic Field-based Localization System

Diese Dissertation beschäftigt sich mit einem auf oszillierenden Magnetfeldern basierenden Lokalisierungssystem und der zugrunde liegenden Verarbeitungsstruktur. Das System besteht aus mehreren stationären Ankerpunkten (Sender), die oszillierende Magnetfelder erzeugen, und tragbaren Magnetfeldmesseinheiten (Empfänger), deren Positionen bestimmt werden sollen. Das System wird in verschiedenen Umgebungen und Anwendungsgebieten evaluiert. Desweiteren werden verschiedene Einsatzmöglichkeiten des Lokalisierungssystems in den Bereichen Ubiquitous und Pervasive Computing sowie Ambient Assisted Living diskutiert und bewertet. Zuletzt wird die Kombination von magnetfeldbasierten Abstandsinformationen und Positionsinformationen aus LIDAR Abstandsmessungen beschrieben und evaluiert. Die Systemarchitektur besteht aus drei Schichten: einer physikalischen Schicht, einer Schicht, die für die Positions- und Abstandsbestimmung zwischen einem Magnetfeldtransmitter und einem Empfänger verantwortlich ist und einer Schicht, die Abstands- und Positionsinformationen zu verschiedenen Ankerpunkten betrachtet, um daraus die absolute Position der tragbaren Messeinheit zu bestimmen. Jede der Schichten beleuchtet dabei verschiedene Aspekte, die bei der Verarbeitung der Magnetfeldinformationen beachtet werden müssen. Insbesondere die Eigenschaften der erzeugten Magnetfelder fließen in die Verarbeitungsalgorithmen ein. Die physikalische Schicht deckt dabei Magnetfelderzeugung, magnetfeldbasierte Informationsübertragung, Synchronisation von Magnetfeldtransmittern und Empfänger sowie die Abbildung des Verhaltens der Magnetfelder ab. Nachdem diese Information dann an einen zentralen Verarbeitungsrechner übertragen wurde, werden die hardwarespezfischen Signallevel auf das Niveau des theoretischen Magnetfeldmodelles gehoben, und dann mittels des physikalischen Modelles in Kandidatenpositionen und Abstands- informationen umgewandelt. Bedingt durch Magnetfeldsymmetrien können die Messdaten auf nur acht Punkte (ein Punkt pro Koordinatensystemoktant) reduziert werden. Die ermittelten Positionen haben einen durchschnittlichen Fehler von 108 cm, der ermittelte Abstand einen durchschnittlichen Fehler von 40 cm. Abschließend werden die Abstands- und Positionsdaten verschiedener Transmitterankerpunkte zusammengeführt. Hierbei spielen sowohl die zeitliche Synchronisation der Transmitter untereinander und die Reihenfolge der Auslösung der Transmitter als auch die abstands- und punktbasierten Lokalisierungs- und Trackingalgorithmen eine Rolle. Das Lokalisierungssystem wird in verschiedenen Anwendungen und Umgebungen evaluiert, die Position kann vom magnetfeldbasierten Lokalisierungssystem abhängig von der Umgebung mit einem durchschnittlichen Fehler von 60 cm - 70 cm ermittelt werden. Ein Vergleich mit einem funkbasierten Innenraumlokalisierungssystem zeigt die Robustheit des Magnetfeldes auch in Bereichen mit Funk-Abschattungen wie zum Beispiel unter großen metallischen Gegenständen. Wir zeigen Algorithmen zur Bereichserkennung (Regions of Interest, ROIs), die sowohl auf den Magnetfeldrohdaten als auch auf den transformierten Positions- und Abstandsinformationen arbeiten. Eingesetzt in größeren Räumen, können Bereiche unterschieden werden, die mindestens 50cm voneinander entfernt liegen, kleine Spulenaufbauten (3 Spulen in 2m^3) ermöglichen eine Auflösung von unter 20 cm. Abschließend zeigen wir die Kombination eines tragbaren, auf 4 LIDAR Abstandssensoren basierenden Lokalisierungssystems mit dem magnetfeldbasierten Lokalisierungssystem. Das Magnetfeldlokalisierungssystem stellt dabei Abstandsinformationen zur Verfügung, um mehrdeutige Sensorinformationen des LIDAR Systems zu unterscheiden. Hier ist in einem Raum mit 8m × 10m Fläche eine durchschnittlicher Positionsfehler von 8 cm zu erwarten.This dissertation describes an indoor localization system based on oscillating magnetic fields and the underlying processing architecture. The system consists of several fixed anchor points, generating the magnetic fields (transmitter), and wearable magnetic field measurement units, whose position should be determined (receiver). The system is evaluated in different environments and application areas. Additionally, various fields of application are discussed and assessed in ubiquitous and pervasive computing and Ambient Assisted Living. The fusion of magnetic field-based distance information and positions derived from LIDAR distance measurements is described and evaluated. The system architecture consists of three layers, a physical layer, a layer for position and distance estimation between a magnetic field transmitter and a receiver, and a layer which uses several measurements to different transmitters to estimate the overall position of a wearable measurement unit. Each layer covers different aspects which have to be taken care of when magnetic field information is processed. Especially the properties of the generated magnetic field information are considered in the processing algorithms. The physical layer covers the magnetic field generation and magnetic Field-Based information transfer, synchronization of a transmitter and the receivers and the description of the locally measured magnetic fields on the receiver side. After a transfer of this information to a central processing unit, the hardware specific signal levels are transformed to the levels of the theoretical magnetic field models. The values are then used to estimate candidate positions and distances. Due to symmetrical effects of the magnetic fields, it is only possible to reduce the receiver position to 8 points around the transmitter (one position in each of the octants of the coordinate system). The determined positions have a mean error of 108 cm, the average error of the distance is 40 cm. On top of this, the distance and position information against different transmitters are fused, this covers clock synchronization of transmitters, triggering and scheduling sequences and distance and position based localization and tracking algorithms. The magnetic-field-based indoor localization system has been evaluated in different applications and environments; the mean position error is 60 cm to 70 cm depending on the environment. A comparison against an RF-based indoor localization system shows the robustness of magnetic fields against RF shadows caused by big metal objects. We additionally present algorithms for regions of interest detection, working on raw magnetic field information and transformed position and distance information. Setups in larger areas can distinguish regions which are further than 50 cm apart, small scale coil setups (3 transmitters in 2m^3) allow to resolve regions below 20 cm. In the end, we describe a fusion algorithm for a wearable localization system based on 4 LIDAR distance measurement units and magnetic field-based distance estimation. The magnetic field indoor localization system provides distance proximity information which is used to resolve ambiguous position estimates of the LIDAR system. In a room (8m × 10m), we achieve a mean error of 8 cm

Magnetic Field-based Localization System

Diese Dissertation beschäftigt sich mit einem auf oszillierenden Magnetfeldern basierenden Lokalisierungssystem und der zugrunde liegenden Verarbeitungsstruktur. Das System besteht aus mehreren stationären Ankerpunkten (Sender), die oszillierende Magnetfelder erzeugen, und tragbaren Magnetfeldmesseinheiten (Empfänger), deren Positionen bestimmt werden sollen. Das System wird in verschiedenen Umgebungen und Anwendungsgebieten evaluiert. Desweiteren werden verschiedene Einsatzmöglichkeiten des Lokalisierungssystems in den Bereichen Ubiquitous und Pervasive Computing sowie Ambient Assisted Living diskutiert und bewertet. Zuletzt wird die Kombination von magnetfeldbasierten Abstandsinformationen und Positionsinformationen aus LIDAR Abstandsmessungen beschrieben und evaluiert. Die Systemarchitektur besteht aus drei Schichten: einer physikalischen Schicht, einer Schicht, die für die Positions- und Abstandsbestimmung zwischen einem Magnetfeldtransmitter und einem Empfänger verantwortlich ist und einer Schicht, die Abstands- und Positionsinformationen zu verschiedenen Ankerpunkten betrachtet, um daraus die absolute Position der tragbaren Messeinheit zu bestimmen. Jede der Schichten beleuchtet dabei verschiedene Aspekte, die bei der Verarbeitung der Magnetfeldinformationen beachtet werden müssen. Insbesondere die Eigenschaften der erzeugten Magnetfelder fließen in die Verarbeitungsalgorithmen ein. Die physikalische Schicht deckt dabei Magnetfelderzeugung, magnetfeldbasierte Informationsübertragung, Synchronisation von Magnetfeldtransmittern und Empfänger sowie die Abbildung des Verhaltens der Magnetfelder ab. Nachdem diese Information dann an einen zentralen Verarbeitungsrechner übertragen wurde, werden die hardwarespezfischen Signallevel auf das Niveau des theoretischen Magnetfeldmodelles gehoben, und dann mittels des physikalischen Modelles in Kandidatenpositionen und Abstands- informationen umgewandelt. Bedingt durch Magnetfeldsymmetrien können die Messdaten auf nur acht Punkte (ein Punkt pro Koordinatensystemoktant) reduziert werden. Die ermittelten Positionen haben einen durchschnittlichen Fehler von 108 cm, der ermittelte Abstand einen durchschnittlichen Fehler von 40 cm. Abschließend werden die Abstands- und Positionsdaten verschiedener Transmitterankerpunkte zusammengeführt. Hierbei spielen sowohl die zeitliche Synchronisation der Transmitter untereinander und die Reihenfolge der Auslösung der Transmitter als auch die abstands- und punktbasierten Lokalisierungs- und Trackingalgorithmen eine Rolle. Das Lokalisierungssystem wird in verschiedenen Anwendungen und Umgebungen evaluiert, die Position kann vom magnetfeldbasierten Lokalisierungssystem abhängig von der Umgebung mit einem durchschnittlichen Fehler von 60 cm - 70 cm ermittelt werden. Ein Vergleich mit einem funkbasierten Innenraumlokalisierungssystem zeigt die Robustheit des Magnetfeldes auch in Bereichen mit Funk-Abschattungen wie zum Beispiel unter großen metallischen Gegenständen. Wir zeigen Algorithmen zur Bereichserkennung (Regions of Interest, ROIs), die sowohl auf den Magnetfeldrohdaten als auch auf den transformierten Positions- und Abstandsinformationen arbeiten. Eingesetzt in größeren Räumen, können Bereiche unterschieden werden, die mindestens 50cm voneinander entfernt liegen, kleine Spulenaufbauten (3 Spulen in 2m^3) ermöglichen eine Auflösung von unter 20 cm. Abschließend zeigen wir die Kombination eines tragbaren, auf 4 LIDAR Abstandssensoren basierenden Lokalisierungssystems mit dem magnetfeldbasierten Lokalisierungssystem. Das Magnetfeldlokalisierungssystem stellt dabei Abstandsinformationen zur Verfügung, um mehrdeutige Sensorinformationen des LIDAR Systems zu unterscheiden. Hier ist in einem Raum mit 8m × 10m Fläche eine durchschnittlicher Positionsfehler von 8 cm zu erwarten.This dissertation describes an indoor localization system based on oscillating magnetic fields and the underlying processing architecture. The system consists of several fixed anchor points, generating the magnetic fields (transmitter), and wearable magnetic field measurement units, whose position should be determined (receiver). The system is evaluated in different environments and application areas. Additionally, various fields of application are discussed and assessed in ubiquitous and pervasive computing and Ambient Assisted Living. The fusion of magnetic field-based distance information and positions derived from LIDAR distance measurements is described and evaluated. The system architecture consists of three layers, a physical layer, a layer for position and distance estimation between a magnetic field transmitter and a receiver, and a layer which uses several measurements to different transmitters to estimate the overall position of a wearable measurement unit. Each layer covers different aspects which have to be taken care of when magnetic field information is processed. Especially the properties of the generated magnetic field information are considered in the processing algorithms. The physical layer covers the magnetic field generation and magnetic Field-Based information transfer, synchronization of a transmitter and the receivers and the description of the locally measured magnetic fields on the receiver side. After a transfer of this information to a central processing unit, the hardware specific signal levels are transformed to the levels of the theoretical magnetic field models. The values are then used to estimate candidate positions and distances. Due to symmetrical effects of the magnetic fields, it is only possible to reduce the receiver position to 8 points around the transmitter (one position in each of the octants of the coordinate system). The determined positions have a mean error of 108 cm, the average error of the distance is 40 cm. On top of this, the distance and position information against different transmitters are fused, this covers clock synchronization of transmitters, triggering and scheduling sequences and distance and position based localization and tracking algorithms. The magnetic-field-based indoor localization system has been evaluated in different applications and environments; the mean position error is 60 cm to 70 cm depending on the environment. A comparison against an RF-based indoor localization system shows the robustness of magnetic fields against RF shadows caused by big metal objects. We additionally present algorithms for regions of interest detection, working on raw magnetic field information and transformed position and distance information. Setups in larger areas can distinguish regions which are further than 50 cm apart, small scale coil setups (3 transmitters in 2m^3) allow to resolve regions below 20 cm. In the end, we describe a fusion algorithm for a wearable localization system based on 4 LIDAR distance measurement units and magnetic field-based distance estimation. The magnetic field indoor localization system provides distance proximity information which is used to resolve ambiguous position estimates of the LIDAR system. In a room (8m × 10m), we achieve a mean error of 8 cm

Adapting magnetic resonant coupling based relative positioning technology for wearable activitiy recogniton

We demonstrate how modulated magnetic field technology that is well established in high precision, stationary motion tracking systems can be adapted to wearable activity recog-nition. To this end we describe the design and implementa-tion of a cheap (components cost about 20 Euro for the trans-mitter and 15 Euro for the receiver), low power (17mA for the transmitter and 40mA for the receiver), and easily wear-able (the main size constraint are the coils which are about 25mm3) system for tracking the relative position and orienta-tion of body parts. We evaluate our system on two recognition tasks. On a set of 6 subtle nutrition related gestures it achieves 99.25 % recognition rate compared to 94.1 % for a XSens in-ertial device ( operated calibrated, euler angle mode). On the recognition of 8 Tai Chi moves it reaches 94 % compared to 86 % of an accelerometer. Combining our sensor with the ac-celerometer leads to 100 % correct recognition (as compared to 90 % when combining the accelerometer with a gyro).

Making Cognitive Ergonomics in the Human–Computer Interaction of Manufacturing Execution Systems Assessable: Experimental and Validation Approaches to Closing Research Gaps

Cognitive ergonomics and the mental health of production workers have attracted increasing interest in industrial companies. However, there is still not much research available as it is regarding physical ergonomics and muscular load. This paper designs an experiment to analyze the cognitive ergonomics and mental stress of shop floor production workers interacting with different user interfaces of a Manufacturing Execution System (MES) that is adjustable for analyzing the influence of other assistive systems, too. This approach is going to be designed with the Design of Experiments (DoE) method. Therefore, the respective goals and factors are going to be determined. The environment will be the laboratories of the University of Applied Sciences Amberg-Weiden and its Campus for Digitalization in Amberg. In detail, there will be a sample assembly process from the automotive supplier industry for demonstration purposes. At this laboratory, the MES software from the European benchmark SAP is installed, and the respective standard Production Operator Desk is going to be used with slight adaptions. In order to make the cognitive ergonomics measurable, different approaches are going to be used. For instance, body temperature, heart rate and skin conductance as well as subjective methods of self-assessment are planned. The result of this paper is a ready-to-run experiment with sample data for each classification of participants. Further, possible limitations and adjustments are going to be discussed. Finally, an approach to validating the expected results is going to be shown and future intentions are going to be discussed

LifeNet: an Ad-hoc Sensor Network and Wearable System to Provide Firefighters with Navigation Support

Our everyday life is surrounded by pervasive services such as those offered by printers, public displays, multimedia systems, workstations, etc. While on the run we should be able to use the mobile devices we carry along to interact with these services. Yet, a number of studies have shown that it is rather complicated for users to identify available services in an unknown context. The RelateGateways project extends the mobile desktop with a new kind of widget: the gateways. These components are arranged around the edge of the screen, pointing towards the co-located services. Once identified, a service can be consumed in a consistent manner by dragging-and-dropping an object on the gateway, without the need for the user to install, configure or learn how to use yet another system for each service

Smart-watch life saver: smart-watch interactive-feedback system for improving bystander CPR

In this work a Smart-Watch application, that is able to monitor the frequency and depth of Cardiopulmonary Resuscitation (CPR) and provide interactive corrective feedback is described. We have evaluated the system with a total of 41 subjects who had undertaken a single episode of CPR training several years previously. This training was part of a First Aid course for lay people, commonly accessed in this population. The evaluation was conducted by measuring participant CPR competence using the "gold standard" of CPR training [10], namely frequency and compression depth. The evaluation demonstrated that the Smart Watch feedback system provided a significant improvement in the participant performance. For example, it doubled the number of people who could maintain bot the parameters in the recommended range for at least 50% of the time