49 research outputs found
Bullying10K: A Large-Scale Neuromorphic Dataset towards Privacy-Preserving Bullying Recognition
The prevalence of violence in daily life poses significant threats to
individuals' physical and mental well-being. Using surveillance cameras in
public spaces has proven effective in proactively deterring and preventing such
incidents. However, concerns regarding privacy invasion have emerged due to
their widespread deployment. To address the problem, we leverage Dynamic Vision
Sensors (DVS) cameras to detect violent incidents and preserve privacy since it
captures pixel brightness variations instead of static imagery. We introduce
the Bullying10K dataset, encompassing various actions, complex movements, and
occlusions from real-life scenarios. It provides three benchmarks for
evaluating different tasks: action recognition, temporal action localization,
and pose estimation. With 10,000 event segments, totaling 12 billion events and
255 GB of data, Bullying10K contributes significantly by balancing violence
detection and personal privacy persevering. And it also poses a challenge to
the neuromorphic dataset. It will serve as a valuable resource for training and
developing privacy-protecting video systems. The Bullying10K opens new
possibilities for innovative approaches in these domains.Comment: Accepted at the 37th Conference on Neural Information Processing
Systems (NeurIPS 2023) Track on Datasets and Benchmark
Signal enhancement and efficient DTW-based comparison for wearable gait recognition
The popularity of biometrics-based user identification has significantly increased over the last few years. User identification based on the face, fingerprints, and iris, usually achieves very high accuracy only in controlled setups and can be vulnerable to presentation attacks, spoofing, and forgeries. To overcome these issues, this work proposes a novel strategy based on a relatively less explored biometric trait, i.e., gait, collected by a smartphone accelerometer, which can be more robust to the attacks mentioned above. According to the wearable sensor-based gait recognition state-of-the-art, two main classes of approaches exist: 1) those based on machine and deep learning; 2) those exploiting hand-crafted features. While the former approaches can reach a higher accuracy, they suffer from problems like, e.g., performing poorly outside the training data, i.e., lack of generalizability. This paper proposes an algorithm based on hand-crafted features for gait recognition that can outperform the existing machine and deep learning approaches. It leverages a modified Majority Voting scheme applied to Fast Window Dynamic Time Warping, a modified version of the Dynamic Time Warping (DTW) algorithm with relaxed constraints and majority voting, to recognize gait patterns. We tested our approach named MV-FWDTW on the ZJU-gaitacc, one of the most extensive datasets for the number of subjects, but especially for the number of walks per subject and walk lengths. Results set a new state-of-the-art gait recognition rate of 98.82% in a cross-session experimental setup. We also confirm the quality of the proposed method using a subset of the OU-ISIR dataset, another large state-of-the-art benchmark with more subjects but much shorter walk signals
Detecting User’s Behavior Shift with Sensorized Shoes and Stigmergic Perceptrons
As populations become increasingly aged, health monitoring has gained increasing importance. Recent advances in engineering of sensing, processing and artificial learning, make the development of non-invasive systems able to observe changes over time possible. In this context, the Ki-Foot project aims at developing a sensorized shoe and a machine learning architecture based on computational stigmergy to detect small variations in subjects gait and to learn and detect users behaviour shift. This paper outlines the challenges in the field and summarizes the proposed approach. The machine learning architecture has been developed and publicly released after early experimentation, in order to foster its application on real environments
A spatiotemporal deep learning approach for automatic pathological Gait classification
Human motion analysis provides useful information for the diagnosis and recovery assessment of people suffering from pathologies, such as those affecting the way of walking, i.e., gait. With recent developments in deep learning, state-of-the-art performance can now be achieved using a single 2D-RGB-camera-based gait analysis system, offering an objective assessment of gait-related pathologies. Such systems provide a valuable complement/alternative to the current standard practice of subjective assessment. Most 2D-RGB-camera-based gait analysis approaches rely on compact gait representations, such as the gait energy image, which summarize the characteristics of a walking sequence into one single image. However, such compact representations do not
fully capture the temporal information and dependencies between successive gait movements. This limitation is addressed by proposing a spatiotemporal deep learning approach that uses a selection of key frames to represent a gait cycle. Convolutional and recurrent deep neural networks were combined, processing each gait cycle as a collection of silhouette key frames, allowing the system to learn temporal patterns among the spatial features extracted at individual time instants. Trained with gait sequences from the GAIT-IT dataset, the proposed system is able to improve gait pathology classification accuracy, outperforming state-of-the-art solutions and achieving improved generalization on cross-dataset tests.info:eu-repo/semantics/publishedVersio
Runner re-identification from single-view video in the open-world setting
In many sports, player re-identification is crucial for automatic video
processing and analysis. However, most of the current studies on player
re-identification in multi- or single-view sports videos focus on
re-identification in the closed-world setting using labeled image dataset, and
player re-identification in the open-world setting for automatic video analysis
is not well developed. In this paper, we propose a runner re-identification
system that directly processes single-view video to address the open-world
setting. In the open-world setting, we cannot use labeled dataset and have to
process video directly. The proposed system automatically processes raw video
as input to identify runners, and it can identify runners even when they are
framed out multiple times. For the automatic processing, we first detect the
runners in the video using the pre-trained YOLOv8 and the fine-tuned
EfficientNet. We then track the runners using ByteTrack and detect their shoes
with the fine-tuned YOLOv8. Finally, we extract the image features of the
runners using an unsupervised method using the gated recurrent unit autoencoder
model. To improve the accuracy of runner re-identification, we use dynamic
features of running sequence images. We evaluated the system on a running
practice video dataset and showed that the proposed method identified runners
with higher accuracy than one of the state-of-the-art models in unsupervised
re-identification. We also showed that our unsupervised running dynamic feature
extractor was effective for runner re-identification. Our runner
re-identification system can be useful for the automatic analysis of running
videos.Comment: 18 pages, 8 figure
A review on visual privacy preservation techniques for active and assisted living
This paper reviews the state of the art in visual privacy protection techniques, with particular attention paid to techniques applicable to the field of Active and Assisted Living (AAL). A novel taxonomy with which state-of-the-art visual privacy protection methods can be classified is introduced. Perceptual obfuscation methods, a category in this taxonomy, is highlighted. These are a category of visual privacy preservation techniques, particularly relevant when considering scenarios that come under video-based AAL monitoring. Obfuscation against machine learning models is also explored. A high-level classification scheme of privacy by design, as defined by experts in privacy and data protection law, is connected to the proposed taxonomy of visual privacy preservation techniques. Finally, we note open questions that exist in the field and introduce the reader to some exciting avenues for future research in the area of visual privacy.Open Access funding provided thanks to the CRUE-CSIC agreement with Springer Nature. This work is part of the visuAAL project on Privacy-Aware and Acceptable Video-Based Technologies and Services for Active and Assisted Living (https://www.visuaal-itn.eu/). This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 861091. The authors would also like to acknowledge the contribution of COST Action CA19121 - GoodBrother, Network on Privacy-Aware Audio- and Video-Based Applications for Active and Assisted Living (https://goodbrother.eu/), supported by COST (European Cooperation in Science and Technology) (https://www.cost.eu/)
Monitoring Indoor People Presence in Buildings Using Low-Cost Infrared Sensor Array in Doorways
We propose a device for monitoring the number of people who are physically present
inside indoor environments. The device performs local processing of infrared array sensor data
detecting people’s direction, which allows monitoring users’ occupancy in any space of the building
and also respects people privacy. The device implements a novel real-time pattern recognition
algorithm for processing data sensed by a low-cost infrared (IR) array sensor. The computed
information is transferred through a Z-Wave network. On-field evaluation of the algorithm has been
conducted by placing the device on top of doorways in offices and laboratory rooms. To evaluate
the performance of the algorithm in varying ambient temperatures, two groups of stress tests have
been designed and performed. These tests established the detection limits linked to the difference
between the average ambient temperature and perturbation. For an in-depth analysis of the
accuracy of the algorithm, synthetic data have been generated considering temperature ranges
typical of a residential environment, different human walking speeds (normal, brisk, running), and
distance between the person and the sensor (1.5 m, 5 m, 7.5 m). The algorithm performed with high
accuracy for routine human passage detection through a doorway, considering indoor ambient
conditions of 21–30 °C