HMM-based activity recognition with a ceiling RGB-D camera

Automated recognition of Activities of Daily Living allows to identify possible health problems and apply corrective strategies in Ambient Assisted Living (AAL). Activities of Daily Living analysis can provide very useful information for elder care and long-term care services. This paper presents an automated RGB-D video analysis system that recognises human ADLs activities, related to classical daily actions. The main goal is to predict the probability of an analysed subject action. Thus, the abnormal behaviour can be detected. The activity detection and recognition is performed using an affordable RGB-D camera. Human activities, despite their unstructured nature, tend to have a natural hierarchical structure; for instance, generally making a coffee involves a three-step process of turning on the coffee machine, putting sugar in cup and opening the fridge for milk. Action sequence recognition is then handled using a discriminative Hidden Markov Model (HMM). RADiaL, a dataset with RGB-D images and 3D position of each person for training as well as evaluating the HMM, has been built and made publicly available

Human Behaviour Understanding using Top-View RGB-D Data

I moderni sistemi di visione sono in grado di individuare in maniera automatica gli spostamenti delle persone in modo da poterne comprendere i comportamenti. Questa tesi si focalizza sullo sviluppo di algoritmi di visione e di modelli matematici basati su dati provenienti da sensori RGB-D posti in modalità top-view. Dopo uno studio approfondito sullo stato dell'arte, verranno presentate due tipologie di approcci per l'individuazione delle persone all'interno delle immagini di profondità. Esse sfruttano algoritmi che si basano su tecniche di image processing e su recenti metodi di deep learning. Ulteriori informazioni presenti nell'immagine a colori verranno usate per identificare i soggetti anche in un secondo momento e per individuare eventuali interazioni che questi hanno con l'ambiente circostante. Infine, gli algoritmi verranno testati in diversi casi d'uso reali al fine di valutarne le prestazioni.The capability of automatically detecting people and understanding their behaviours is an important functionality of intelligent video systems. The interest in behaviour understanding has effectively increased in recent years, motivated by a societal needs. This thesis is focused on the development of algorithms and solutions for different environments exploiting top-view RGB-D data. In particular, the addressed topics refer to HBU in different research areas. The first goal is to implement people detection algorithms in order to monitor the people activities. To this aim, a thorough study of the state of the art has been conducted to identify the advantages and weakness. An initial approach, proposed in this thesis, is based on CV techniques, it regards the extraction the head of each person using depth data. Another approach is based on deep learning and is proposed to simplify the heads detection implementation in chaotic environments and in the presence of people with different heights. These solutions are validated with a specific dataset. The second goal is to extract several feature from subject and to identify possible interactions that they have with the surrounding environment. Finally, in order to demonstrate the actual contribution of algorithms for understanding the human behaviour in different environments, several use cases have been realized and tested

Person re-identification dataset with RGB-D camera in a top-view configuration

Video analytics, involves a variety of techniques to monitor, analyse, and extract meaningful information from video streams. In this light, person re-identification is an important topic in scene monitoring, human computer interaction, retail, people counting, ambient assisted living and many other computer vision research. The existing datasets are not suitable for activity monitoring and human behaviour analysis. For this reason we build a novel dataset for person re-identification that uses an RGB-D camera in a top-view configuration. This setup choice is primarily due to the reduction of occlusions and it has also the advantage of being privacy preserving, because faces are not recorded by the camera. The use of an RGB-D camera allows to extract anthropometric features for the recognition of people passing under the camera. The paper describes in details the collection and construction modalities of the dataset TVPR. This is composed by 100 people and for each video frame nine depth and colour features are computed and provided together with key descriptive statistics

Convolutional Networks for Semantic Heads Segmentation using Top-View Depth Data in Crowded Environment

Detecting and tracking people is a challenging task in a persistent crowded environment (i.e. retail, airport, station, etc.) for human behaviour analysis of security purposes. This paper introduces an approach to track and detect people in cases of heavy occlusions based on CNNs for semantic segmentation using top-view depth visual data. The purpose is the design of a novel U-Net architecture, U-Net3, that has been modified compared to the previous ones at the end of each layer. In particular, a batch normalization is added after the first ReLU activation function and after each max-pooling and up-sampling functions. The approach was applied and tested on a new and public available dataset, TVHeads Dataset, consisting of depth images of people recorded from an RGB-D camera installed in top-view configuration. Our variant outperforms baseline architectures while remaining computationally efficient at inference time. Results show high accuracy, demonstrating the effectiveness and suitability of our approach

Design of an interoperable framework with domotic sensors network integration

Nowadays, home devices with network capabilities are widely used. The technology integration offers new and exciting opportunities to increase the device connectivity within a home for many proposals of home automation. In this paper, it has been developed a framework that allows to quickly develop new hardware and software complex systems, rapidly integrate new classes of devices in existing systems and control and centralize the data. The preliminary results obtained are already consistent and demonstrate its suitability and its effectiveness

Modelling and Forecasting Customer Navigation in Intelligent Retail Environments

Understanding shopper behaviour is one of the keys to success for retailers. In particular, it is necessary that managers know which retail attributes are important to which shoppers and their main goal is to improve the consumer shopping experience. In this work, we present sCREEN (Consumer REtail ExperieNce), an intelligent mechatronic system for indoor navigation assistance in retail environments that minimizes the need for active tagging and does not require metrics maps. The tracking system is based on Ultra-wideband technology. The digital devices are installed in the shopping carts and baskets and sCREEN allows modelling and forecasting customer navigation in retail environments. This paper contributes the design of an intelligent mechatronic system with the use of a novel Hidden Markov Models (HMMs) for the representation of shoppers\u2019 shelf/category attraction and usual retail scenarios such as product out of stock or changes on store layout. Observations are viewed as a perceived intelligent system performance. By forecasting consumers next shelf/category attraction, the system can present the item location information to the consumer, including a walking route map to a location of the product in the retail store, and/or the number of an aisle in which the product is located. Effective and efficient design processes for mechatronic systems are a prerequisite for competitiveness in an intelligent retail environment. Experiments are performed in a real retail environment that is a German supermarket, during business hours. A dataset, with consumers trajectories, timestamps and the corresponding ground truth for training as well as evaluating the HMM, have been built and made publicly available. The results in terms of Precision, Recall and F1-score demonstrate the effectiveness and suitability of our approach, with a precision value that exceeds the 76% in all test cases