Skeleton-based human action and gesture recognition for human-robot collaboration

openThe continuous development of robotic and sensing technologies has led in recent years to an increased interest in human-robot collaborative systems, in which humans and robots perform tasks in shared spaces and interact with close and direct contacts. In these scenarios, it is fundamental for the robot to be aware of the behaviour that a person in its proximity has, to ensure their safety and anticipate their actions in performing a shared and collaborative task. To this end, human activity recognition (HAR) techniques have been often applied in human-robot collaboration (HRC) settings. The works in this field usually focus on case-specific applications. Instead, in this thesis we propose a general framework for human action and gesture recognition in a HRC scenario. In particular, a transfer learning enabled skeleton-based approach that employs as backbone the Shift-GCN architecture is used to classify general actions related to HRC scenarios. Pose-based body and hands features are exploited to recognise actions in a way that is independent from the environment in which these are performed and from the tools and objects involved in their execution. The fusion of small network modules, each dedicated to the recognition of either the body or hands movements, is then explored. This allows to better understand the importance of different body parts in the recognition of the actions as well as to improve the classification outcomes. For our experiments, we used the large-scale NTU RGB+D dataset to pre-train the networks. Moreover, a new HAR dataset, named IAS-Lab Collaborative HAR dataset, was collected, containing general actions and gestures related to HRC contexts. On this dataset, our approach reaches a 76.54% accuracy

Human-Robot Collaborations in Industrial Automation

Technology is changing the manufacturing world. For example, sensors are being used to track inventories from the manufacturing floor up to a retail shelf or a customer’s door. These types of interconnected systems have been called the fourth industrial revolution, also known as Industry 4.0, and are projected to lower manufacturing costs. As industry moves toward these integrated technologies and lower costs, engineers will need to connect these systems via the Internet of Things (IoT). These engineers will also need to design how these connected systems interact with humans. The focus of this Special Issue is the smart sensors used in these human–robot collaborations

Technologies of information transmission and processing

Сборник содержит статьи, тематика которых посвящена научно-теоретическим разработкам в области сетей телекоммуникаций, информационной безопасности, технологий передачи и обработки информации. Предназначен для научных сотрудников в области инфокоммуникаций, преподавателей, аспирантов, магистрантов и студентов технических вузов

Fuzzy Logic in Surveillance Big Video Data Analysis: Comprehensive Review, Challenges, and Research Directions

CCTV cameras installed for continuous surveillance generate enormous amounts of data daily, forging the term “Big Video Data” (BVD). The active practice of BVD includes intelligent surveillance and activity recognition, among other challenging tasks. To efficiently address these tasks, the computer vision research community has provided monitoring systems, activity recognition methods, and many other computationally complex solutions for the purposeful usage of BVD. Unfortunately, the limited capabilities of these methods, higher computational complexity, and stringent installation requirements hinder their practical implementation in real-world scenarios, which still demand human operators sitting in front of cameras to monitor activities or make actionable decisions based on BVD. The usage of human-like logic, known as fuzzy logic, has been employed emerging for various data science applications such as control systems, image processing, decision making, routing, and advanced safety-critical systems. This is due to its ability to handle various sources of real world domain and data uncertainties, generating easily adaptable and explainable data-based models. Fuzzy logic can be effectively used for surveillance as a complementary for huge-sized artificial intelligence models and tiresome training procedures. In this paper, we draw researchers’ attention towards the usage of fuzzy logic for surveillance in the context of BVD. We carry out a comprehensive literature survey of methods for vision sensory data analytics that resort to fuzzy logic concepts. Our overview highlights the advantages, downsides, and challenges in existing video analysis methods based on fuzzy logic for surveillance applications. We enumerate and discuss the datasets used by these methods, and finally provide an outlook towards future research directions derived from our critical assessment of the efforts invested so far in this exciting field

Analyzing Granger causality in climate data with time series classification methods

Attribution studies in climate science aim for scientifically ascertaining the influence of climatic variations on natural or anthropogenic factors. Many of those studies adopt the concept of Granger causality to infer statistical cause-effect relationships, while utilizing traditional autoregressive models. In this article, we investigate the potential of state-of-the-art time series classification techniques to enhance causal inference in climate science. We conduct a comparative experimental study of different types of algorithms on a large test suite that comprises a unique collection of datasets from the area of climate-vegetation dynamics. The results indicate that specialized time series classification methods are able to improve existing inference procedures. Substantial differences are observed among the methods that were tested

Digital workflows for the management of existing structures in the pre- and post-earthquake phases: BIM, CDE, drones, laser-scanning and AI

La metodologia BIM, sviluppata in America negli anni '70, ha rivoluzionato l'industria delle costruzioni introducendo i principi di innovazione e digitalizzazione per la gestione dei progetti, in un settore settore produttivo troppo legato a logiche tradizionali. I numerosi processi digitali che sono stati sviluppati da allora hanno riguardato in gran parte la progettazione di nuovi edifici, e sono principalmente legati alla disciplina del construction management. Alcune prime sperimentazioni condotte nel tempo hanno mostrato come l'estensione di questa metodologia agli edifici esistenti comporti molte difficoltà. In questo panorama, il lavoro di tesi si concentra sulla gestione delle strutture nella fase pre e post-sisma con l'obiettivo di sviluppare processi digitali basati sull'uso di tecnologie innovative applicate sia agli edifici ordinari che a quelli storici. Il primo workflow sviluppato, relativo alla fase pre-sisma, è stato denominato scan-to-FEM, ed è finalizzato a particolarizzare il classico processo scan-to-BIM nel campo dell'ingegneria strutturale, analizzando così tutti i passaggi dal rilievo dell'edificio con le tecniche digitali di fotogrammetria e laser-scanning fino all'analisi strutturale e alla valutazione della sicurezza nei confronti delle azioni sismiche. I processi di gestione delle strutture post-sisma sono invece incentrati sulla stima della sicurezza della struttura e sulla definizione delle strategie di intervento, e si basano sull'analisi delle caratteristiche intrinseche della struttura e dei danni indotti dagli eventi sismici. L'intero processo di valutazione del livello operativo di un edificio è stato quindi rivisto alla luce delle moderne tecnologie digitali. Nel dettaglio, sono state sviluppate Reti Neurali Convoluzionali (CNN) per la crack detection, e l'estrazione delle informazioni numeriche associate alle lesioni, gestite poi grazie ai modelli BIM. I quadri fessurativi sono stati digitalizzati grazie allìintroduzione un nuovo oggetto BIM "lesione" (attualmente non codificato nello standard IFC), al quale è stato aggiunto un set di parametri in parte valutati con le CNN ed in parte qualitativi. Durante lo sviluppo di questi processi, sono stati sviluppati nuovi strumenti adhoc per la gestione degli edifici esistenti. In particolare, sono state definite specifiche per lo sviluppo di schede tecniche digitali dei danni, e per la creazione del nuovo oggetto BIM "lesione". I processi di gestione degli edifici danneggiati, grazie agli sviluppi tecnologici realizzati, sono stati applicati per la digitalizzazione dell'edificio storico della chiesa di San Pietro in Vinculis danneggiato a seguito di eventi sismici, grazie ai quali sono stati sperimentati i massimi benefici in termini di riduzione di tempo e risparmio di risorse

Bridging the gap between emotion and joint action

Our daily human life is filled with a myriad of joint action moments, be it children playing, adults working together (i.e., team sports), or strangers navigating through a crowd. Joint action brings individuals (and embodiment of their emotions) together, in space and in time. Yet little is known about how individual emotions propagate through embodied presence in a group, and how joint action changes individual emotion. In fact, the multi-agent component is largely missing from neuroscience-based approaches to emotion, and reversely joint action research has not found a way yet to include emotion as one of the key parameters to model socio-motor interaction. In this review, we first identify the gap and then stockpile evidence showing strong entanglement between emotion and acting together from various branches of sciences. We propose an integrative approach to bridge the gap, highlight five research avenues to do so in behavioral neuroscience and digital sciences, and address some of the key challenges in the area faced by modern societies

Информационная безопасность

В сборнике опубликованы материалы докладов, представленных на 59-й научной конференции аспирантов, магистрантов и студентов БГУИР. Материалы одобрены оргкомитетом и публикуются в авторской редакции. Для научных и инженерно-технических работников, преподавателей, аспирантов, магистрантов и студентов вузов

Data-driven prognostics for critical electronic assemblies and electromechanical components

The industrial digitalisation enables the adoption of robust, data-driven maintenance strategies that increase safety and reliability of critical assets such as electronics. And yet, an implementation of data-driven methods which primarily address the industrialisation of diagnostic and prognostic strategies is opposed by various, application specific challenges. This thesis collates such restricting factors encountered within the oil and gas industry, in particular for the critical electrical systems and components in upstream deep drilling tools. A fleet-level, tuned machine learning approach is presented that classifies the operational state (no-failure/ failure) of downhole tool printed circuit board assemblies. It supports maintenance decision making under varying levels of failure costs and fleet reliability scenarios. Applied within a maintenance scheme it has the potential to minimise non-productive time while increasing operational reliability. Likewise, a tailored and efficient deep learning data pipeline is proposed for a component-level forecast of the end of life of electromagnetic relays. It is evaluated using high resolution life-cycle data which has been collected as a part of this thesis. In combination with a failure analysis, the proposed method improves the prognostics capabilities compared to traditional methods which have been proposed so far in order to assess the operational health of electromagnetic relays. Two case studies underpin the need for tailored prognostic methods in order to provide viable solutions that can de-risk deep drilling operations. In consequence, the proposed approaches alleviate the pressure on current maintenance strategies which can no longer meet the stringent reliability requirements of upstream assets

Computer aided assessment of CT scans of traumatic brain injury patients

A thesis submitted in partial fulfilment for the degree of Doctor of PhilosophyOne of the serious public health problems is the Traumatic Brain Injury, also known as silent epidemic, affecting millions every year. Management of these patients essentially involves neuroimaging and noncontrast CT scans are the first choice amongst doctors. Significant anatomical changes identified on the neuroimages and volumetric assessment of haemorrhages and haematomas are of critical importance for assessing the patients’ condition for targeted therapeutic and/or surgical interventions. Manual demarcation and annotation by experts is still considered gold standard, however, the interpretation of neuroimages is fraught with inter-observer variability and is considered ’Achilles heel’ amongst radiologists. Errors and variability can be attributed to factors such as poor perception, inaccurate deduction, incomplete knowledge or the quality of the image and only a third of doctors confidently report the findings. The applicability of computer aided dianosis in segmenting the apposite regions and giving ’second opinion’ has been positively appraised to assist the radiologists, however, results of the approaches vary due to parameters of algorithms and manual intervention required from doctors and this presents a gap for automated segmentation and estimation of measurements of noncontrast brain CT scans. The Pattern Driven, Content Aware Active Contours (PDCAAC) Framework developed in this thesis provides robust and efficient segmentation of significant anatomical landmarks, estimations of their sizes and correlation to CT rating to assist the radiologists in establishing the diagnosis and prognosis more confidently. The integration of clinical profile of the patient into image segmentation algorithms has significantly improved their performance by highlighting characteristics of the region of interest. The modified active contour method in the PDCAAC framework achieves Jaccard Similarity Index (JI) of 0.87, which is a significant improvement over the existing methods of active contours achieving JI of 0.807 with Simple Linear Iterative Clustering and Distance Regularized Level Set Evolution. The Intraclass Correlation Coefficient of intracranial measurements is >0.97 compared with radiologists. Automatic seeding of the initial seed curve within the region of interest is incorporated into the method which is a novel approach and alleviates limitation of existing methods. The proposed PDCAAC framework can be construed as a contribution towards research to formulate correlations between image features and clinical variables encompassing normal development, ageing, pathological and traumatic cases propitious to improve management of such patients. Establishing prognosis usually entails survival but the focus can also be extended to functional outcomes, residual disability and quality of life issues