A Boxology of Design Patterns for Hybrid Learning and Reasoning Systems

We propose a set of compositional design patterns to describe a large variety of systems that combine statistical techniques from machine learning with symbolic techniques from knowledge representation. As in other areas of computer science (knowledge engineering, software engineering, ontology engineering, process mining and others), such design patterns help to systematize the literature, clarify which combinations of techniques serve which purposes, and encourage re-use of software components. We have validated our set of compositional design patterns against a large body of recent literature.Comment: 12 pages,55 reference

Discovery and recognition of motion primitives in human activities

We present a novel framework for the automatic discovery and recognition of motion primitives in videos of human activities. Given the 3D pose of a human in a video, human motion primitives are discovered by optimizing the `motion flux', a quantity which captures the motion variation of a group of skeletal joints. A normalization of the primitives is proposed in order to make them invariant with respect to a subject anatomical variations and data sampling rate. The discovered primitives are unknown and unlabeled and are unsupervisedly collected into classes via a hierarchical non-parametric Bayes mixture model. Once classes are determined and labeled they are further analyzed for establishing models for recognizing discovered primitives. Each primitive model is defined by a set of learned parameters. Given new video data and given the estimated pose of the subject appearing on the video, the motion is segmented into primitives, which are recognized with a probability given according to the parameters of the learned models. Using our framework we build a publicly available dataset of human motion primitives, using sequences taken from well-known motion capture datasets. We expect that our framework, by providing an objective way for discovering and categorizing human motion, will be a useful tool in numerous research fields including video analysis, human inspired motion generation, learning by demonstration, intuitive human-robot interaction, and human behavior analysis

Action Recognition in Videos: from Motion Capture Labs to the Web

This paper presents a survey of human action recognition approaches based on visual data recorded from a single video camera. We propose an organizing framework which puts in evidence the evolution of the area, with techniques moving from heavily constrained motion capture scenarios towards more challenging, realistic, "in the wild" videos. The proposed organization is based on the representation used as input for the recognition task, emphasizing the hypothesis assumed and thus, the constraints imposed on the type of video that each technique is able to address. Expliciting the hypothesis and constraints makes the framework particularly useful to select a method, given an application. Another advantage of the proposed organization is that it allows categorizing newest approaches seamlessly with traditional ones, while providing an insightful perspective of the evolution of the action recognition task up to now. That perspective is the basis for the discussion in the end of the paper, where we also present the main open issues in the area.Comment: Preprint submitted to CVIU, survey paper, 46 pages, 2 figures, 4 table

Augmented Reality in Learning Settings: A Systematic Analysis of its Benefits and Avenues for Future Studies

Despite its increasing use in various settings, Augmented Reality (AR) technology is still often considered experimental, partly due to a lack of clear understanding of the benefits of using AR. This study systematically reviews research on the use of AR in learning settings. Our analysis of 93 relevant articles offers 21 benefits related to the learning gains and outcomes of using AR. Our study shows that the positive effects of using AR on learners’ motivation and joy have been well-studied, whereas the effects on independent learning, concentration, spontaneous learning, critical thinking, and practical skills have not yet been examined in detail. Beyond classifying and discussing the benefits of using AR in learning settings, we elaborate avenues for future studies. We specifically point to the importance of conducting long-term studies to determine the value of using AR in learning beyond the initial novelty and exploring the integration of AR with other technologies

Survey on video anomaly detection in dynamic scenes with moving cameras

The increasing popularity of compact and inexpensive cameras, e.g.~dash cameras, body cameras, and cameras equipped on robots, has sparked a growing interest in detecting anomalies within dynamic scenes recorded by moving cameras. However, existing reviews primarily concentrate on Video Anomaly Detection (VAD) methods assuming static cameras. The VAD literature with moving cameras remains fragmented, lacking comprehensive reviews to date. To address this gap, we endeavor to present the first comprehensive survey on Moving Camera Video Anomaly Detection (MC-VAD). We delve into the research papers related to MC-VAD, critically assessing their limitations and highlighting associated challenges. Our exploration encompasses three application domains: security, urban transportation, and marine environments, which in turn cover six specific tasks. We compile an extensive list of 25 publicly-available datasets spanning four distinct environments: underwater, water surface, ground, and aerial. We summarize the types of anomalies these datasets correspond to or contain, and present five main categories of approaches for detecting such anomalies. Lastly, we identify future research directions and discuss novel contributions that could advance the field of MC-VAD. With this survey, we aim to offer a valuable reference for researchers and practitioners striving to develop and advance state-of-the-art MC-VAD methods.Comment: Under revie

A variability taxonomy to support automation decision-making for manufacturing processes

Although many manual operations have been replaced by automation in the manufacturing domain, in various industries skilled operators still carry out critical manual tasks such as final assembly. The business case for automation in these areas is difficult to justify due to increased complexity and costs arising out of process variabilities associated with those tasks. The lack of understanding of process variability in automation design means that industrial automation often does not realise the full benefits at the first attempt, resulting in the need to spend additional resource and time, to fully realise the potential. This article describes a taxonomy of variability when considering automation of manufacturing processes. Three industrial case studies were analysed to develop the proposed taxonomy. The results obtained from the taxonomy are discussed with a further case study to demonstrate its value in supporting automation decision-making

A variability taxonomy to support automation decision-making for manufacturing processes

Although many manual operations have been replaced by automation in the manufacturing domain, in various industries skilled operators still carry out critical manual tasks such as final assembly. The business case for automation in these areas is difficult to justify due to increased complexity and costs arising out of process variabilities associated with those tasks. The lack of understanding of process variability in automation design means that industrial automation often does not realise the full benefits at the first attempt, resulting in the need to spend additional resource and time, to fully realise the potential. This article describes a taxonomy of variability when considering automation of manufacturing processes. Three industrial case studies were analysed to develop the proposed taxonomy. The results obtained from the taxonomy are discussed with a further case study to demonstrate its value in supporting automation decision-making