Autonomous computational intelligence-based behaviour recognition in security and surveillance

This paper presents a novel approach to sensing both suspicious, and task-specific behaviours through the use of advanced computational intelligence techniques. Locating suspicious activity in surveillance camera networks is an intensive task due to the volume of information and large numbers of camera sources to monitor. This results in countless hours of video data being streamed to disk without being screened by a human operator. To address this need, there are emerging video analytics solutions that have introduced new metrics such as people counting and route monitoring, alongside more traditional alerts such as motion detection. There are however few solutions that are sufficiently robust to reduce the need for human operators in these environments, and new approaches are needed to address the uncertainty in identifying and classifying human behaviours, autonomously, from a video stream. In this work we present an approach to address the autonomous identification of human behaviours derived from human pose analysis. Behavioural recognition is a significant challenge due to the complex subtleties that often make up an action; the large overlap in cues results in high levels of classification uncertainty. False alarms are significant impairments to autonomous detection and alerting systems, and over reporting can lead to systems being muted, disabled, or decommissioned. We present results on a Computational-Intelligence based Behaviour Recognition (CIBR) that utilises artificial intelligence to learn, optimise, and classify human activity. We achieve this through extraction of skeleton recognition of human forms within an image. A type-2 Fuzzy logic classifier then converts the human skeletal forms into a set of base atomic poses (standing, walking, etc.), after which a Markov-chain model is used to order a pose sequence. Through this method we are able to identify, with good accuracy, several classes of human behaviour that correlate with known suspicious, or anomalous, behaviours

Adaptive and Dynamic Feedback Loops between Production System and Production Network based on the Asset Administration Shell

In production networks, production must run efficiently across company boundaries. Companies must be able to react quickly as a single unit. Two trends are influencing this situation: On the one hand, the progressing servitization leads to the increased offering of digital services in the field of manufacturing. From the literature, it is known that digital services let manufacturers, suppliers, and industrial customers interact more closely and frequently in a production network. On the other hand, the concept of the digital twin is trending. It promises the real-time prognosis and control of production systems. Although the concept of the digital twin itself can be vague there are some technologies trying to implement the digital twin of production. The asset administration shell (AAS) is an example of such a technology that draws growing attention. Picking up the initial situation these two trends could be used to create a feedback loop between the production system and network and thus improve the overall efficiency in production networks. Based on this idea, the paper first presents an approach to model systematically a possibility for a feedback loop orienting to the business model concept. Second, a reference architecture is derived from the RAMI 4.0 standard. The specified reference architecture is the basis for the specific implementation. Third, a procedure is developed to implement a specific architecture. For implementing an architecture, the usage of the asset administration shell is assumed. Finally, the approach is validated in a use case from the high precision weight industry

Developing a model for analysis of the cooling loads of a hybrid electric vehicle by using co-simulations of verified submodels

The requirement for including the air-conditioning and the battery-cooling loads within the energy efficiency analyses of a hybrid electric vehicle is widely recognized and has promoted system-level simulations and integrated modelling, escalating the challenge of balancing the accuracy and the speed of simulations. In this paper, a hybrid electric vehicle model is created through co-simulation of the passenger cabin, the air conditioning, the battery cooling, and the powertrai. Calibration and verification of the submodels help determine their accuracy in representing the target vehicle and achieve a balance between the model fidelity and the simulation speed. The result is a model which has a higher accuracy and a higher speed than those of similar models developed previously and which provides a reliable tool for a thorough investigation of the cooling loads for different ambient conditions and different duty cycles

Synthesized lengthening of function words - The fuzzy boundary between fluency and disfluency

Betz S, Zarrieß S, Wagner P. Synthesized lengthening of function words - The fuzzy boundary between fluency and disfluency. In: Proceedings of the International Conference Fluency and Disfluency. 2017

MemeGraphs: Linking Memes to Knowledge Graphs

Memes are a popular form of communicating trends and ideas in social media and on the internet in general, combining the modalities of images and text. They can express humor and sarcasm but can also have offensive content. Analyzing and classifying memes automatically is challenging since their interpretation relies on the understanding of visual elements, language, and background knowledge. Thus, it is important to meaningfully represent these sources and the interaction between them in order to classify a meme as a whole. In this work, we propose to use scene graphs, that express images in terms of objects and their visual relations, and knowledge graphs as structured representations for meme classification with a Transformer-based architecture. We compare our approach with ImgBERT, a multimodal model that uses only learned (instead of structured) representations of the meme, and observe consistent improvements. We further provide a dataset with human graph annotations that we compare to automatically generated graphs and entity linking. Analysis shows that automatic methods link more entities than human annotators and that automatically generated graphs are better suited for hatefulness classification in memes

The greennn tree - lengthening position influences uncertainty perception

Betz S, Zarrieß S, Székely É, Wagner P. The greennn tree - lengthening position influences uncertainty perception. In: Proceedings of Interspeech. 2019: 3990-3994.Synthetic speech can be used to express uncertainty in dialogue systems by means of hesitation. If a phrase like “Next to the green tree” is uttered in a hesitant way, that is, containing lengthening, silences, and fillers, the listener can infer that the speaker is not certain about the concepts referred to. However, we do not know anything about the referential domain of the uncertainty; if only a particular word in this sentence would be uttered hesitantly, e.g. “the greee:n tree”, the listener could infer that the uncertainty refers to the color in the statement, but not to the object. In this study, we show that the domain of the uncertainty is controllable. We conducted an experiment in which color words in sentences like “search for the green tree” were lengthened in two different positions: word onsets or final consonants, and participants were asked to rate the uncertainty regarding color and object. The results show that initial lengthening is predominantly associated with uncertainty about the word itself, whereas final lengthening is primarily associated with the following object. These findings enable dialogue system developers to finely control the attitudinal display of uncertainty, adding nuances beyond the lexical content to message delivery

Increasing Recall of Lengthening Detection via Semi-Automatic Classification

Betz S, Voße J, Zarrieß S, Wagner P. Increasing Recall of Lengthening Detection via Semi-Automatic Classification. In: Proceedings of Interspeech. 2017: 1084-1088

Do Hesitations Facilitate Processing of Partially Defective System Utterances? An Exploratory Eye Tracking Study

Haake K, Schimke S, Betz S, Zarrieß S. Do Hesitations Facilitate Processing of Partially Defective System Utterances? An Exploratory Eye Tracking Study. In: Proceedings of Interspeech. 2019: 1906-1910

High throughput non-invasive determination of foetal Rhesus D status using automated extraction of cell-free foetal DNA in maternal plasma and mass spectrometry

Purpose: To examine the potential high throughput capability and efficiency of an automated DNA extraction system in combination with mass spectrometry for the non-invasive determination of the foetal Rhesus D status. Methods: A total of 178 maternal plasma samples from RHD-negative pregnant women were examined, from which DNA was extracted using the automated Roche MagNA Pure™ system. Presence of the foetal RHD gene was detected by PCR for RHD exon 7 and subsequent analysis using the Sequenom MassArray™ mass spectrometric system. Results: We determined that as little as 15pg of RHD-positive genomic DNA could be detected in a background of 585pg of RHD-negative genomic DNA. The analysis of the clinical samples yielded a sensitivity and specificity of 96.1 and 96.1%, respectively. Conclusion: Our study indicated that automated DNA extraction in combination with mass spectrometry permits the determination of foetal Rhesus D genotype with an accuracy comparable to the current approaches using real-time PC