Action-oriented Scene Understanding

In order to allow robots to act autonomously it is crucial that they do not only describe their environment accurately but also identify how to interact with their surroundings. While we witnessed tremendous progress in descriptive computer vision, approaches that explicitly target action are scarcer. This cumulative dissertation approaches the goal of interpreting visual scenes “in the wild” with respect to actions implied by the scene. We call this approach action-oriented scene understanding. It involves identifying and judging opportunities for interaction with constituents of the scene (e.g. objects and their parts) as well as understanding object functions and how interactions will impact the future. All of these aspects are addressed on three levels of abstraction: elements, perception and reasoning. On the elementary level, we investigate semantic and functional grouping of objects by analyzing annotated natural image scenes. We compare object label-based and visual context definitions with respect to their suitability for generating meaningful object class representations. Our findings suggest that representations generated from visual context are on-par in terms of semantic quality with those generated from large quantities of text. The perceptive level concerns action identification. We propose a system to identify possible interactions for robots and humans with the environment (affordances) on a pixel level using state-of-the-art machine learning methods. Pixel-wise part annotations of images are transformed into 12 affordance maps. Using these maps, a convolutional neural network is trained to densely predict affordance maps from unknown RGB images. In contrast to previous work, this approach operates exclusively on RGB images during both, training and testing, and yet achieves state-of-the-art performance. At the reasoning level, we extend the question from asking what actions are possible to what actions are plausible. For this, we gathered a dataset of household images associated with human ratings of the likelihoods of eight different actions. Based on the judgement provided by the human raters, we train convolutional neural networks to generate plausibility scores from unseen images. Furthermore, having considered only static scenes previously in this thesis, we propose a system that takes video input and predicts plausible future actions. Since this requires careful identification of relevant features in the video sequence, we analyze this particular aspect in detail using a synthetic dataset for several state-of-the-art video models. We identify feature learning as a major obstacle for anticipation in natural video data. The presented projects analyze the role of action in scene understanding from various angles and in multiple settings while highlighting the advantages of assuming an action-oriented perspective. We conclude that action-oriented scene understanding can augment classic computer vision in many real-life applications, in particular robotics

A dynamic and relational perspective on vulnerability and fear of crime : The role of physical, psychological, and social factors as well as life events and neighborhood contexts using a between-within person approach

This thesis investigates the usefulness of the concept of vulnerability in explaining the fear of crime. Previous vulnerability approaches in fear-of-crime research are reworked and expanded, integrating a stronger temporal perspective and differentiating more precisely between persons and their contexts. It is demonstrated that between-person differences and within-person changes of most vulnerability factors (e.g., personality traits, financial strain, and supportive networks) are related to fear of crime. This longitudinal perspective provides more reliable support for the vulnerability approach than previous cross-sectional studies because unobserved heterogeneity is reduced. Victimization leads to increased perceived environmental adversity although not having the hypothesized influence on the locus of control. The impact of (early) life events on fear of crime and whether the examined theoretical mechanisms mediate vulnerability factors is investigated in cross-sectional analyses, suggesting that early life events influence fear of crime. The theoretically derived vulnerability mechanisms mediate all investigated vulnerability factors. An examination of neighborhood characteristics and their spatial lags shows that social disadvantage in the (adjacent) neighborhood has a strong contextual influence on fear of crime. Vulnerability links people and environments, indicating an interactive relationship between individual vulnerability factors and external stressors (neighborhood characteristics and victimization). The most substantial interaction is that older people are less affected by neighborhood characteristics than younger people

Consciosusness in Cognitive Architectures. A Principled Analysis of RCS, Soar and ACT-R

This report analyses the aplicability of the principles of consciousness developed in the ASys project to three of the most relevant cognitive architectures. This is done in relation to their aplicability to build integrated control systems and studying their support for general mechanisms of real-time consciousness.\ud To analyse these architectures the ASys Framework is employed. This is a conceptual framework based on an extension for cognitive autonomous systems of the General Systems Theory (GST).\ud A general qualitative evaluation criteria for cognitive architectures is established based upon: a) requirements for a cognitive architecture, b) the theoretical framework based on the GST and c) core design principles for integrated cognitive conscious control systems