Automatische Schätzung der Körperpose mit CNNs und LSTMs

In this thesis, we present an end-to-end approach to human pose estimation task that based on a deep hybrid architecture that combines convolutional neural network (CNNs) and recurrent neural networks (RNNs)

Investigating Semantic Alignment in Character Learning of Chinese as a Foreign Language: The Use and Effect of the Imagery Based Encoding Strategy

For learners of Chinese as a foreign language (CFL), character learning is frustrating. This research postulated that this difficulty may mainly come from a lack of semantic understanding of character-denoted meanings. Language theories support that when a learner’s semantic meaning increases, the orthographic structures that represent the underlying meanings also improve. This study aimed to reveal CFL learners’ cognitive abilities and processes in visual-semantic learning of Chinese characters. Particularly, this study investigated the process by which English-speaking adolescent CFL learners, at the beginning to intermediate level, made mental images of character-denoted meanings to visually encode and retrieve character forms. Quantitative and qualitative data were gathered from image making questionnaires, writing, and reading tests, after learning characters in three commonly-used teaching methods (i.e., English, pictorial, and verbal). The data were analyzed based on a triangulation of the literature from Neuro-Semantic Language Learning Theory, scientific findings in cognitive psychology, and neuroscience. The study found that participants’ semantic abilities to understand character-denoted meanings emerged, but were still restricted in familiar orthographic forms. The use of the imagery strategy as a semantic ability predicted better performances, most evidently in writing; however, the ability in using the imagery strategy to learn characters was still underdeveloped, and needed to be supported with sufficient contextual information. Implications and further research in visual-semantic learning and teaching characters were suggested

Spatial Aggregation: Theory and Applications

Visual thinking plays an important role in scientific reasoning. Based on the research in automating diverse reasoning tasks about dynamical systems, nonlinear controllers, kinematic mechanisms, and fluid motion, we have identified a style of visual thinking, imagistic reasoning. Imagistic reasoning organizes computations around image-like, analogue representations so that perceptual and symbolic operations can be brought to bear to infer structure and behavior. Programs incorporating imagistic reasoning have been shown to perform at an expert level in domains that defy current analytic or numerical methods. We have developed a computational paradigm, spatial aggregation, to unify the description of a class of imagistic problem solvers. A program written in this paradigm has the following properties. It takes a continuous field and optional objective functions as input, and produces high-level descriptions of structure, behavior, or control actions. It computes a multi-layer of intermediate representations, called spatial aggregates, by forming equivalence classes and adjacency relations. It employs a small set of generic operators such as aggregation, classification, and localization to perform bidirectional mapping between the information-rich field and successively more abstract spatial aggregates. It uses a data structure, the neighborhood graph, as a common interface to modularize computations. To illustrate our theory, we describe the computational structure of three implemented problem solvers -- KAM, MAPS, and HIPAIR --- in terms of the spatial aggregation generic operators by mixing and matching a library of commonly used routines.Comment: See http://www.jair.org/ for any accompanying file

Using Neuroeducation as a Model to Evaluate the Effect of Imagery on Chinese Character Writing

Using imagery as a strategy for language learning may be helpful to encode linguistic forms into conceptual networks for long-term memory. Based on Arwood’s neuroeducation model of language learning, this research evaluated the effect of imagery in Chinese character writing by English-speaking adolescent students. After comparing imagery effects under three instructional conditions (i.e., English translation, pictorial presentation, and verbal-contextual interpretation), the results showed that the use of imagery predicted significantly better writing results in the immediate and one-week writing tests, but not in the four-week writing test. Cognitive analyses found that imagery was commonly used as a mediational strategy in the pictorial and verbal-contextual methods in the early learning phases. The pictorial method mainly elicited perceptual visual patterns which failed to support sustained memory. For a better character encoding and retrieval, images had to be generated associated with sufficient and relevant contextual information

Public Transit Arrival Prediction: a Seq2Seq RNN Approach

Arrival/Travel times for public transit exhibit variability on account of factors like seasonality, dwell times at bus stops, traffic signals, travel demand fluctuation etc. The developing world in particular is plagued by additional factors like lack of lane discipline, excess vehicles, diverse modes of transport and so on. This renders the bus arrival time prediction (BATP) to be a challenging problem especially in the developing world. A novel data-driven model based on recurrent neural networks (RNNs) is proposed for BATP (in real-time) in the current work. The model intelligently incorporates both spatial and temporal correlations in a unique (non-linear) fashion distinct from existing approaches. In particular, we propose a Gated Recurrent Unit (GRU) based Encoder-Decoder(ED) OR Seq2Seq RNN model (originally introduced for language translation) for BATP. The geometry of the dynamic real time BATP problem enables a nice fit with the Encoder-Decoder based RNN structure. We feed relevant additional synchronized inputs (from previous trips) at each step of the decoder (a feature classically unexplored in machine translation applications). Further motivated from accurately modelling congestion influences on travel time prediction, we additionally propose to use a bidirectional layer at the decoder (something unexplored in other time-series based ED application contexts). The effectiveness of the proposed algorithms is demonstrated on real field data collected from challenging traffic conditions. Our experiments indicate that the proposed method outperforms diverse existing state-of-art data-driven approaches proposed for the same problem

Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image

We describe the first method to automatically estimate the 3D pose of the human body as well as its 3D shape from a single unconstrained image. We estimate a full 3D mesh and show that 2D joints alone carry a surprising amount of information about body shape. The problem is challenging because of the complexity of the human body, articulation, occlusion, clothing, lighting, and the inherent ambiguity in inferring 3D from 2D. To solve this, we first use a recently published CNN-based method, DeepCut, to predict (bottom-up) the 2D body joint locations. We then fit (top-down) a recently published statistical body shape model, called SMPL, to the 2D joints. We do so by minimizing an objective function that penalizes the error between the projected 3D model joints and detected 2D joints. Because SMPL captures correlations in human shape across the population, we are able to robustly fit it to very little data. We further leverage the 3D model to prevent solutions that cause interpenetration. We evaluate our method, SMPLify, on the Leeds Sports, HumanEva, and Human3.6M datasets, showing superior pose accuracy with respect to the state of the art.Comment: To appear in ECCV 201

Analysis of Marginal Discourse: Identity Construction in Campus Wall Graffiti

Campus wall graffiti plays a significant role in the identity construction of graffiti artists as a form of marginal discourse. However, due to its long-standing perception as a destructive activity and the unique cultural environment, graffiti research in China has been relatively underdeveloped, with a focus on theoretical investigations. This study collected a total of 391 textual and pictorial graffiti samples from a language university in China as the corpus. From the perspective of identity construction, this study analyzed the characteristics of campus wall graffiti, as well as the methods and content of identity construction. The findings revealed that campus graffiti exhibits characteristics such as informality, anonymity, and counter-culturality. The language behaviors and symbol usage in graffiti include both one-way output and two-way interaction, the employment of textual and pictorial symbols, and the use of metaphors and symbolism. The construction of identity in graffiti primarily involves cultural identity, social identity, and creative identity among others. Lastly, in terms of campus graffiti management, this paper proposes effective strategies from the aspects of discourse power allocation and management, as well as feedback and reconstruction of identity construction. These strategies aim to transform disorder into order and foster an open and inclusive campus culture