Local Temporal Bilinear Pooling for Fine-grained Action Parsing

Fine-grained temporal action parsing is important in many applications, such as daily activity understanding, human motion analysis, surgical robotics and others requiring subtle and precise operations in a long-term period. In this paper we propose a novel bilinear pooling operation, which is used in intermediate layers of a temporal convolutional encoder-decoder net. In contrast to other work, our proposed bilinear pooling is learnable and hence can capture more complex local statistics than the conventional counterpart. In addition, we introduce exact lower-dimension representations of our bilinear forms, so that the dimensionality is reduced with neither information loss nor extra computation. We perform intensive experiments to quantitatively analyze our model and show the superior performances to other state-of-the-art work on various datasets.Comment: 11 pages, 2 figures. Cam.

Model investigation on contribution of feedback in distortion induced motion adaptation

Motion information is processed in a neural circuit formed by synaptic organization of feedforward (FF) and feedback (FB) connections between different cortical areas. However, the contribution of a recurrent FB information to adaptation process is not well explored. Here, a biologically plausible neural model that predicts motion adaptation aftereffect (MAE) induced by exposure to geometrically skewed natural image sequences is suggested. The model constitutes two stage recurrent motion processing within cortical areas V1 and MT [1]. It comprises FF excitatory, FB modulatory and lateral inhibitory connections, and a fast and a slow adaptive synapse in the FF and FB streams, respectively, to introduce plasticity. Simulation results of the model show the following main contributions of FB in distortion induced motion adaptation: FB disambiguates the main signal from a noisy natural stimulus input: results in adaptation to globally consistent salient information. A model with distinct adaptive mechanisms in FF and FB streams predicts MAE at different time scales of exposure to skewed natural stimuli more accurately than other model variants constituting single adaptive mechanism: Multiple adaptive mechanisms might be implemented via FB pathways. FB allows similar response tuning in model area V1 and MT during adaptation in line with physiological findings [2]. [1] Bayerl, P. and H. Neumann, Disambiguating visual motion through contextual feedback modulation. Neural computation, 2004. 16(10): p. 2041-2066. [2] Patterson, C.A., et al., Similar adaptation effects in primary visual cortex and area MT of the macaque monkey under matched stimulus conditions. Journal of neurophysiology, 2013. 111(6): p. 1203-1213

The Role of Bottom-Up and Top-Down Cortical Interactions in Adaptation to Natural Scene Statistics

Adaptation is a mechanism by which cortical neurons adjust their responses according to recently viewed stimuli. Visual information is processed in a circuit formed by feedforward (FF) and feedback (FB) synaptic connections of neurons in different cortical layers. Here, the functional role of FF-FB streams and their synaptic dynamics in adaptation to natural stimuli is assessed in psychophysics and neural model. We propose a cortical model which predicts psychophysically observed motion adaptation aftereffects (MAE) after exposure to geometrically distorted natural image sequences. The model comprises direction selective neurons in V1 and MT connected by recurrent FF and FB dynamic synapses. Psychophysically plausible model MAEs were obtained from synaptic changes within neurons tuned to salient direction signals of the broadband natural input. It is conceived that, motion disambiguation by FF-FB interactions is critical to encode this salient information. Moreover, only FF-FB dynamic synapses operating at distinct rates predicted psychophysical MAEs at different adaptation time-scales which could not be accounted for by single rate dynamic synapses in either of the streams. Recurrent FF-FB pathways thereby play a role during adaptation in a natural environment, specifically in inducing multilevel cortical plasticity to salient information and in mediating adaptation at different time-scales

Incorporating Feedback in Convolutional Neural Networks

This projects investigates schemes to combine feedforward and feedback signals in deep neural networks

Poster - CCN 2019

Poster presented at Cognitive Computational Neuroscience Conference 2019 in Berlin, for paper see: https://doi.org/10.32470/CCN.2019.1191-

The Alexithymia Questionnaire for Children – German version (AQC-G)

Here, we present the German adaptation of the Alexithymia Questionnaire for Children (AQC, Rieffe et al., 2006)

Does time-to-surgery affect mortality in patients with acute osteoporotic vertebral compression fractures?

Introduction!#!Osteoporotic vertebral compression fractures (VCFs) are common. An increase in mortality associated with osteoporotic VCFs has been well documented. The purpose of this study was to assess the impact of time to surgery on 1-year survival in patients with osteoporotic vertebral compression fractures.!##!Methods!#!In a retrospective cohort study with prospective mortality follow-up, consecutive patients aged ≥ 60 years who had operative treatment of a low-energy fracture of a thoracolumbar vertebra and had undergone surgical stabilization between January 2015 and December 2018 were identified from our institutional database. By chart review, additional information on hospitalization time, comorbidities (expressed as ASA - American Society of Anesthesiologists Scale), complications and revision surgery was obtained. Time-to-surgery was defined as the time between admission and surgery. Mortality data was assessed by contacting the patients by phone, mail or the national social insurance database.!##!Results!#!Two hundred sixty patients (mean age 78 years, SD 7 years, range, 60 to 93; 172 female) were available for final analysis. Mean follow-up was 40 months (range, 12 to 68 months). Fifty-nine patients (22.7%) had died at final follow-up and 27/260 patients (10.4%) had died within 1 year after the surgery. Time-to-surgery was not different for patients who died within 1 year after the surgery and those who survived (p = .501). In-hospital complications were seen in 40/260 (15.4%) patients. Time-to-surgery showed a strong correlation with hospitalization time (Pearson's r = .614, p < .001), but only a very weak correlation with the time spent in hospital after the surgery (Pearson's r = .146, p = .018).!##!Conclusions!#!In contrast to patients with proximal femur factures, time-to-surgery had no significant effect on one-year mortality in geriatric patients with osteoporotic vertebral compression fractures. Treatment decisions for these fractures in the elderly should be individualized

Kinematics of the Lumbo–Pelvic Complex under Different Loading Conditions

The lumbo-pelvic complex is a highly complex structural system. The current investigation aims to identify the kinematics between interacting bone segments under different loading conditions. A specimen of the lumbo-pelvic complex was obtained from a human body donor and tested in a self-developed test rig. The experimental setup was designed to imitate extension, flexion, right and left lateral bending and axial rotation to the left and to the right, respectively. The vertebra L3 was firmly embedded and load was introduced via hip joints. Using a digital image correlation (DIC) system, the 3D motions of 15 markers at different landmarks were measured for each loadcase under cyclic loading. For each loadcase, the kinematics were analyzed in terms of three-dimensional relative movements between L3 and the sacrum. The usefulness of the experimental technique was demonstrated. It may serve for further biomechanical investigations of relative motion of sacroiliac and vertebral joints and deformation of bony structures

Reversal Learning in Humans and Gerbils: Dynamic Control Network Facilitates Learning

Biologically plausible modeling of behavioral reinforcement learning tasks has seen great improvements over the past decades. Less work has been dedicated to tasks involving contingency reversals, i.e., tasks in which the original behavioral goal is reversed one or multiple times. The ability to adjust to such reversals is a key element of behavioral flexibility. Here, we investigate the neural mechanisms underlying contingency-reversal tasks. We first conduct experiments with humans and gerbils to demonstrate memory effects, including multiple reversals in which subjects (humans and animals) show a faster learning rate when a previously learned contingency re-appears. Motivated by recurrent mechanisms of learning and memory for object categories, we propose a network architecture which involves reinforcement learning to steer an orienting system that monitors the success in reward acquisition. We suggest that a model sensory system provides feature representations which are further processed by category-related subnetworks which constitute a neural analog of expert networks. Categories are selected dynamically in a competitive field and predict the expected reward. Learning occurs in sequentialized phases to selectively focus the weight adaptation to synapses in the hierarchical network and modulate their weight changes by a global modulator signal. The orienting subsystem itself learns to bias the competition in the presence of continuous monotonic reward accumulation. In case of sudden changes in the discrepancy of predicted and acquired reward the activated motor category can be switched. We suggest that this subsystem is composed of a hierarchically organized network of dis-inhibitory mechanisms, dubbed a dynamic control network (DCN), which resembles components of the basal ganglia. The DCN selectively activates an expert network, corresponding to the current behavioral strategy. The trace of the accumulated reward is monitored such that large sudden deviations from the monotonicity of its evolution trigger a reset after which another expert subnetwork can be activated-if it has already been established before-or new categories can be recruited and associated with novel behavioral patterns