Multilevel Chinese takeaway process and label-based processes for rule induction in the context of automated sports video annotation

We propose four variants of a novel hierarchical hidden Markov models strategy for rule induction in the context of automated sports video annotation including a multilevel Chinese takeaway process (MLCTP) based on the Chinese restaurant process and a novel Cartesian product label-based hierarchical bottom-up clustering (CLHBC) method that employs prior information contained within label structures. Our results show significant improvement by comparison against the flat Markov model: optimal performance is obtained using a hybrid method, which combines the MLCTP generated hierarchical topological structures with CLHBC generated event labels. We also show that the methods proposed are generalizable to other rule-based environments including human driving behavior and human actions

Quantum dot conjugated nanobodies for multiplex imaging of protein dynamics at synapses

Neurons communicate with each other through synapses, which show enrichment for specialized receptors. Although many studies have explored spatial enrichment and diffusion of these receptors in dissociated neurons using single particle tracking, much less is known about their dynamic properties at synapses in complex tissue like brain slices. Here we report the use of smaller and highly specific quantum dots conjugated with a recombinant single domain antibody fragment (VHH fragment) against green fluorescent protein to provide information on diffusion of adhesion molecules at the growth cone and neurotransmitter receptors at synapses. Our data reveals that QD-nanobodies can measure neurotransmitter receptor dynamics at both excitatory and inhibitory synapses in primary neuronal cultures as well as in ex vivo rat brain slices. We also demonstrate that this approach can be applied to tagging multiple proteins to simultaneously monitor their behavior. Thus, we provide a strategy for multiplex imaging of tagged membrane proteins to study their clustering, diffusion and transport both in vitro as well as in native tissue environments such as brain slices

Computationally Efficient Implementation of Convolution-based Locally Adaptive Binarization Techniques

One of the most important steps of document image processing is binarization. The computational requirements of locally adaptive binarization techniques make them unsuitable for devices with limited computing facilities. In this paper, we have presented a computationally efficient implementation of convolution based locally adaptive binarization techniques keeping the performance comparable to the original implementation. The computational complexity has been reduced from O(W2N2) to O(WN2) where WxW is the window size and NxN is the image size. Experiments over benchmark datasets show that the computation time has been reduced by 5 to 15 times depending on the window size while memory consumption remains the same with respect to the state-of-the-art algorithmic implementation

A novel Markov logic rule induction strategy for characterizing sports video footage

The grounding of high-level semantic concepts is a key requirement of video annotation systems. Rule induction can thus constitute an invaluable intermediate step in characterizing protocol-governed domains, such as broadcast sports footage. We here set out a novel “clause grammar template” approach to the problem of rule-induction in video footage of court games that employs a second-order meta grammar for Markov Logic Network construction. The aim is to build an adaptive system for sports video annotation capable, in principle, both of learning ab initio and also adaptively transferring learning between distinct rule domains. The method is tested with respect to both a simulated game predicate generator and also real data derived from tennis footage via computer-vision based approaches including HOG3D based player-action classification, Hough-transform based court detection, and graph-theoretic ball-tracking. Experiments demonstrate that the method exhibits both error resilience and learning transfer in the court domain context. Moreover the clause template approach naturally generalizes to any suitably-constrained, protocol-governed video domain characterized by feature noise or detector error

Automatic annotation of tennis games: an integration of audio, vision, and learning

Fully automatic annotation of tennis game using broadcast video is a task with a great potential but with enormous challenges. In this paper we describe our approach to this task, which integrates computer vision, machine listening, and machine learning. At the low level processing, we improve upon our previously proposed state-of-the-art tennis ball tracking algorithm and employ audio signal processing techniques to detect key events and construct features for classifying the events. At high level analysis, we model event classification as a sequence labelling problem, and investigate four machine learning techniques using simulated event sequences. Finally, we evaluate our proposed approach on three real world tennis games, and discuss the interplay between audio, vision and learning. To the best of our knowledge, our system is the only one that can annotate tennis game at such a detailed level

Domain anomaly detection in machine perception: a system architecture and taxonomy

We address the problem of anomaly detection in machine perception. The concept of domain anomaly is introduced as distinct from the conventional notion of anomaly used in the literature. We propose a unified framework for anomaly detection which exposes the multifacetted nature of anomalies and suggest effective mechanisms for identifying and distinguishing each facet as instruments for domain anomaly detection. The framework draws on the Bayesian probabilistic reasoning apparatus which clearly defines concepts such as outlier, noise, distribution drift, novelty detection (object, object primitive), rare events, and unexpected events. Based on these concepts we provide a taxonomy of domain anomaly events. One of the mechanisms helping to pinpoint the nature of anomaly is based on detecting incongruence between contextual and noncontextual sensor(y) data interpretation. The proposed methodology has wide applicability. It underpins in a unified way the anomaly detection applications found in the literature

Evaluation of face recognition system in heterogeneous environments (visible vs NIR)

Performing facial recognition between Near Infrared (NIR) and visible-light (VIS) images has been established as a common method of countering illumination variation problems in face recognition. In this paper we present a new database to enable the evaluation of cross-spectral face recognition. A series of preprocessing algorithms, followed by Local Binary Pattern Histogram (LBPH) representation and combinations with Linear Discriminant Analysis (LDA) are used for recognition. These experiments are conducted on both NIR→VIS and the less common VIS→NIR protocols, with permutations of uni-modal training sets. 12 individual baseline algorithms are presented. In addition, the best performing fusion approaches involving a subset of 12 algorithms are also described. © 2011 IEEE

Gaussian-weighted moving-window robust automatic threshold selection

A multi-scale, moving-window method for local thresholding based on Robust Automatic Threshold Selection (RATS) is developed. Using a model for the noise response of the optimal edge detector in this context, the reliability of thresholds computed at different scales is determined. The threshold computed at the smallest scale at which the reliability is suffcient is used. The performance on 2-D images is evaluated on synthetic an natural images in the presence of varying background and noise. Results show the method deals better with these problems than earlier versions of RATS at most noise levels

Neuronal activity mediated regulation of glutamate transporter GLT-1 surface diffusion in rat astrocytes in dissociated and slice cultures.

The astrocytic GLT-1 (or EAAT2) is the major glutamate transporter for clearing synaptic glutamate. While the diffusion dynamics of neurotransmitter receptors at the neuronal surface are well understood, far less is known regarding the surface trafficking of transporters in subcellular domains of the astrocyte membrane. Here, we have used live-cell imaging to study the mechanisms regulating GLT-1 surface diffusion in astrocytes in dissociated and brain slice cultures. Using GFP-time lapse imaging, we show that GLT-1 forms stable clusters that are dispersed rapidly and reversibly upon glutamate treatment in a transporter activity-dependent manner. Fluorescence recovery after photobleaching and single particle tracking using quantum dots revealed that clustered GLT-1 is more stable than diffuse GLT-1 and that glutamate increases GLT-1 surface diffusion in the astrocyte membrane. Interestingly, the two main GLT-1 isoforms expressed in the brain, GLT-1a and GLT-1b, are both found to be stabilized opposed to synapses under basal conditions, with GLT-1b more so. GLT-1 surface mobility is increased in proximity to activated synapses and alterations of neuronal activity can bidirectionally modulate the dynamics of both GLT-1 isoforms. Altogether, these data reveal that astrocytic GLT-1 surface mobility, via its transport activity, is modulated during neuronal firing, which may be a key process for shaping glutamate clearance and glutamatergic synaptic transmission