Biologically-inspired motion detection and classification : human and machine perception

Humans are good at the perception of biological motion, i.e. the motion of living things. The human perceptual system can tolerate not only variations in lighting conditions, distance, etc., but it can also perceive such motion and categorise it as walking, running, jumping etc. from minimal information systems such as moving light displays (MLDs). In these displays only specific points (e.g. joints in the case of a human being) are visible. Although a static display looks like a random configuration of dots, a dynamic display is perceptually organised into a moving figure. Some kind of temporal integration of the spatial contents seems to be a part of the perception mechanism; as manifested from the minimum presentation time required for biological motion to become apparent. One possible way to understand human perception may be to build an equivalent machine model. An analysis of the workings of this machine may lend us an insight into human perception. In this work, we considered a closed set of 12 different categories of MLD sequences. These sequences were shown to 93 participants and their responses are used as the basis of comparison of human and machine perception. Human responses were compared with the performance of /c-nearest neighbour and neural network detectors. Machine perception is found to differ from human perception in some important respects. We also examined the related aspect of person identification on the basis of gait. This has important applications in the fields of surveillance and biometrics. In recent years, gait has been investigated as a potential biometric; as this may be the only information available to identify a distant and/or otherwise masked person. Humans can learn to recognise different subjects in MLDs. In our experiments with a dataset of 21 subjects, an accuracy of nearly 90% and 100% was achieved with neural network and support vector machine classifiers respectively. Also the machines were able to make this recognition in a fraction of a gait cycle.</p

Computational model of MST neuron receptive field and interaction effect for the perception of self-motion

Biologically plausible approach is an alternative to conventional engineering approaches when developing algorithms for intelligent systems. It is apparent that biologically inspired algorithms may yield more expensive calculations when comparing its run time to the more commonly used engineering algorithms. However, biologically inspired approaches have great potential in generating better and more accurate outputs as healthy human brains. Therefore more and more new and exciting researches are being experimented everyday in hope to develop better models of our brain that can be utilized by the machines. This thesis work is an effort to design and implement a computational model of neurons from the visual cortex\u27s MST area (medial superior temporal area). MST\u27s primary responsibility is detecting self-motion from optic flow stimulus that are segmented from the visual input. The computational models are to be built with dual Gaussian functions and genetic algorithm as its principle training method, from the data collected through lab monkey\u27s MST neurons. The resulting computational models can be used in further researches as part of motion detection mechanism by machine vision applications, which may prove to be an effective alternative motion detection algorithm in contrast to the conventional computer vision algorithms such as frame differencing. This thesis work will also explore the interaction effect that has been discovered from the newly gathered data, provided by University of Rochester Medical Center, Neurology Department

A biologically inspired spiking model of visual processing for image feature detection

To enable fast reliable feature matching or tracking in scenes, features need to be discrete and meaningful, and hence edge or corner features, commonly called interest points are often used for this purpose. Experimental research has illustrated that biological vision systems use neuronal circuits to extract particular features such as edges or corners from visual scenes. Inspired by this biological behaviour, this paper proposes a biologically inspired spiking neural network for the purpose of image feature extraction. Standard digital images are processed and converted to spikes in a manner similar to the processing that transforms light into spikes in the retina. Using a hierarchical spiking network, various types of biologically inspired receptive fields are used to extract progressively complex image features. The performance of the network is assessed by examining the repeatability of extracted features with visual results presented using both synthetic and real images

Object Segmentation from Motion Discontinuities and Temporal Occlusions–A Biologically Inspired Model

BACKGROUND: Optic flow is an important cue for object detection. Humans are able to perceive objects in a scene using only kinetic boundaries, and can perform the task even when other shape cues are not provided. These kinetic boundaries are characterized by the presence of motion discontinuities in a local neighbourhood. In addition, temporal occlusions appear along the boundaries as the object in front covers the background and the objects that are spatially behind it. METHODOLOGY/PRINCIPAL FINDINGS: From a technical point of view, the detection of motion boundaries for segmentation based on optic flow is a difficult task. This is due to the problem that flow detected along such boundaries is generally not reliable. We propose a model derived from mechanisms found in visual areas V1, MT, and MSTl of human and primate cortex that achieves robust detection along motion boundaries. It includes two separate mechanisms for both the detection of motion discontinuities and of occlusion regions based on how neurons respond to spatial and temporal contrast, respectively. The mechanisms are embedded in a biologically inspired architecture that integrates information of different model components of the visual processing due to feedback connections. In particular, mutual interactions between the detection of motion discontinuities and temporal occlusions allow a considerable improvement of the kinetic boundary detection. CONCLUSIONS/SIGNIFICANCE: A new model is proposed that uses optic flow cues to detect motion discontinuities and object occlusion. We suggest that by combining these results for motion discontinuities and object occlusion, object segmentation within the model can be improved. This idea could also be applied in other models for object segmentation. In addition, we discuss how this model is related to neurophysiological findings. The model was successfully tested both with artificial and real sequences including self and object motion

Search Tracker: Human-derived object tracking in-the-wild through large-scale search and retrieval

Humans use context and scene knowledge to easily localize moving objects in conditions of complex illumination changes, scene clutter and occlusions. In this paper, we present a method to leverage human knowledge in the form of annotated video libraries in a novel search and retrieval based setting to track objects in unseen video sequences. For every video sequence, a document that represents motion information is generated. Documents of the unseen video are queried against the library at multiple scales to find videos with similar motion characteristics. This provides us with coarse localization of objects in the unseen video. We further adapt these retrieved object locations to the new video using an efficient warping scheme. The proposed method is validated on in-the-wild video surveillance datasets where we outperform state-of-the-art appearance-based trackers. We also introduce a new challenging dataset with complex object appearance changes.Comment: Under review with the IEEE Transactions on Circuits and Systems for Video Technolog