Integrated 2-D Optical Flow Sensor

I present a new focal-plane analog VLSI sensor that estimates optical flow in two visual dimensions. The chip significantly improves previous approaches both with respect to the applied model of optical flow estimation as well as the actual hardware implementation. Its distributed computational architecture consists of an array of locally connected motion units that collectively solve for the unique optimal optical flow estimate. The novel gradient-based motion model assumes visual motion to be translational, smooth and biased. The model guarantees that the estimation problem is computationally well-posed regardless of the visual input. Model parameters can be globally adjusted, leading to a rich output behavior. Varying the smoothness strength, for example, can provide a continuous spectrum of motion estimates, ranging from normal to global optical flow. Unlike approaches that rely on the explicit matching of brightness edges in space or time, the applied gradient-based model assures spatiotemporal continuity on visual information. The non-linear coupling of the individual motion units improves the resulting optical flow estimate because it reduces spatial smoothing across large velocity differences. Extended measurements of a 30x30 array prototype sensor under real-world conditions demonstrate the validity of the model and the robustness and functionality of the implementation

A survey of visual preprocessing and shape representation techniques

Many recent theories and methods proposed for visual preprocessing and shape representation are summarized. The survey brings together research from the fields of biology, psychology, computer science, electrical engineering, and most recently, neural networks. It was motivated by the need to preprocess images for a sparse distributed memory (SDM), but the techniques presented may also prove useful for applying other associative memories to visual pattern recognition. The material of this survey is divided into three sections: an overview of biological visual processing; methods of preprocessing (extracting parts of shape, texture, motion, and depth); and shape representation and recognition (form invariance, primitives and structural descriptions, and theories of attention)

A SIFT-Based Fingerprint Verification System Using Cellular Neural Networks

Recently, with the increasing demand of high security, person identification has become more and more important in our everyday life. The purpose of establishing the identity is to ensure that only a legitimate user, and not anyone else, accesses the rendered services. The traditional identification methods are based on “something that you possess ” and “somethin

Conditions for wave trains in spiking neural networks

Spatiotemporal patterns such as traveling waves are frequently observed in recordings of neural activity. The mechanisms underlying the generation of such patterns are largely unknown. Previous studies have investigated the existence and uniqueness of different types of waves or bumps of activity using neural-field models, phenomenological coarse-grained descriptions of neural-network dynamics. But it remains unclear how these insights can be transferred to more biologically realistic networks of spiking neurons, where individual neurons fire irregularly. Here, we employ mean-field theory to reduce a microscopic model of leaky integrate-and-fire (LIF) neurons with distance-dependent connectivity to an effective neural-field model. In contrast to existing phenomenological descriptions, the dynamics in this neural-field model depends on the mean and the variance in the synaptic input, both determining the amplitude and the temporal structure of the resulting effective coupling kernel. For the neural-field model we employ liner stability analysis to derive conditions for the existence of spatial and temporal oscillations and wave trains, that is, temporally and spatially periodic traveling waves. We first prove that wave trains cannot occur in a single homogeneous population of neurons, irrespective of the form of distance dependence of the connection probability. Compatible with the architecture of cortical neural networks, wave trains emerge in two-population networks of excitatory and inhibitory neurons as a combination of delay-induced temporal oscillations and spatial oscillations due to distance-dependent connectivity profiles. Finally, we demonstrate quantitative agreement between predictions of the analytically tractable neural-field model and numerical simulations of both networks of nonlinear rate-based units and networks of LIF neurons.Comment: 36 pages, 8 figures, 4 table

Objekt-Manipulation und Steuerung der Greifkraft durch Verwendung von Taktilen Sensoren

This dissertation describes a new type of tactile sensor and an improved version of the dynamic tactile sensing approach that can provide a regularly updated and accurate estimate of minimum applied forces for use in the control of gripper manipulation. The pre-slip sensing algorithm is proposed and implemented into two-finger robot gripper. An algorithm that can discriminate between types of contact surface and recognize objects at the contact stage is also proposed. A technique for recognizing objects using tactile sensor arrays, and a method based on the quadric surface parameter for classifying grasped objects is described. Tactile arrays can recognize surface types on contact, making it possible for a tactile system to recognize translation, rotation, and scaling of an object independently.Diese Dissertation beschreibt eine neue Art von taktilen Sensoren und einen verbesserten Ansatz zur dynamischen Erfassung von taktilen daten, der in regelmäßigen Zeitabständen eine genaue Bewertung der minimalen Greifkraft liefert, die zur Steuerung des Greifers nötig ist. Ein Berechnungsverfahren zur Voraussage des Schlupfs, das in einen Zwei-Finger-Greifarm eines Roboters eingebaut wurde, wird vorgestellt. Auch ein Algorithmus zur Unterscheidung von verschiedenen Oberflächenarten und zur Erkennung von Objektformen bei der Berührung wird vorgestellt. Ein Verfahren zur Objekterkennung mit Hilfe einer Matrix aus taktilen Sensoren und eine Methode zur Klassifikation ergriffener Objekte, basierend auf den Daten einer rechteckigen Oberfläche, werden beschrieben. Mit Hilfe dieser Matrix können unter schiedliche Arten von Oberflächen bei Berührung erkannt werden, was es für das Tastsystem möglich macht, Verschiebung, Drehung und Größe eines Objektes unabhängig voneinander zu erkennen

Deep Liquid State Machines with Neural Plasticity and On-Device Learning

The Liquid State Machine (LSM) is a recurrent spiking neural network designed for efficient processing of spatio-temporal streams of information. LSMs have several inbuilt features such as robustness, fast training and inference speed, generalizability, continual learning (no catastrophic forgetting), and energy efficiency. These features make LSM’s an ideal network for deploying intelligence on-device. In general, single LSMs are unable to solve complex real-world tasks. Recent literature has shown emergence of hierarchical architectures to support temporal information processing over different time scales. However, these approaches do not typically investigate the optimum topology for communication between layers in the hierarchical network, or assume prior knowledge about the target problem and are not generalizable. In this thesis, a deep Liquid State Machine (deep-LSM) network architecture is proposed. The deep-LSM uses staggered reservoirs to process temporal information on multiple timescales. A key feature of this network is that neural plasticity and attention are embedded in the topology to bolster its performance for complex spatio-temporal tasks. An advantage of the deep-LSM is that it exploits the random projection native to the LSM as well as local plasticity mechanisms to optimize the data transfer between sequential layers. Both random projections and local plasticity mechanisms are ideal for on-device learning due to their low computational complexity and the absence of backpropagating error. The deep-LSM is deployed on a custom learning architecture with memristors to study the feasibility of on-device learning. The performance of the deep-LSM is demonstrated on speech recognition and seizure detection applications

Quantifying joint behavioral states in zebrafish (Danio rerio) dyadic contests through interpretable variables

Tese de Mestrado, Engenharia Biomédica e Biofísica, 2021, Universidade de Lisboa, Faculdade de CiênciasO comportamento animal é uma área fascinante do ponto de vista físico, no entanto ainda existem vários desafios associados à construção de modelos ou ao desenvolvimento de teorias do comportamento em física. Um dos desafios é desenvolver modelos diretamente dos dados, eliminando o viés antropocêntrico que existe na definição de estados comportamentais. Um bom exemplo da complexidade associada ao comportamento pode ser encontrado em interações sociais, nomeadamente interações agonistas entre peixes-zebra (Danio rerio). Estas interações são bem compreendidas e estereotípicas, e existem catálogos a descrever os estados comportamentais associados a cada fase da interação. Isto e a versatilidade genética a que o peixe zebra se encontra associado, tornam esta interação ideal para o nosso estudo. O nosso objetivo principal consiste na tentativa de derivar um conjunto de estados comportamentais diretamente a partir dos dados experimentais obtidos, sendo estes estados definidos para o conjunto, e não individualmente. Fazemos isso sob a assunção de em interações sociais, estados comportamentais dependem dos elementos envolvidos nessa interação (neste caso, são peixes-zebra) e que esta não é completamente descrita, exceto se levar ambos em conta simultaneamente. Os dados são esqueletos tridimensionais dos 2 peixes-zebra num volume. O processo de aquisição desses dados consiste na aquisição de imagens em 3 planos bidimensionais com câmaras de alta definição, e um pipeline de processamento, que combina várias redes neuronais para a identificação de pontos corporais, a atribuição de identidade temporal aos peixes envolvidos, e a interpolação das imagens nos diferentes planos. Este processo permite a conversão de vários vídeos em sinais temporais que podem ser manipulados e processados de forma adequada. Usamos variáveis interpretáveis, no caso, a distância, os alinhamentos de direcção e aceleração, e os ritmos de batimento de cauda. Essas variáveis embora sejam simples, podem dizer bastante informação sobre a natureza do comportamento, sendo úteis numa exploração inicial. Definimos estados comportamentais compostos (colecção de vários comportamentos efectuados pelos peixes ao longo de um determinado período de tempo) e exploramos a dinâmica de uma luta nesta descrição simplificada. O sistema que resulta das variáveis definidas possui 6 dimensões, projectamos esse sistema para um plano bidimensional para melhor análise. Efectua-se um histograma das novas variáveis, e ter uma estimativa da densidade de probabilidade através da convolução do mesmo com uma gaussiana bidimensional. Detecta-se os picos de densidade, que neste sistema podem ser interpretados como estados comportamentais. Com essa descrição é possível gerar uma sequência simbólica que representa a dinâmica da interacção como sendo a transição entre vários estados comportamentais discretos. Constrói-se uma matriz que representa a transição entre os vários estados, e por decomposição espectral pode-se observar o comportamento dos valores próprios em função do número de transições e é possível decompor os estados em vários conjuntos através dos vectores próprios, cuja dinâmica entre eles é representada pelo valor próprio associado. Através da sequência simbólica é possível uma descrição da interacção entre os elementos, tendo inclusive informação sobre a escala temporal associada à dinâmica entre esses estados. Ao associar os clusters aos diferentes estados comportamentais compostos definidos previamente, é possível ver que certos clusters se encontram associados, e apresenta uma certa estrutura, que pode ser representativa da dinâmica real. Também é possível determinar a escala temporal de interações entre diferentes conjuntos de clusters. Foi possível determinar que os comportamentos ocorrem em escalas temporais maiores do que a escala típica para processo de Markov, e a escala temporal mais elevada se encontra associada a transição entre estados associados à agressão entre o par, e estados associados aos períodos entre lutas. Mostramos que é possível obter uma estrutura comportamental da luta entre dois peixes-zebra utilizando as variáveis simples que definimos. Isto é um framework que permite explorar a dinâmica da sua interação em maior detalhe, a utilizar variáveis ou representações mais precisas, que podem não ser interpretáveis

Analog Spiking Neuromorphic Circuits and Systems for Brain- and Nanotechnology-Inspired Cognitive Computing

Human society is now facing grand challenges to satisfy the growing demand for computing power, at the same time, sustain energy consumption. By the end of CMOS technology scaling, innovations are required to tackle the challenges in a radically different way. Inspired by the emerging understanding of the computing occurring in a brain and nanotechnology-enabled biological plausible synaptic plasticity, neuromorphic computing architectures are being investigated. Such a neuromorphic chip that combines CMOS analog spiking neurons and nanoscale resistive random-access memory (RRAM) using as electronics synapses can provide massive neural network parallelism, high density and online learning capability, and hence, paves the path towards a promising solution to future energy-efficient real-time computing systems. However, existing silicon neuron approaches are designed to faithfully reproduce biological neuron dynamics, and hence they are incompatible with the RRAM synapses, or require extensive peripheral circuitry to modulate a synapse, and are thus deficient in learning capability. As a result, they eliminate most of the density advantages gained by the adoption of nanoscale devices, and fail to realize a functional computing system. This dissertation describes novel hardware architectures and neuron circuit designs that synergistically assemble the fundamental and significant elements for brain-inspired computing. Versatile CMOS spiking neurons that combine integrate-and-fire, passive dense RRAM synapses drive capability, dynamic biasing for adaptive power consumption, in situ spike-timing dependent plasticity (STDP) and competitive learning in compact integrated circuit modules are presented. Real-world pattern learning and recognition tasks using the proposed architecture were demonstrated with circuit-level simulations. A test chip was implemented and fabricated to verify the proposed CMOS neuron and hardware architecture, and the subsequent chip measurement results successfully proved the idea. The work described in this dissertation realizes a key building block for large-scale integration of spiking neural network hardware, and then, serves as a step-stone for the building of next-generation energy-efficient brain-inspired cognitive computing systems