Adaptive ON-OFF spiking photoreceptor

Accepted versio

Tasking networked CCTV cameras and mobile phones to identify and localize multiple people

We present a method to identify and localize people by leveraging existing CCTV camera infrastructure along with inertial sensors (accelerometer and magnetometer) within each person’s mobile phones. Since a person’s motion path, as observed by the camera, must match the local motion measurements from their phone, we are able to uniquely identify people with the phones ’ IDs by detecting the statistical dependence between the phone and camera measurements. For this, we express the problem as consisting of a twomeasurement HMM for each person, with one camera measurement and one phone measurement. Then we use a maximum a posteriori formulation to find the most likely ID assignments. Through sensor fusion, our method largely bypasses the motion correspondence problem from computer vision and is able to track people across large spatial or temporal gaps in sensing. We evaluate the system through simulations and experiments in a real camera network testbed

Interfacing of neuromorphic vision, auditory and olfactory sensors with digital neuromorphic circuits

The conventional Von Neumann architecture imposes strict constraints on the development of intelligent adaptive systems. The requirements of substantial computing power to process and analyse complex data make such an approach impractical to be used in implementing smart systems. Neuromorphic engineering has produced promising results in applications such as electronic sensing, networking architectures and complex data processing. This interdisciplinary field takes inspiration from neurobiological architecture and emulates these characteristics using analogue Very Large Scale Integration (VLSI). The unconventional approach of exploiting the non-linear current characteristics of transistors has aided in the development of low-power adaptive systems that can be implemented in intelligent systems. The neuromorphic approach is widely applied in electronic sensing, particularly in vision, auditory, tactile and olfactory sensors. While conventional sensors generate a huge amount of redundant output data, neuromorphic sensors implement the biological concept of spike-based output to generate sparse output data that corresponds to a certain sensing event. The operation principle applied in these sensors supports reduced power consumption with operating efficiency comparable to conventional sensors. Although neuromorphic sensors such as Dynamic Vision Sensor (DVS), Dynamic and Active pixel Vision Sensor (DAVIS) and AEREAR2 are steadily expanding their scope of application in real-world systems, the lack of spike-based data processing algorithms and complex interfacing methods restricts its applications in low-cost standalone autonomous systems. This research addresses the issue of interfacing between neuromorphic sensors and digital neuromorphic circuits. Current interfacing methods of these sensors are dependent on computers for output data processing. This approach restricts the portability of these sensors, limits their application in a standalone system and increases the overall cost of such systems. The proposed methodology simplifies the interfacing of these sensors with digital neuromorphic processors by utilizing AER communication protocols and neuromorphic hardware developed under the Convolution AER Vision Architecture for Real-time (CAVIAR) project. The proposed interface is simulated using a JAVA model that emulates a typical spikebased output of a neuromorphic sensor, in this case an olfactory sensor, and functions that process this data based on supervised learning. The successful implementation of this simulation suggests that the methodology is a practical solution and can be implemented in hardware. The JAVA simulation is compared to a similar model developed in Nengo, a standard large-scale neural simulation tool. The successful completion of this research contributes towards expanding the scope of application of neuromorphic sensors in standalone intelligent systems. The easy interfacing method proposed in this thesis promotes the portability of these sensors by eliminating the dependency on computers for output data processing. The inclusion of neuromorphic Field Programmable Gate Array (FPGA) board allows reconfiguration and deployment of learning algorithms to implement adaptable systems. These low-power systems can be widely applied in biosecurity and environmental monitoring. With this thesis, we suggest directions for future research in neuromorphic standalone systems based on neuromorphic olfaction

High speed event-based visual processing in the presence of noise

Standard machine vision approaches are challenged in applications where large amounts of noisy temporal data must be processed in real-time. This work aims to develop neuromorphic event-based processing systems for such challenging, high-noise environments. The novel event-based application-focused algorithms developed are primarily designed for implementation in digital neuromorphic hardware with a focus on noise robustness, ease of implementation, operationally useful ancillary signals and processing speed in embedded systems

Traitement d'images bas niveau intégré dans un capteur de vision CMOS

Le traitement d images classique est basé sur l évaluation des données délivrées par un système à basede capteur de vision sous forme d images. L information lumineuse captée est extraiteséquentiellement de chaque élément photosensible (pixel) de la matrice avec un certain cadencementet à fréquence fixe. Ces données, une fois mémorisées, forment une matrice de données qui estréactualisée de manière exhaustive à l arrivée de chaque nouvelle image. De fait, Pour des capteurs àforte résolution, le volume de données à gérer est extrêmement important. De plus, le système neprend pas en compte le fait que l information stockée ai changé ou non par rapport à l imageprécédente. Cette probabilité est, en effet, assez importante. Ceci nous mène donc, selon l activité de la scène filmée à un haut niveau de redondances temporelles. De même, la méthode de lectureusuelle ne prend pas en compte le fait que le pixel en phase de lecture a la même valeur ou non que lepixel voisin lu juste avant. Cela rajoute aux redondances temporelles un taux de redondances spatialesplus ou moins élevé selon le spectre de fréquences spatiales de la scène filmée. Dans cette thèse, nousavons développé plusieurs solutions qui visent contrôler le flot de données en sortie de l imageur enessayant de réduire les redondances spatiales et temporelles des pixels. Les contraintes de simplicité etd intelligence des techniques de lecture développées font la différence entre ce que nousprésentons et ce qui a été publié dans la littérature. En effet, les travaux présentés dans l état de l artproposent des solutions à cette problématique, qui en général, exigent de gros sacrifices en terme desurface du pixel, vu qu elles implémentent des fonctions électroniques complexes in situ.Les principes de fonctionnement, les émulations sous MATLAB, la conception et les simulationsélectriques ainsi que les résultats expérimentaux des techniques proposées sont présentés en détailsdans ce manuscrit.The classical image processing is based on the evaluation of data delivered by a vision sensor systemas images. The captured light information is extracted sequentially from each photosensitive element(pixel) of the matrix with a fixed frequency called frame rate. These data, once stored, form a matrixof data that is entirely updated at the acquisition of each new image. Therefore, for high resolutionimagers, the data flow is huge. Moreover, the conventional systems do not take into account the factthat the stored data have changed or not compared to the previously acquired image. Indeed, there is ahigh probability that this information is not changed. Therefore, this leads, depending on the "activity"of the filmed scene, to a high level of temporal redundancies. Similarly, the usual scanning methodsdo not take into account that the read pixel has or not the same value of his neighbor pixel read oncebefore. This adds to the temporal redundancies, spatial redundancies rate that depends on the spatialfrequency spectrum of the scene. In this thesis, we have developed several solutions that aim to controlthe output data flow from the imager trying to reduce both spatial and temporal pixels redundancies. Aconstraint of simplicity and "Smartness" of the developed readout techniques makes the differencebetween what we present and what has been published in the literature. Indeed, the works presented inthe literature suggest several solutions to this problem, but in general, these solutions require largesacrifices in terms of pixel area, since they implement complex electronic functions in situ.The operating principles, the emulation in MATLAB, the electrical design and simulations and theexperimental results of the proposed techniques are explained in detail in this manuscriptSAVOIE-SCD - Bib.électronique (730659901) / SudocGRENOBLE1/INP-Bib.électronique (384210012) / SudocGRENOBLE2/3-Bib.électronique (384219901) / SudocSudocFranceF