Digital processor array implementation aspects of a 3D multi-layer vision architecture

Trabajo presentado al 12th CNNA celebrado en Berkeley (USA) del 3 al 5 de febrero de 2010.Technological aspects of the 3D integration of a multilayer combined mixed-signal and digital sensor-processor array chip is described. The 3D integration raises the question of signal routing, power distribution, and heat dissipation, which aspects are considered systematically in the digital processor array layer as part of the multi layer structure. We have developed a linear programming based evaluation system to identify the proper architecture and its parameters.The work is supported by the Eutecus ONR-BAA Co. Num N00173-08-C-4005 VISCUBE project.Peer Reviewe

Focal-plane generation of multi-resolution and multi-scale image representation for low-power vision applications

Early vision stages represent a considerably heavy computational load. A huge amount of data needs to be processed under strict timing and power requirements. Conventional architectures usually fail to adhere to the specifications in many application fields, especially when autonomous vision-enabled devices are to be implemented, like in lightweight UAVs, robotics or wireless sensor networks. A bioinspired architectural approach can be employed consisting of a hierarchical division of the processing chain, conveying the highest computational demand to the focal plane. There, distributed processing elements, concurrent with the photosensitive devices, influence the image capture and generate a pre-processed representation of the scene where only the information of interest for subsequent stages remains. These focal-plane operators are implemented by analog building blocks, which may individually be a little imprecise, but as a whole render the appropriate image processing very efficiently. As a proof of concept, we have developed a 176x144-pixel smart CMOS imager that delivers lighter but enriched representations of the scene. Each pixel of the array contains a photosensor and some switches and weighted paths allowing reconfigurable resolution and spatial filtering. An energy-based image representation is also supported. These functionalities greatly simplify the operation of the subsequent digital processor implementing the high level logic of the vision algorithm. The resulting figures, 5.6m W@30fps, permit the integration of the smart image sensor with a wireless interface module (Imote2 from Memsic Corp.) for the development of vision-enabled WSN applications.Junta de Andalucía 2006-TIC-2352Ministerio de Ciencia e Innovación TEC 2009-11812Office of Naval Research (USA) N00014111031

3D multi-layer vision architecture for surveillance and reconnaissance applications

The architecture and the design details of a multilayer combined mixed-signal and digital sensor-processor array chip is shown. The processor layers are fabricated with 3D integration technology, and the sensor layer is integrated via bump bonding technology. The chip is constructed of a 320 x 240 sensor array layer, closely coupled with a 160 x 120 mixed-signal processor array layer, a digital frame buffer layer, and an 8 x 8 digital fovea processor array layer. The chip is designed to solve image registration and feature extraction above 1000FPS.Office of Naval Research (USA) N00173-08-C-400

Focal-plane generation of multi-resolution and multi-scale image representation for low-power vision applications

Comunicación presentada al "XXXVII Infrared Technology and Applications" celebrado en Orlando (USA) el 25 de Abril del 2011.Early vision stages represent a considerably heavy computational load. A huge amount of data needs to be processed under strict timing and power requirements. Conventional architectures usually fail to adhere to the specifications in many application fields, especially when autonomous vision-enabled devices are to be implemented, like in lightweight UAVs, robotics or wireless sensor networks. A bioinspired architectural approach can be employed consisting of a hierarchical division of the processing chain, conveying the highest computational demand to the focal plane. There, distributed processing elements, concurrent with the photosensitive devices, influence the image capture and generate a pre-processed representation of the scene where only the information of interest for subsequent stages remains. These focal-plane operators are implemented by analog building blocks, which may individually be a little imprecise, but as a whole render the appropriate image processing very efficiently. As a proof of concept, we have developed a 176x144-pixel smart CMOS imager that delivers lighter but enriched representations of the scene. Each pixel of the array contains a photosensor and some switches and weighted paths allowing reconfigurable resolution and spatial filtering. An energy-based image representation is also supported. These functionalities greatly simplify the operation of the subsequent digital processor implementing the high level logic of the vision algorithm. The resulting figures, 5.6m W@30fps, permit the integration of the smart image sensor with a wireless interface module (Imote2 from Memsic Corp.) for the development of vision-enabled WSN applications.This work is partially funded by the Andalusian regional government (Junta de Andalucía-CICE) through project 2006-TIC-2352 and the Spanish Ministry of Science (MICINN) through project TEC 2009-11812, co-funded by the European Regional Development Fund, and also supported by the Office of Naval Research (USA), through grant N000141110312.Peer Reviewe

Multi-resolution low-power Gaussian filtering by reconfigurable focal-plane binning

Gaussian filtering is a basic tool for image processing. Noise reduction, scale-space generation or edge detection are examples of tasks where different Gaussian filters can be successfully utilized. However, their implementation in a conventional digital processor by applying a convolution kernel throughout the image is quite inefficient. Not only the value of every single pixel is taken into consideration sucessively, but also contributions from their neighbors need to be taken into account. Processing of the frame is serialized and memory access is intensive and recurrent. The result is a low operation speed or, alternatively, a high power consumption. This inefficiency is specially remarkable for filters with large variance, as the kernel size increases significantly. In this paper, a different approach to achieve Gaussian filtering is proposed. It is oriented to applications with very low power budgets. The key point is a reconfigurable focal-plane binning. Pixels are grouped according to the targeted resolution by means of a division grid. Then, two consecutive shifts of this grid in opposite directions carry out the spread of information to the neighborhood of each pixel in parallel. The outcome is equivalent to the application of a 3×3 binomial filter kernel, which in turns is a good approximation of a Gaussian filter, on the original image. The variance of the closest Gaussian filter is around 0.5. By repeating the operation, Gaussian filters with larger variances can be achieved. A rough estimation of the necessary energy for each repetition until reaching the desired filter is below 20nJ for a QCIF-size array. Finally, experimental results of a QCIF proofof- concept focal-plane array manufactured in 0.35μm CMOS technology are presented. A maximum RMSE of only 1.2% is obtained by the on-chip Gaussian filtering with respect to the corresponding equivalent ideal filter implemented off-chip.Junta de Andalucía 2006-TIC-2352Ministerio de Ciencia e Innovación TEC 2009-11812Office of Naval Research (USA) N00014111031

Demo: Real-time remote reporting of active regions with Wi-FLIP

This paper describes a real-time application programmed into Wi-FLIP, a wireless smart camera resulting from the integration of FLIP-Q, a focal-plane low-power image processor, and Imote2, a commercial WSN platform. The application, though simple, shows the potentiality of the reduced scene representations achievable at FLIP-Q to speed up the processing. It consists of detecting the active regions within the scene being surveyed, that is, those regions undergoing thresholded variations with respect to the background. If an activity pattern is prescribed, FLIP-Q enables the reconfigurability of the image plane accordingly, making its detection and tracking easier. For each frame, the number of active regions is calculated and wirelessly reported in real time. A base station picks up the radio signal and sends the information to a PC via USB, also in real time. Frame rates up to around 10fps have been achieved, although it greatly depends on the light conditions and the image plane division grid.Junta de Andalucía 2006-TIC-2352Ministerio de Ciencia TEC 2009-11812Office of Naval Research (USA) N00014111031

Demo: Real-time remote reporting of active regions with Wi-FLIP

Trabajo presentado al Fifth ICDSC celebrado en Ghent del 22 al 25 de agosto de 2011.This paper describes a real-time application programmed into Wi-FLIP, a wireless smart camera resulting from the integration of FLIP-Q, a focal-plane low-power image processor, and Imote2, a commercial WSN platform. The application, though simple, shows the potentiality of the reduced scene representations achievable at FLIP-Q to speed up the processing. It consists of detecting the active regions within the scene being surveyed, that is, those regions undergoing thresholded variations with respect to the background. If an activity pattern is prescribed, FLIP-Q enables the reconfigurability of the image plane accordingly, making its detection and tracking easier. For each frame, the number of active regions is calculated and wirelessly reported in real time. A base station picks up the radio signal and sends the information to a PC via USB, also in real time. Frame rates up to around 10fps have been achieved, although it greatly depends on the light conditions and the image plane division grid.This work is partially funded by the Andalusian regional government through project 2006-TIC-2352, the Spanish Ministry of Science through project TEC 2009-11812, co-funded by the European Regional Development Fund, and by the Office of Naval Research (USA), through grant N000141110312.Peer Reviewe

3D multi-layer vision architecture for surveillance and reconnaissance applications

Trabajo presentado al ECCTD celebrado en Antalya (Turquia) del 23 al 27 de agosto de 2009.The architecture and the design details of a multilayer combined mixed-signal and digital sensor-processor array chip is shown. The processor layers are fabricated with 3D integration technology, and the sensor layer is integrated via bump bonding technology. The chip is constructed of a 320 x 240 sensor array layer, closely coupled with a 160 x 120 mixed-signal processor array layer, a digital frame buffer layer, and an 8 x 8 digital fovea processor array layer. The chip is designed to solve image registration and feature extraction above 1000FPS.The work is supported by the Eutecus ONR-BAA Co. Num N00173-08-C-4005 VISCUBE project.Peer Reviewe