Design of an FPGA-based smart camera and its application towards object tracking : a thesis presented in partial fulfilment of the requirements for the degree of Master of Engineering in Electronics and Computer Engineering at Massey University, Manawatu, New Zealand

Smart cameras and hardware image processing are not new concepts, yet despite the fact both have existed several decades, not much literature has been presented on the design and development process of hardware based smart cameras. This thesis will examine and demonstrate the principles needed to develop a smart camera on hardware, based on the experiences from developing an FPGA-based smart camera. The smart camera is applied on a Terasic DE0 FPGA development board, using Terasic’s 5 megapixel GPIO camera. The algorithm operates at 120 frames per second at a resolution of 640x480 by utilising a modular streaming approach. Two case studies will be explored in order to demonstrate the development techniques established in this thesis. The first case study will develop the global vision system for a robot soccer implementation. The algorithm will identify and calculate the positions and orientations of each robot and the ball. Like many robot soccer implementations each robot has colour patches on top to identify each robot and aid finding its orientation. The ball is comprised of a single solid colour that is completely distinct from the colour patches. Due to the presence of uneven light levels a YUV-like colour space labelled YC1C2 is used in order to make the colour values more light invariant. The colours are then classified using a connected components algorithm to segment the colour patches. The shapes of the classified patches are then used to identify the individual robots, and a CORDIC function is used to calculate the orientation. The second case study will investigate an improved colour segmentation design. A new HSY colour space is developed by remapping the Cartesian coordinate system from the YC1C2 to a polar coordinate system. This provides improved colour segmentation results by allowing for variations in colour value caused by uneven light patterns and changing light levels

SVITE: A Spike-Based VITE Neuro-Inspired Robot Controller

This paper presents an implementation of a neuro-inspired algorithm called VITE (Vector Integration To End Point) in FPGA in the spikes domain. VITE aims to generate a non-planned trajectory for reaching tasks in robots. The algorithm has been adapted to work completely in the spike domain under Simulink simulations. The FPGA implementation consists in 4 VITE in parallel for controlling a 4-degree-of-freedom stereo-vision robot. This work represents the main layer of a complex spike-based architecture for robot neuro-inspired reaching tasks in FPGAs. It has been implemented in two Xilinx FPGA families: Virtex-5 and Spartan-6. Resources consumption comparative between both devices is presented. Results obtained for Spartan device could allow controlling complex robotic structures with up to 96 degrees of freedom per FPGA, providing, in parallel, high speed connectivity with other neuromorphic systems sending movement references. An exponential and gamma distribution test over the inter spike interval has been performed to proof the approach to the neural code proposed.Ministerio de Economía y Competitividad TEC2012-37868-C04-0

FPGA-based module for SURF extraction

We present a complete hardware and software solution of an FPGA-based computer vision embedded module capable of carrying out SURF image features extraction algorithm. Aside from image analysis, the module embeds a Linux distribution that allows to run programs specifically tailored for particular applications. The module is based on a Virtex-5 FXT FPGA which features powerful configurable logic and an embedded PowerPC processor. We describe the module hardware as well as the custom FPGA image processing cores that implement the algorithm's most computationally expensive process, the interest point detection. The module's overall performance is evaluated and compared to CPU and GPU based solutions. Results show that the embedded module achieves comparable disctinctiveness to the SURF software implementation running in a standard CPU while being faster and consuming significantly less power and space. Thus, it allows to use the SURF algorithm in applications with power and spatial constraints, such as autonomous navigation of small mobile robots

AER Neuro-Inspired interface to Anthropomorphic Robotic Hand

Address-Event-Representation (AER) is a communication protocol for transferring asynchronous events between VLSI chips, originally developed for neuro-inspired processing systems (for example, image processing). Such systems may consist of a complicated hierarchical structure with many chips that transmit data among them in real time, while performing some processing (for example, convolutions). The information transmitted is a sequence of spikes coded using high speed digital buses. These multi-layer and multi-chip AER systems perform actually not only image processing, but also audio processing, filtering, learning, locomotion, etc. This paper present an AER interface for controlling an anthropomorphic robotic hand with a neuro-inspired system.Unión Europea IST-2001-34124 (CAVIAR)Ministerio de Ciencia y Tecnología TIC-2003-08164-C03-02Ministerio de Ciencia y Tecnología TIC2000-0406-P4- 0

An FPGA Architecture and CAD Flow Supporting Dynamically Controlled Power Gating

© 2015 IEEE.Leakage power is an important component of the total power consumption in field-programmable gate arrays (FPGAs) built using 90-nm and smaller technology nodes. Power gating was shown to be effective at reducing the leakage power. Previous techniques focus on turning OFF unused FPGA resources at configuration time; the benefit of this approach depends on resource utilization. In this paper, we present an FPGA architecture that enables dynamically controlled power gating, in which FPGA resources can be selectively powered down at run-time. This could lead to significant overall energy savings for applications having modules with long idle times. We also present a CAD flow that can be used to map applications to the proposed architecture. We study the area and power tradeoffs by varying the different FPGA architecture parameters and power gating granularity. The proposed CAD flow is used to map a set of benchmark circuits that have multiple power-gated modules to the proposed architecture. Power savings of up to 83% are achievable for these circuits. Finally, we study a control system of a robot that is used in endoscopy. Using the proposed architecture combined with clock gating results in up to 19% energy savings in this application

From Vision Sensor to Actuators, Spike Based Robot Control through Address-Event-Representation

One field of the neuroscience is the neuroinformatic whose aim is to develop auto-reconfigurable systems that mimic the human body and brain. In this paper we present a neuro-inspired spike based mobile robot. From commercial cheap vision sensors converted into spike information, through spike filtering for object recognition, to spike based motor control models. A two wheel mobile robot powered by DC motors can be autonomously controlled to follow a line drown in the floor. This spike system has been developed around the well-known Address-Event-Representation mechanism to communicate the different neuro-inspired layers of the system. RTC lab has developed all the components presented in this work, from the vision sensor, to the robot platform and the FPGA based platforms for AER processing.Ministerio de Ciencia e Innovación TEC2006-11730-C03-02Junta de Andalucía P06-TIC-0141

A modified model for the Lobula Giant Movement Detector and its FPGA implementation

The Lobula Giant Movement Detector (LGMD) is a wide-field visual neuron located in the Lobula layer of the Locust nervous system. The LGMD increases its firing rate in response to both the velocity of an approaching object and the proximity of this object. It has been found that it can respond to looming stimuli very quickly and trigger avoidance reactions. It has been successfully applied in visual collision avoidance systems for vehicles and robots. This paper introduces a modified neural model for LGMD that provides additional depth direction information for the movement. The proposed model retains the simplicity of the previous model by adding only a few new cells. It has been simplified and implemented on a Field Programmable Gate Array (FPGA), taking advantage of the inherent parallelism exhibited by the LGMD, and tested on real-time video streams. Experimental results demonstrate the effectiveness as a fast motion detector

An AER-Based Actuator Interface for Controlling an Anthropomorphic Robotic Hand

Bio-Inspired and Neuro-Inspired systems or circuits are a relatively novel approaches to solve real problems by mimicking the biology in its efficient solutions. Robotic also tries to mimic the biology and more particularly the human body structure and efficiency of the muscles, bones, articulations, etc. Address-Event-Representation (AER) is a communication protocol for transferring asynchronous events between VLSI chips, originally developed for neuro-inspired processing systems (for example, image processing). Such systems may consist of a complicated hierarchical structure with many chips that transmit data among them in real time, while performing some processing (for example, convolutions). The information transmitted is a sequence of spikes coded using high speed digital buses. These multi-layer and multi-chip AER systems perform actually not only image processing, but also audio processing, filtering, learning, locomotion, etc. This paper present an AER interface for controlling an anthropomorphic robotic hand with a neuro-inspired system.Unión Europea IST-2001-34124 (CAVIAR)Ministerio de Ciencia y Tecnología TIC-2003-08164-C03-0