A Real-Time Stereo SmartCam, using FPGA, SIMD and VLIW

Abstract

Abstract. This paper describes the architecture of a SmartCamera, applied for real-time dense stereo vision. The system is based on two TriMedia SmartCameras, an FPGA for low level image pre- and postprocessing, a single chip SIMD array for disparity calculation for all epipolar lines in parallel, and a VLIW processor for further processing of the image processing applications at hand. The FPGA is the heart of the system, its architecture is a synchronous ring with time interleaved dataslots, enabling virtually parallel image channels. We have simplified the development of image processing functions by creating libraries and an automated compile flow, allowing easy debugging and coefficient tweaking of the functions and run-time controllability for a per-frame image processing sub-task schedular. We have implemented a dense stereo vision algorithm on the platform and measured a 40x processing speedup compared to the same implementation on an existing SmartCamera