FPGA-based multi-view stereo system with flexible measurement setup

In recent years, stereoscopic image processing algorithms have gained importance for a variety of applications. To capture larger measurement volumes, multiple stereo systems are combined into a multi-view stereo (MVS) system. To reduce the amount of data and the data rate, calculation steps close to the sensors are outsourced to Field Programmable Gate Arrays (FPGAs) as upstream computing units. The calculation steps include lens distortion correction, rectification and stereo matching. In this paper a FPGA-based MVS system with flexible camera arrangement and partly overlapping field of view is presented. The system consists of four FPGA-based passive stereoscopic systems (Xilinx Zynq-7000 7020 SoC, EV76C570 CMOS sensor) and a downstream processing unit (Zynq Ultrascale ZU9EG SoC). This synchronizes the sensor near processing modules and receives the disparity maps with corresponding left camera image via HDMI. The subsequent computing unit calculates a coherent 3D point cloud. Our developed FPGA-based 3D measurement system captures a large measurement volume at 24 fps by combining a multiple view with eight cameras (using Semi-Global Matching for an image size of 640 px × 460 px, up to 256 px disparity range and with aggregated costs over 4 directions). The capabilities and limitation of the system are shown by an application example with optical non-cooperative surface

Investigation into the implementation of a multimodal 3D measurement system for a forestry harvesting process

In the context of digitalization, monitoring and traceability are also becoming increasingly important in the forestry sector. An essential component of the most efficient value creation is the recording of relevant characteristics right from the start. The optical and tactile recording of characteristics, such as diameter and volume, have been solved to a large extent in the harvesting of heavy timber, but differs significantly from that of small timber. This paper is about an investigation on the implementation of a multimodal 3D sensor system, which is used for the stable detection of biomass directly in the harvesting process of weak wood. System technical possibilities are shown how biomass can be determined directly during the harvesting process by means of multimodal 3D measurement technology. Considerations regarding possible measurement principles and methods result in two methods, which are discussed within this thesis regarding their advantages and disadvantages. The development stages are presented in detail up to the practical tests, which also includes the acquisition of empirical a priori information. Finally, data are determined by means of test scenarios, which prove the principle functionality and make the methods evaluable

SoC-based real-time passive stereo image processing implementation and optimization

Stereo Image Processing as a part of three dimensional image processing become more and more important for industrial measuring, quality assurance and industrial automation. While classical image processing get it features from an image plane, additional information is obtained in direction of the optical axis. In comparison to active stereo methods, which need a projector or laser source and scanning device, passive stereo need at minimum two images from different perspectives. The paper starts with the basics of passive stereo, required optical setup and electronics. Some Information about the implementation of a stereo IP core in the used Xilinx SoC FPGA embedded system given. The program flow in ARM core and FPGA is illustrated. To get a high performance image processing system, the optimization of the parameters and the implementation settings on the used FPGA is very important. A comparison of several core parameter setups is done. Finally, some ways for further optimization with new hardware technologies are given

Investigation on chromatic aberrations and its potential for application in depth measurement

Chromatic aberrations are mostly supressed in typical industrial lenses to ensure a well-defined colour image. Beside the main wavelengths which are recognizable for the human eye, multispectral cameras can sample with a more detailed chromatic (spectral) resolution. This leads to a partial wavelength depended unsharpness in the images. If the imaging system can be well defined out of these imperfections, a depth information can be calculated. In this paper the theoretical model as well as a method for the 3D-reconstruction out of different colour (spectral) channels will be discussed

Investigations on the potential application of machine vision lenses for depth measurement by exploiting chromatic aberrations

Chromatic (spectral) aberrations are image imperfections that are disadvantageous for standard image processing tasks and are typically compensated through the application of different types of glass during lens design. The longitudinal chromatic aberration causes a relative unsharpness over different spectral channels. Since this error is corrected in most multi-chromatic lenses, this paper investigates to which extent the shift of the focal planes in a standard lens can be used specifically for image processing applications. Theoretical investigations of the longitudinal chromatic aberration are carried out. Based on this, conditions and a method to generate a 3D depth reconstruction out of different spectral channels are presented

Smart parallel spectral imager based on heterogeneous FPGA system on chip

In the last years, industrial image processing has been shifting to areas and tasks that are increasing in complexity. This results in new challenges in order to contrast features to be detected or evaluated. Systems for the acquisition and interpretation of multispectral images are thus becoming more and more interesting. A major issue is, depending on the sensor principle, the time to acquire this spectral data. FPGA (Field Programmable Gate Array) and in particular heterogeneous FPGA SoC (System-on-Chip) offer the possibility to accelerate these acquisition methods decisively. In addition to the image acquisition, it is also possible to calculate decisive preprocessing steps in the hardware. A frequently used algorithm for analyzing but also compressing hyperspectral data is the PCA (principal component analysis). This paper presents a research setup that combines a heterogeneous FPGA SoC with a 12-channel filter wheel camera. With the help of the device a parallel working PCA is to be integrated, which works distributed in hardware and software. The paper presents the concept for this implementation and the current state of development in the project. In addition, restrictions on the development of algorithms with hardware systems and the current distribution in hardware and software are discussed