An overview of the methods of synthesis, realization and implementation of orthogonal 3-D rotation filters and possibilities of further research and development

In the paper, an overview of the methods and algorithms of synthesis, realization and implementation used by the author to obtain orthogonal 3-D filters with a structure made of Givens rotations has been presented. The main advantage of orthogonal filters, which may have a lower sensitivity to quantization of the coefficients, was indicated. The author proposed a number of possible changes and modifications of individual stages, which may result in obtaining filters with even better parameters. The work will be the basis for the direction of further research

Hardware Based Scale- and Rotation-Invariant Feature Extraction: A Retrospective Analysis and Future Directions

Computer Vision techniques represent a class of algorithms that are highly computation and data intensive in nature. Generally, performance of these algorithms in terms of execution speed on desktop computers is far from real-time. Since real-time performance is desirable in many applications, special-purpose hardware is required in most cases to achieve this goal. Scale- and rotation-invariant local feature extraction is a low level computer vision task with very high computational complexity. The state-of-the-art algorithms that currently exist in this domain, like SIFT and SURF, suffer from slow execution speeds and at best can only achieve rates of 2-3 Hz on modern desktop computers. Hardware-based scale- and rotation-invariant local feature extraction is an emerging trend enabling real-time performance for these computationally complex algorithms. This paper takes a retrospective look at the advances made so far in this field, discusses the hardware design strategies employed and results achieved, identifies current research gaps and suggests future research directions

A high-performance hardware architecture of an image matching system based on the optimised SIFT algorithm

The Scale Invariant Feature Transform (SIFT) is one of the most popular matching algorithms in the field of computer vision. It takes over many other algorithms because features detected are fully invariant to image scaling and rotation, and are also shown to be robust to changes in 3D viewpoint, addition of noise, changes in illumination and a sustainable range of affine distortion. However, the computational complexity is high, which prevents it from achieving real-time. The aim of this project, therefore, is to develop a high-performance image matching system based on the optimised SIFT algorithm to perform real-time feature detection, description and matching. This thesis presents the stages of the development of the system. To reduce the computational complexity, an alternative to the grid layout of standard SIFT is proposed, which is termed as SRI-DASIY (Scale and Rotation Invariant DAISY). The SRI-DAISY achieves comparable performance with the standard SIFT descriptor, but is more efficient to be implemented using hardware, in terms of both computational complexity and memory usage. The design takes only 7.57 µs to generate a descriptor with a system frequency of 100 MHz, which is equivalent to approximately 132,100 descriptors per second and is of the highest throughput when compared with existing designs. Besides, a novel keypoint matching strategy is also presented in this thesis, which achieves higher precision than the widely applied distance ratio based matching and is computationally more efficient. All phases of the SIFT algorithm have been investigated, including feature detection, descriptor generation and descriptor matching. The characterisation of each individual part of the design is carried out and compared with the software simulation results. A fully stand-alone image matching system has been developed that consists of a CMOS camera front-end for image capture, a SIFT processing core embedded in a Field Programmable Logic Array (FPGA) device, and a USB back-end for data transfer. Experiments are conducted by using real-world images to verify the system performance. The system has been tested by integrating into two practical applications. The resulting image matching system eliminates the bottlenecks that limit the overall throughput of the system, and hence allowing the system to process images in real-time without interruption. The design can be modified to adapt to the applications processing images with higher resolution and is still able to achieve real-time

A high-performance hardware architecture of an image matching system based on the optimised SIFT algorithm

The Scale Invariant Feature Transform (SIFT) is one of the most popular matching algorithms in the field of computer vision. It takes over many other algorithms because features detected are fully invariant to image scaling and rotation, and are also shown to be robust to changes in 3D viewpoint, addition of noise, changes in illumination and a sustainable range of affine distortion. However, the computational complexity is high, which prevents it from achieving real-time. The aim of this project, therefore, is to develop a high-performance image matching system based on the optimised SIFT algorithm to perform real-time feature detection, description and matching. This thesis presents the stages of the development of the system. To reduce the computational complexity, an alternative to the grid layout of standard SIFT is proposed, which is termed as SRI-DASIY (Scale and Rotation Invariant DAISY). The SRI-DAISY achieves comparable performance with the standard SIFT descriptor, but is more efficient to be implemented using hardware, in terms of both computational complexity and memory usage. The design takes only 7.57 µs to generate a descriptor with a system frequency of 100 MHz, which is equivalent to approximately 132,100 descriptors per second and is of the highest throughput when compared with existing designs. Besides, a novel keypoint matching strategy is also presented in this thesis, which achieves higher precision than the widely applied distance ratio based matching and is computationally more efficient. All phases of the SIFT algorithm have been investigated, including feature detection, descriptor generation and descriptor matching. The characterisation of each individual part of the design is carried out and compared with the software simulation results. A fully stand-alone image matching system has been developed that consists of a CMOS camera front-end for image capture, a SIFT processing core embedded in a Field Programmable Logic Array (FPGA) device, and a USB back-end for data transfer. Experiments are conducted by using real-world images to verify the system performance. The system has been tested by integrating into two practical applications. The resulting image matching system eliminates the bottlenecks that limit the overall throughput of the system, and hence allowing the system to process images in real-time without interruption. The design can be modified to adapt to the applications processing images with higher resolution and is still able to achieve real-time

Front-end receiver for miniaturised ultrasound imaging

Point of care ultrasonography has been the focus of extensive research over the past few decades. Miniaturised, wireless systems have been envisaged for new application areas, such as capsule endoscopy, implantable ultrasound and wearable ultrasound. The hardware constraints of such small-scale systems are severe, and tradeoffs between power consumption, size, data bandwidth and cost must be carefully balanced. To address these challenges, two synthetic aperture receiver architectures are proposed and compared. The architectures target highly miniaturised, low cost, B-mode ultrasound imaging systems. The first architecture utilises quadrature (I/Q) sampling to minimise the signal bandwidth and computational load. Synthetic aperture beamforming is carried out using a single-channel, pipelined protocol in order to minimise system complexity and power consumption. A digital beamformer dynamically apodises and focuses the data by interpolating and applying complex phase rotations to the I/Q samples. The beamformer is implemented on a Spartan-6 FPGA and consumes 296mW for a frame rate of 7Hz. The second architecture employs compressive sensing within the finite rate of innovation (FRI) framework to further reduce the data bandwidth. Signals are sampled below the Nyquist frequency, and then transmitted to a digital back-end processor, which reconstructs I/Q components non-linearly, and then carries out synthetic aperture beamforming. Both architectures were tested in hardware using a single-channel analogue front-end (AFE) that was designed and fabricated in AMS 0.35μm CMOS. The AFE demodulates RF ultrasound signals sequentially into I/Q components, and comprises a low-noise preamplifier, mixer, programmable gain amplifier (PGA) and lowpass filter. A variable gain low noise preamplifier topology is used to enable quasi-exponential time-gain control (TGC). The PGA enables digital selection of three gain values (15dB, 22dB and 25.5dB). The bandwidth of the lowpass filter is also selectable between 1.85MHz, 510kHz and 195kHz to allow for testing of both architectural frameworks. The entire AFE consumes 7.8 mW and occupies an area of 1.5×1.5 mm. In addition to the AFE, this thesis also presents the design of a pseudodifferential, log-domain multiplier-filter or “multer” which demodulates low-RF signals in the current-domain. This circuit targets high impedance transducers such as capacitive micromachined ultrasound transducers (CMUTs) and offers a 20dB improvement in dynamic range over the voltage-mode AFE. The bandwidth is also electronically tunable. The circuit was implemented in 0.35μm BiCMOS and was simulated in Cadence; however, no fabrication results were obtained for this circuit. B-mode images were obtained for both architectures. The quadrature SAB method yields a higher image SNR and 9% lower root mean squared error with respect to the RF-beamformed reference image than the compressive SAB method. Thus, while both architectures achieve a significant reduction in sampling rate, system complexity and area, the quadrature SAB method achieves better image quality. Future work may involve the addition of multiple receiver channels and the development of an integrated system-on-chip.Open Acces

Development of real-time cellular impedance analysis system

The cell impedance analysis technique is a label-free, non-invasive method, which simplifies sample preparation and allows applications requiring unmodified cell retrieval. However, traditional impedance measurement methods suffer from various problems (speed, bandwidth, accuracy) for extracting the cellular impedance information. This thesis proposes an improved system for extracting precise cellular impedance in real-time, with a wide bandwidth and satisfactory accuracy. The system hardware consists of five main parts: a microelectrode array (MEA), a stimulation circuit, a sensing circuit, a multi-function card and a computer. The development of system hardware is explored. Accordingly, a novel bioimpedance measurement method coined digital auto balancing bridge method, which is improved from the traditional analogue auto balancing bridge circuitry, is realized for real-time cellular impedance measurement. Two different digital bridge balancing algorithms are proposed and realized, which are based on least mean squares (LMS) algorithm and fast block LMS (FBLMS) algorithm for single- and multi-frequency measurements respectively. Details on their implementation in FPGA are discussed. The test results prove that the LMS-based algorithm is suitable for accelerating the measurement speed in single-frequency situation, whilst the FBLMS-based algorithm has advantages in stable convergence in multi-frequency applications. A novel algorithm, called the All Phase Fast Fourier Transform (APFFT), is applied for post-processing of bioimpedance measurement results. Compared with the classical FFT algorithm, the APFFT significantly reduces spectral leakage caused by truncation error. Compared to the traditional FFT and Digital Quadrature Demodulation (DQD) methods, the APFFT shows excellent performance for extracting accurate phase and amplitude in the frequency spectrum. Additionally, testing and evaluation of the realized system has been performed. The results show that our system achieved a satisfactory accuracy within a wide bandwidth, a fast measurement speed and a good repeatability. Furthermore, our system is compared with a commercial impedance analyzer (Agilent 4294A) in biological experiments. The results reveal that our system achieved a comparable accuracy to the commercial instrument in the biological experiments. Finally, conclusions are given and the future work is proposed

Survey of FPGA applications in the period 2000 – 2015 (Technical Report)

Romoth J, Porrmann M, Rückert U. Survey of FPGA applications in the period 2000 – 2015 (Technical Report).; 2017.Since their introduction, FPGAs can be seen in more and more different fields of applications. The key advantage is the combination of software-like flexibility with the performance otherwise common to hardware. Nevertheless, every application field introduces special requirements to the used computational architecture. This paper provides an overview of the different topics FPGAs have been used for in the last 15 years of research and why they have been chosen over other processing units like e.g. CPUs

Real-time Visual Flow Algorithms for Robotic Applications

Vision offers important sensor cues to modern robotic platforms. Applications such as control of aerial vehicles, visual servoing, simultaneous localization and mapping, navigation and more recently, learning, are examples where visual information is fundamental to accomplish tasks. However, the use of computer vision algorithms carries the computational cost of extracting useful information from the stream of raw pixel data. The most sophisticated algorithms use complex mathematical formulations leading typically to computationally expensive, and consequently, slow implementations. Even with modern computing resources, high-speed and high-resolution video feed can only be used for basic image processing operations. For a vision algorithm to be integrated on a robotic system, the output of the algorithm should be provided in real time, that is, at least at the same frequency as the control logic of the robot. With robotic vehicles becoming more dynamic and ubiquitous, this places higher requirements to the vision processing pipeline. This thesis addresses the problem of estimating dense visual flow information in real time. The contributions of this work are threefold. First, it introduces a new filtering algorithm for the estimation of dense optical flow at frame rates as fast as 800 Hz for 640x480 image resolution. The algorithm follows a update-prediction architecture to estimate dense optical flow fields incrementally over time. A fundamental component of the algorithm is the modeling of the spatio-temporal evolution of the optical flow field by means of partial differential equations. Numerical predictors can implement such PDEs to propagate current estimation of flow forward in time. Experimental validation of the algorithm is provided using high-speed ground truth image dataset as well as real-life video data at 300 Hz. The second contribution is a new type of visual flow named structure flow. Mathematically, structure flow is the three-dimensional scene flow scaled by the inverse depth at each pixel in the image. Intuitively, it is the complete velocity field associated with image motion, including both optical flow and scale-change or apparent divergence of the image. Analogously to optic flow, structure flow provides a robotic vehicle with perception of the motion of the environment as seen by the camera. However, structure flow encodes the full 3D image motion of the scene whereas optic flow only encodes the component on the image plane. An algorithm to estimate structure flow from image and depth measurements is proposed based on the same filtering idea used to estimate optical flow. The final contribution is the spherepix data structure for processing spherical images. This data structure is the numerical back-end used for the real-time implementation of the structure flow filter. It consists of a set of overlapping patches covering the surface of the sphere. Each individual patch approximately holds properties such as orthogonality and equidistance of points, thus allowing efficient implementations of low-level classical 2D convolution based image processing routines such as Gaussian filters and numerical derivatives. These algorithms are implemented on GPU hardware and can be integrated to future Robotic Embedded Vision systems to provide fast visual information to robotic vehicles