Low power motion estimation based frame rate up-conversion hardware designs

Recently flat panel high definition television (HDTV) displays with 100 Hz, 120 Hz and 240 Hz picture rates are introduced. However, video materials are captured and broadcast in different temporal resolutions ranging from 24 Hz to 60 Hz. In order to display these video formats correctly on high picture rate displays, new frames should be generated and inserted into the original video sequence to increase its frame rate. Therefore, frame rate upconversion (FRUC) has become a necessity. Motion compensated FRUC (MC-FRUC) algorithms provide better quality results than non-motion compensated FRUC algorithms. These MC-FRUC algorithms consist of two main stages, motion estimation (ME) and motion compensated interpolation (MCI). In ME, motion vectors (MV) are calculated between successive frames, and in MCI this MV data is used to generate a new frame that is inserted between two successive frames, thus doubling the frame rate. In addition to these two main steps, intermediate steps such as refinement of the MV field by various algorithms like motion vector smoothing and bilateral ME refinement may be used to improve the quality of the interpolated video. In this thesis, a perfect absolute difference technique for block matching ME hardware is proposed. The proposed technique reduces the power consumption of a full search ME hardware by 2.2% on a XC2VP30-7 FPGA without any PSNR loss. In addition, a global motion estimation (GME) algorithm and its hardware implementation are proposed. The proposed GME algorithm increases PSNR of 3D recursive search ME algorithm by 2.5% and its hardware implementation is capable of processing 341 720p frames per second. An adaptive technique for GME, which reduces the energy consumption of the GME hardware by 14.37% on a XC6VLX75T FPGA with a 0.17% PSNR loss, is also proposed. Furthermore, an early termination technique for the adaptive bilateral motion estimation (ABIME) algorithm is proposed. The proposed technique reduces the energy consumption of the ABIME hardware by 29% with a 0.04% PSNR loss on a XC6VLX75T FPGA. In addition, an efficient weighted coefficient overlapped block motion compensation (WC-OBMC) hardware which reduces the dynamic power consumption of the reference WC-OBMC hardware by 22% is proposed. The proposed hardware is capable of processing 57 720p frames per second on a XC6VLX75T FPGA. Finally, the ABIME hardware is implemented on a Xilinx ML605 FPGA board

An adaptive true motion estimation algorithm for frame rate up-conversion and its hardware design

With the advancement in video and display technologies, recently flat panel High Definition Television (HDTV) displays with 100 Hz, 120 Hz and most recently 240 Hz picture rates are introduced. However, video materials are captured and broadcast in different temporal resolutions ranging from 24 Hz to 60 Hz. In order to display these video formats correctly on high picture rate displays, new frames should be generated and inserted into the original video sequence to increase its frame rate. Therefore, Frame Rate Up-Conversion (FRUC) has become a necessity. Motion Compensated FRUC algorithms provide better quality results than non-motion compensated FRUC algorithms. Motion Estimation (ME) is the process of finding motion vectors which describe the motion of the objects between adjacent frames and is the most computationally intensive part of motion compensated FRUC algorithms. For FRUC applications, it is important to find the motion vectors that represent real motions of the objects which is called true ME. In this thesis, an Adaptive True Motion Estimation (ATME) algorithm is proposed. ATME algorithm produces similar quality results with less number of calculations or better quality results with similar number of calculations compared to 3-D Recursive Search true ME algorithm by adaptively using optimized sets of candidate search locations and several redundancy removal techniques. In addition, 3 different complexity hardware architectures for ATME are proposed. The proposed hardware use efficient data re-use schemes for the non-regular data flow of ATME algorithm. 2 of these hardware architectures are implemented on Xilinx Virtex-4 FPGA and are capable of processing ~158 and ~168 720p HD frames per second respectively

Scene analysis and risk estimation for domestic robots, security and smart homes

The evaluation of risk within a scene is a new and emerging area of research. With the advent of smart enabled homes and the continued development and implementation of domestic robotics, the platform for automated risk assessment within the home is now a possibility. The aim of this thesis is to explore a subsection of the problems facing the detection and quantification of risk in a domestic setting. A Risk Estimation framework is introduced which provides a flexible and context aware platform from which measurable elements of risk can be combined to create a final risk score for a scene. To populate this framework, three elements of measurable risk are proposed and evaluated: Firstly, scene stability, assessing the location and stability of objects within an environment through the use of physics simulation techniques. Secondly, hazard feature analysis using two specifically designed novel feature descriptors (3D Voxel HOG and the Physics Behaviour Feature) which determine if the objects within a scene have dangerous or risky properties such as blades or points. Finally, environment interaction, which uses human behaviour simulation to predict human reactions to detected risks and highlight areas of a scene most likely to be visited. Additionally methodologies are introduced to support these concepts including: a simulation prediction framework which reduces the computational cost of physics simulation, a Robust Filter and Complex Adaboost which aim to improve the robustness and training times required for hazard feature classification models. The Human and Group Behaviour Evaluation framework is introduced to provide a platform from which simulation algorithms can be evaluated without the need for extensive ground truth data. Finally the 3D Risk Scenes (3DRS) dataset is introduced, creating a risk specific dataset for the evaluation of future domestic risk analysis methodologies