Algorithm/Architecture Co-Exploration of Visual Computing: Overview and Future Perspectives

Concurrently exploring both algorithmic and architectural optimizations is a new design paradigm. This survey paper addresses the latest research and future perspectives on the simultaneous development of video coding, processing, and computing algorithms with emerging platforms that have multiple cores and reconfigurable architecture. As the algorithms in forthcoming visual systems become increasingly complex, many applications must have different profiles with different levels of performance. Hence, with expectations that the visual experience in the future will become continuously better, it is critical that advanced platforms provide higher performance, better flexibility, and lower power consumption. To achieve these goals, algorithm and architecture co-design is significant for characterizing the algorithmic complexity used to optimize targeted architecture. This paper shows that seamless weaving of the development of previously autonomous visual computing algorithms and multicore or reconfigurable architectures will unavoidably become the leading trend in the future of video technology

Duality based optical flow algorithms with applications

We consider the popular TV-L1 optical flow formulation, and the so-called dual-ity based algorithm for minimizing the TV-L1 energy. The original formulation is extended to allow for vector valued images, and minimization results are given. In addition we consider di↵erent definitions of total variation regulariza-tion, and related formulations of the optical flow problem that may be used with a duality based algorithm. We present a highly optimized algorithmic setup to estimate optical flows, and give five novel applications. The first application is registration of medical images, where X-ray images of di↵erent hands, taken using di↵erent imaging devices are registered using a TV-L1 optical flow algo-rithm. We propose to regularize the input images, using sparsity enhancing regularization of the image gradient to improve registration results. The second application is registration of 2D chromatograms, where registration only have to be done in one of the two dimensions, resulting in a vector valued registration problem with values having several hundred dimensions. We propose a nove

Intelligent Side Information Generation in Distributed Video Coding

Distributed video coding (DVC) reverses the traditional coding paradigm of complex encoders allied with basic decoding to one where the computational cost is largely incurred by the decoder. This is attractive as the proven theoretical work of Wyner-Ziv (WZ) and Slepian-Wolf (SW) shows that the performance by such a system should be exactly the same as a conventional coder. Despite the solid theoretical foundations, current DVC qualitative and quantitative performance falls short of existing conventional coders and there remain crucial limitations. A key constraint governing DVC performance is the quality of side information (SI), a coarse representation of original video frames which are not available at the decoder. Techniques to generate SI have usually been based on linear motion compensated temporal interpolation (LMCTI), though these do not always produce satisfactory SI quality, especially in sequences exhibiting non-linear motion. This thesis presents an intelligent higher order piecewise trajectory temporal interpolation (HOPTTI) framework for SI generation with original contributions that afford better SI quality in comparison to existing LMCTI-based approaches. The major elements in this framework are: (i) a cubic trajectory interpolation algorithm model that significantly improves the accuracy of motion vector estimations; (ii) an adaptive overlapped block motion compensation (AOBMC) model which reduces both blocking and overlapping artefacts in the SI emanating from the block matching algorithm; (iii) the development of an empirical mode switching algorithm; and (iv) an intelligent switching mechanism to construct SI by automatically selecting the best macroblock from the intermediate SI generated by HOPTTI and AOBMC algorithms. Rigorous analysis and evaluation confirms that significant quantitative and perceptual improvements in SI quality are achieved with the new framework

Towards Computational Efficiency of Next Generation Multimedia Systems

To address throughput demands of complex applications (like Multimedia), a next-generation system designer needs to co-design and co-optimize the hardware and software layers. Hardware/software knobs must be tuned in synergy to increase the throughput efficiency. This thesis provides such algorithmic and architectural solutions, while considering the new technology challenges (power-cap and memory aging). The goal is to maximize the throughput efficiency, under timing- and hardware-constraints

Framework for the detection and classification of colorectal polyps

In this thesis we propose a framework for the detection and classification of colorectal polyps to assist endoscopists in bowel cancer screening. Such a system will help reduce not only the miss rate of possibly malignant polyps during screening but also reduce the number of unnecessary polypectomies where the histopathologic analysis could be spared. Our polyp detection scheme is based on a cascade filter to pre-process the incoming video frames, select a group of candidate polyp regions and then proceed to algorithmically isolate the most probable polyps based on their geometry. We also tested this system on a number of endoscopic and capsule endoscopy videos collected with the help of our clinical collaborators. Furthermore, we developed and tested a classification system for distinguishing cancerous colorectal polyps from non-cancerous ones. By analyzing the surface vasculature of high magnification polyp images from two endoscopic platforms we extracted a number of features based primarily on the vessel contrast, orientation and colour. The feature space was then filtered as to leave only the most relevant subset and this was subsequently used to train our classifier. In addition, we examined the scenario of splitting up the polyp surface into patches and including only the most feature rich areas into our classifier instead of the surface as a whole. The stability of our feature space relative to patch size was also examined to ensure reliable and robust classification. In addition, we devised a scale selection strategy to minimize the effect of inconsistencies in magnification and geometric polyp size between samples. Lastly, several techniques were also employed to ensure that our results will generalise well in real world practise. We believe this to be a solid step in forming a toolbox designed to aid endoscopists not only in the detection but also in the optical biopsy of colorectal polyps during in vivo colonoscopy.Open Acces

Media gateway utilizando um GPU

Mestrado em Engenharia de Computadores e Telemátic

Energy efficient enabling technologies for semantic video processing on mobile devices

Semantic object-based processing will play an increasingly important role in future multimedia systems due to the ubiquity of digital multimedia capture/playback technologies and increasing storage capacity. Although the object based paradigm has many undeniable benefits, numerous technical challenges remain before the applications becomes pervasive, particularly on computational constrained mobile devices. A fundamental issue is the ill-posed problem of semantic object segmentation. Furthermore, on battery powered mobile computing devices, the additional algorithmic complexity of semantic object based processing compared to conventional video processing is highly undesirable both from a real-time operation and battery life perspective. This thesis attempts to tackle these issues by firstly constraining the solution space and focusing on the human face as a primary semantic concept of use to users of mobile devices. A novel face detection algorithm is proposed, which from the outset was designed to be amenable to be offloaded from the host microprocessor to dedicated hardware, thereby providing real-time performance and reducing power consumption. The algorithm uses an Artificial Neural Network (ANN), whose topology and weights are evolved via a genetic algorithm (GA). The computational burden of the ANN evaluation is offloaded to a dedicated hardware accelerator, which is capable of processing any evolved network topology. Efficient arithmetic circuitry, which leverages modified Booth recoding, column compressors and carry save adders, is adopted throughout the design. To tackle the increased computational costs associated with object tracking or object based shape encoding, a novel energy efficient binary motion estimation architecture is proposed. Energy is reduced in the proposed motion estimation architecture by minimising the redundant operations inherent in the binary data. Both architectures are shown to compare favourable with the relevant prior art