Benchmarking of mobile phone cameras

fi=vertaisarvioitu|en=peerReviewed

Digital Video Stabilization

Ph.DDOCTOR OF PHILOSOPH

An active vision system for tracking and mosaicking on UAV.

Lin, Kai Wun.Thesis (M.Phil.)--Chinese University of Hong Kong, 2011.Includes bibliographical references (leaves 120-127).Abstracts in English and Chinese.Abstract --- p.iAcknowledgement --- p.iiiChapter 1 --- Introduction --- p.1Chapter 1.1 --- Overview of the UAV Project --- p.1Chapter 1.2 --- Challenges on Vision System for UAV --- p.2Chapter 1.3 --- Contributions of this Work --- p.4Chapter 1.4 --- Organization of Thesis --- p.6Chapter 2 --- Image Sensor Selection and Evaluation --- p.8Chapter 2.1 --- Image Sensor Overview --- p.8Chapter 2.1.1 --- Comparing Sensor Features and Performance --- p.9Chapter 2.1.2 --- Rolling Shutter vsGlobal Shutter --- p.10Chapter 2.2 --- Sensor Evaluation through USB Peripheral --- p.11Chapter 2.2.1 --- Interfacing Image Sensor and USB Controller --- p.12Chapter 2.2.2 --- Image Sensor Configuration --- p.14Chapter 2.3 --- Image Data Transmitting and Processing --- p.17Chapter 2.3.1 --- Data Transfer Mode and Buffering on USB Controller --- p.18Chapter 2.3.2 --- Demosaicking of Bayer Image Data --- p.20Chapter 2.4 --- Splitting Images and Exposure Problem --- p.22Chapter 2.4.1 --- Buffer Overflow on USB Controller --- p.22Chapter 2.4.2 --- Image Luminance and Exposure Adjustment --- p.24Chapter 3 --- Embedded System for Vision Processing --- p.26Chapter 3.1 --- Overview of the Embedded System --- p.26Chapter 3.1.1 --- TI OMAP3530 Processor --- p.27Chapter 3.1.2 --- Gumstix Overo Fire Computer-on-Module --- p.27Chapter 3.2 --- Interfacing Camera Module to the Embedded System --- p.28Chapter 3.2.1 --- Image Signal Processing Subsystem --- p.29Chapter 3.2.2 --- Camera Module Adapting Board --- p.30Chapter 3.2.3 --- Image Sensor Driver and Program Development --- p.31Chapter 3.3 --- View-stabilizing Biaxial Camera Platform --- p.34Chapter 3.3.1 --- The New Camera System iv --- p.35Chapter 3.3.2 --- View-stabilizing Pan-tilt Platform --- p.41Chapter 3.4 --- Overall System Architecture and UAV Integration --- p.46Chapter 4 --- Target Tracking and Geo-locating --- p.50Chapter 4.1 --- Camera Calibration --- p.51Chapter 4.1.1 --- The Perspective Camera Model --- p.51Chapter 4.1.2 --- Camera Lens Distortions --- p.53Chapter 4.1.3 --- Calibration Toolbox and Results --- p.54Chapter 4.2 --- Selection of Object Features and Trackers --- p.56Chapter 4.2.1 --- Harris Corner Detection --- p.58Chapter 4.2.2 --- Color Histogram --- p.59Chapter 4.2.3 --- KLT and Mean-shift Tracker --- p.59Chapter 4.3 --- Target Auto-centering --- p.64Chapter 4.3.1 --- Formulation of the PID Controller --- p.65Chapter 4.3.2 --- Control Gain Settings and Tuning --- p.69Chapter 4.4 --- Geo-locating of Tracked Target --- p.69Chapter 4.4.1 --- Coordinate Frame Transformation --- p.70Chapter 4.4.2 --- Depth Estimation and Target Locating --- p.74Chapter 4.5 --- Results and Discussion --- p.77Chapter 5 --- Real-time Aerial Mosaic Building --- p.89Chapter 5.1 --- Motion Model Selection --- p.90Chapter 5.1.1 --- Planar Perspective Motion Model --- p.90Chapter 5.2 --- Feature-based Image Alignment --- p.91Chapter 5.2.1 --- Image Preprocessing --- p.91Chapter 5.2.2 --- Feature Extraction and Matching --- p.92Chapter 5.2.3 --- Image Alignment using RANSAC Algorithm --- p.94Chapter 5.3 --- Image Composition --- p.95Chapter 5.3.1 --- Image Blending with Distance Map --- p.96Chapter 5.3.2 --- Overall Stitching Process --- p.98Chapter 5.4 --- Mosaic Simulation using Google Earth --- p.99Chapter 5.5 --- Results and Discussion --- p.100Chapter 6 --- Conclusion and Further Work --- p.108Chapter A --- System Schematics --- p.111Chapter B --- Image Sensor Sensitivity --- p.118Bibliography --- p.12

Visible Light Optical Camera Communication for Electroencephalography Applications

Due to the cable-free deployment and flexibility of wireless communications, the data transmission in the applications of home and healthcare has shown a trend of moving wired communications to wireless communications. One typical example is electroencephalography (EEG). Evolution in the radio frequency (RF) technology has made it is possible to transmit the EEG data without data cable bundles. However, presently, the RF-based wireless technology used in EEG suffers from electromagnetic interference and might also have adverse effects on the health of patient and other medical equipment used in hospitals or homes. This puts some limits in RF-based EEG solutions, which is particularly true in RF restricted zones like Intensive Care Units (ICUs). As a recently developed optical wireless communication (OWC) technology, visible light communication (VLC) using light-emitting diodes (LEDs) for both simultaneous illumination and data communication has shown its advantages of free from electromagnetic interference, potential huge unlicensed bandwidth and enhanced data privacy due to the line transmission of light. The most recent development of VLC is the optical camera communication (OCC), which is an extension of VLC IEEE standard 802.15.7, also referred to as visible light optical camera communication (VL-OCC). Different from the conventional VLC where traditional photodiodes are used to detect and receive the data, VL-OCC uses the imaging camera as the photodetector to receive the data in the form of visible light signals. The data rate requirement of EEG is dependent on the application; hence this thesis investigates a low cost, organic LED (OLED)-driven VL-OCC wireless data transmission system for EEG applications

Video Magnification for Structural Analysis Testing

The goal of this thesis is to allow a user to see minute motion of an object at different frequencies, using a computer program, to aid in vibration testing analysis without the use of complex setups of accelerometers or expensive laser vibrometers. MIT’s phase-based video motion processing ­was modified to enable modal determination of structures in the field using a cell phone camera. The algorithm was modified by implementing a stabilization algorithm and permitting the magnification filter to operate on multiple frequency ranges to enable visualization of the natural frequencies of structures in the field. To implement multiple frequency ranges a new function was developed to implement the magnification filter at each relevant frequency range within the original video. The stabilization algorithm would allow for a camera to be hand-held instead of requiring a tripod mount. The following methods for stabilization were tested: fixed point video stabilization and image registration. Neither method removed the global motion from the hand-held video, even after masking was implemented, which resulted in poor results. Specifically, fixed point did not remove much motion or created sharp motions and image registration introduced a pulsing effect. The best results occurred when the object being observed had contrast from the background, was the largest feature in the video frame, and the video was captured from a tripod at an appropriate angle. The final program can amplify the motion in user selected frequency bands and can be used as an aid in structural analysis testing

Exploring information retrieval using image sparse representations:from circuit designs and acquisition processes to specific reconstruction algorithms

New advances in the field of image sensors (especially in CMOS technology) tend to question the conventional methods used to acquire the image. Compressive Sensing (CS) plays a major role in this, especially to unclog the Analog to Digital Converters which are generally representing the bottleneck of this type of sensors. In addition, CS eliminates traditional compression processing stages that are performed by embedded digital signal processors dedicated to this purpose. The interest is twofold because it allows both to consistently reduce the amount of data to be converted but also to suppress digital processing performed out of the sensor chip. For the moment, regarding the use of CS in image sensors, the main route of exploration as well as the intended applications aims at reducing power consumption related to these components (i.e. ADC & DSP represent 99% of the total power consumption). More broadly, the paradigm of CS allows to question or at least to extend the Nyquist-Shannon sampling theory. This thesis shows developments in the field of image sensors demonstrating that is possible to consider alternative applications linked to CS. Indeed, advances are presented in the fields of hyperspectral imaging, super-resolution, high dynamic range, high speed and non-uniform sampling. In particular, three research axes have been deepened, aiming to design proper architectures and acquisition processes with their associated reconstruction techniques taking advantage of image sparse representations. How the on-chip implementation of Compressed Sensing can relax sensor constraints, improving the acquisition characteristics (speed, dynamic range, power consumption) ? How CS can be combined with simple analysis to provide useful image features for high level applications (adding semantic information) and improve the reconstructed image quality at a certain compression ratio ? Finally, how CS can improve physical limitations (i.e. spectral sensitivity and pixel pitch) of imaging systems without a major impact neither on the sensing strategy nor on the optical elements involved ? A CMOS image sensor has been developed and manufactured during this Ph.D. to validate concepts such as the High Dynamic Range - CS. A new design approach was employed resulting in innovative solutions for pixels addressing and conversion to perform specific acquisition in a compressed mode. On the other hand, the principle of adaptive CS combined with the non-uniform sampling has been developed. Possible implementations of this type of acquisition are proposed. Finally, preliminary works are exhibited on the use of Liquid Crystal Devices to allow hyperspectral imaging combined with spatial super-resolution. The conclusion of this study can be summarized as follows: CS must now be considered as a toolbox for defining more easily compromises between the different characteristics of the sensors: integration time, converters speed, dynamic range, resolution and digital processing resources. However, if CS relaxes some material constraints at the sensor level, it is possible that the collected data are difficult to interpret and process at the decoder side, involving massive computational resources compared to so-called conventional techniques. The application field is wide, implying that for a targeted application, an accurate characterization of the constraints concerning both the sensor (encoder), but also the decoder need to be defined