Exploring information retrieval using image sparse representations:from circuit designs and acquisition processes to specific reconstruction algorithms

New advances in the field of image sensors (especially in CMOS technology) tend to question the conventional methods used to acquire the image. Compressive Sensing (CS) plays a major role in this, especially to unclog the Analog to Digital Converters which are generally representing the bottleneck of this type of sensors. In addition, CS eliminates traditional compression processing stages that are performed by embedded digital signal processors dedicated to this purpose. The interest is twofold because it allows both to consistently reduce the amount of data to be converted but also to suppress digital processing performed out of the sensor chip. For the moment, regarding the use of CS in image sensors, the main route of exploration as well as the intended applications aims at reducing power consumption related to these components (i.e. ADC & DSP represent 99% of the total power consumption). More broadly, the paradigm of CS allows to question or at least to extend the Nyquist-Shannon sampling theory. This thesis shows developments in the field of image sensors demonstrating that is possible to consider alternative applications linked to CS. Indeed, advances are presented in the fields of hyperspectral imaging, super-resolution, high dynamic range, high speed and non-uniform sampling. In particular, three research axes have been deepened, aiming to design proper architectures and acquisition processes with their associated reconstruction techniques taking advantage of image sparse representations. How the on-chip implementation of Compressed Sensing can relax sensor constraints, improving the acquisition characteristics (speed, dynamic range, power consumption) ? How CS can be combined with simple analysis to provide useful image features for high level applications (adding semantic information) and improve the reconstructed image quality at a certain compression ratio ? Finally, how CS can improve physical limitations (i.e. spectral sensitivity and pixel pitch) of imaging systems without a major impact neither on the sensing strategy nor on the optical elements involved ? A CMOS image sensor has been developed and manufactured during this Ph.D. to validate concepts such as the High Dynamic Range - CS. A new design approach was employed resulting in innovative solutions for pixels addressing and conversion to perform specific acquisition in a compressed mode. On the other hand, the principle of adaptive CS combined with the non-uniform sampling has been developed. Possible implementations of this type of acquisition are proposed. Finally, preliminary works are exhibited on the use of Liquid Crystal Devices to allow hyperspectral imaging combined with spatial super-resolution. The conclusion of this study can be summarized as follows: CS must now be considered as a toolbox for defining more easily compromises between the different characteristics of the sensors: integration time, converters speed, dynamic range, resolution and digital processing resources. However, if CS relaxes some material constraints at the sensor level, it is possible that the collected data are difficult to interpret and process at the decoder side, involving massive computational resources compared to so-called conventional techniques. The application field is wide, implying that for a targeted application, an accurate characterization of the constraints concerning both the sensor (encoder), but also the decoder need to be defined

The design and implementation of a stellar gyroscope for accurate angular rate estimation on CubeSats

Thesis (MEng)--Stellenbosch University, 2015.ENGLISH ABSTRACT: Until recently, small form factor satellites (such as CubeSats) relied almost exclusively on micro electromechanical system (MEMS) gyroscopes for attitude propagation purposes. Unfortunately, the nature of MEMS gyros is such that they exhibit a measure of bias drift. This drift must be compensated for, a task for which stellar gyros have proved to be exceptionally useful. Stellar gyros are satellite subsystems capable of inferring three-axis attitude propagation based on the displacement of a series of stars between successive image frames. Their design is analogous to that of star trackers, using many of the same hardware designs and algorithms. When used in combination with MEMS solutions, stellar gyros provide not only a means for drift compensation, but also a measure of functional redundancy with regard to attitude propagation. This thesis presents the design and implementation of stellar gyroscope algorithms capable of operating alongside existing orientation algorithms on traditional star tracker hardware. The CubeStar star tracker module is used as development platform. The proposed stellar gyro solution retains CubeStar’s existing star extraction algorithms, while investigating alternative methods for star centroiding in addition to the existing centre of gravity (CoG) approach. A dynamic proximity based matching algorithm is suggested to determine star correspondence between image frames. Finally, various well established estimation algorithms are considered for the purpose of rate determination, including singular value decomposition (SVD), Davenport’s q-Method and weighted least-squares (WLS). An initial evaluation of the proposed algorithms is made based on simulations in the MATLAB environment. Simulation results are confirmed through means of practical tests, performed on a simulated night sky in a controlled environment. With a focus on low angular rates, results suggest reliable operation up to ±1 deg/s in all three axes of rotation. As expected for stellar imaging solutions, angular rates estimated in both cross-boresight axes are almost an order of magnitude more accurate than the corresponding estimates in the boresight axis itself.AFRIKAANSE OPSOMMING: Mikrosatelliete, soos CubeSats, het tot onlangs byna uitsluitlik op mikro elektromeganiese (MEMS) vibrerende struktuur giroskope staatgemaak vir die meet van hoeksnelhede. Ongelukkig is die aard van MEMS giroskope sodanig dat hierdie metings afsette toon wat al hoe verder van hul werklike waardes verskuif. Daar moet gekompenseer word vir hierdie verskuiwing, ’n taak waarvoor stergiroskope besonder geskik is. Sterrebeeld gebaseerde giroskope (of bloot gewoon stergiroskope) is satelliet substelsels wat daartoe in staat is om ’n satelliet se oriëntasie in drie dimensies te propageer deur gebruik te maak van die verplasing van ’n reeks sterre tussen twee opeenvolgende beelde. Hulle ontwerp in terme van beide hardeware en algoritmes is soortgelyk aan dié van stervolger kameras. Stergiroskope kan ook saam met MEMS toestelle gebruik word. Hulle verskaf beide ’n metode om te kompenseer vir verskuiwings in MEMS metings sowel as ’n funksionele alternatief met betrekking tot hoekafskatting. Hierdie tesis beskryf die ontwerp en implementering van ster giroskoop algoritmes wat in staat is om hand-in-hand met bestaande oriëntasie algoritmes op tradisionele ster volger hardeware te funksioneer. Die CubeStar stervolger module is as ontwikkelings platform gebruik. Die beoogde stergiroskoop ontwerp behou CubeStar se bestaande ster ontginnings algoritmes. Verskeie metodes benewens die bestaande swaartepunt benadering word wel ondersoek vir die bepaling van ster sentroïedes. Die korrespondensie tussen opeenvolgende sterbeelde word bepaal deur middel van ’n dinamiese nabyheid gebaseerde passings algoritme. Ten slotte word verskeie algoritmes oorweeg vir die afskatting van ’n satelliet se hoeksnelhede. Dit sluit in enkelvoud waarde ontbinding (SVD), Davenport se q-metode en ’n geweegte kleinste kwadraat (WLS) benadering. Die voorgestelde algoritmes is ge-evalueer op grond van simulasies in die MATLAB omgewing. Praktiese toetse is uitgevoer op ’n gesimuleerde sterrebeeld om simulasie resultate te bevestig. Met ’n fokus op lae hoeksnelhede dui resultate op betroubare afskatting teen hoeksnelhede van tot ±1 grade/s rondom al drie rotasie-asse. Soos verwag van ster kameras is die hoekafskattings rondom die transversale asse ’n orde meer akkuraat as die ooreenstemmende afskattings rondom die optiese as

Generation of a Land Cover Atlas of environmental critic zones using unconventional tools

L'abstract è presente nell'allegato / the abstract is in the attachmen

Multimedia Forensics

This book is open access. Media forensics has never been more relevant to societal life. Not only media content represents an ever-increasing share of the data traveling on the net and the preferred communications means for most users, it has also become integral part of most innovative applications in the digital information ecosystem that serves various sectors of society, from the entertainment, to journalism, to politics. Undoubtedly, the advances in deep learning and computational imaging contributed significantly to this outcome. The underlying technologies that drive this trend, however, also pose a profound challenge in establishing trust in what we see, hear, and read, and make media content the preferred target of malicious attacks. In this new threat landscape powered by innovative imaging technologies and sophisticated tools, based on autoencoders and generative adversarial networks, this book fills an important gap. It presents a comprehensive review of state-of-the-art forensics capabilities that relate to media attribution, integrity and authenticity verification, and counter forensics. Its content is developed to provide practitioners, researchers, photo and video enthusiasts, and students a holistic view of the field

Selective Darkening Filter and Welding Arc Observation for the Manual Welding Process

An optical see-through LCD (GLCD) with a resolution of n x m pixels gives the ability to selectively control the darkening in the welders view. The setup of such a Selective Auto Darkening Filter is developed and its applicability tested. The setup is done by integrating a camera into the welding operation for extracting the welding arc position properly. A prototype of a GLCD taylored for welding is mounted in the welder's view. The extraction of the welding arc position requires an enhanced video acquisition during welding. The observation of scenes with high dynamic contrast is an outstanding problem which occurs if very high differences between the darkest and the brightest spot in a scene occur. The application to welding with its harsh conditions needs the development of supporting hardware. The synchronization of the camera with the flickering light conditions of pulsed welding processes in Gas Metal Arc Welding (GMAW) stabilizes the acquisition process and allows the scene to be flashed precisely if required by compact high power LEDs. The image acquisition is enhanced by merging two different exposed images for the resulting image. These source images cover a wider histogram range than it is possible by using only a single shot image with optimal camera parameters. After testing different standard contrast enhancement algorithm a novel content based algorithm is developed. It segments the image into areas with similar content and enhances these independently

Computational Imaging and Artificial Intelligence: The Next Revolution of Mobile Vision

Signal capture stands in the forefront to perceive and understand the environment and thus imaging plays the pivotal role in mobile vision. Recent explosive progresses in Artificial Intelligence (AI) have shown great potential to develop advanced mobile platforms with new imaging devices. Traditional imaging systems based on the "capturing images first and processing afterwards" mechanism cannot meet this unprecedented demand. Differently, Computational Imaging (CI) systems are designed to capture high-dimensional data in an encoded manner to provide more information for mobile vision systems.Thanks to AI, CI can now be used in real systems by integrating deep learning algorithms into the mobile vision platform to achieve the closed loop of intelligent acquisition, processing and decision making, thus leading to the next revolution of mobile vision.Starting from the history of mobile vision using digital cameras, this work first introduces the advances of CI in diverse applications and then conducts a comprehensive review of current research topics combining CI and AI. Motivated by the fact that most existing studies only loosely connect CI and AI (usually using AI to improve the performance of CI and only limited works have deeply connected them), in this work, we propose a framework to deeply integrate CI and AI by using the example of self-driving vehicles with high-speed communication, edge computing and traffic planning. Finally, we outlook the future of CI plus AI by investigating new materials, brain science and new computing techniques to shed light on new directions of mobile vision systems

Multimedia Forensics

This book is open access. Media forensics has never been more relevant to societal life. Not only media content represents an ever-increasing share of the data traveling on the net and the preferred communications means for most users, it has also become integral part of most innovative applications in the digital information ecosystem that serves various sectors of society, from the entertainment, to journalism, to politics. Undoubtedly, the advances in deep learning and computational imaging contributed significantly to this outcome. The underlying technologies that drive this trend, however, also pose a profound challenge in establishing trust in what we see, hear, and read, and make media content the preferred target of malicious attacks. In this new threat landscape powered by innovative imaging technologies and sophisticated tools, based on autoencoders and generative adversarial networks, this book fills an important gap. It presents a comprehensive review of state-of-the-art forensics capabilities that relate to media attribution, integrity and authenticity verification, and counter forensics. Its content is developed to provide practitioners, researchers, photo and video enthusiasts, and students a holistic view of the field

Spherical Image Processing for Immersive Visualisation and View Generation

This research presents the study of processing panoramic spherical images for immersive visualisation of real environments and generation of in-between views based on two views acquired. For visualisation based on one spherical image, the surrounding environment is modelled by a unit sphere mapped with the spherical image and the user is then allowed to navigate within the modelled scene. For visualisation based on two spherical images, a view generation algorithm is developed for modelling an indoor manmade environment and new views can be generated at an arbitrary position with respect to the existing two. This allows the scene to be modelled using multiple spherical images and the user to move smoothly from one sphere mapped image to another one by going through in-between sphere mapped images generated