Automaattisten kuvantamisalgoritmien parantaminen kahden kameran järjestelmä avulla

Mobile camera users trust widely to the automatic camera settings, but the automatic settings do not perform equally well in all situations. All of the automatic settings have their own algorithms, which their operations are based on. Automatic white balance, automatic exposure control and automatic focus are known automatic camera settings. Dual camera system can improve the automatic settings in mobile cameras. In this thesis dual camera algorithms were researched and dual camera algorithm development framework was created. Instead of creating the entire algorithm development framework, the dual camera functionality was added to the existing single camera algorithm development framework. The existing algorithm development framework was a PC application, which could perform the same image processing steps as mobile phone camera image signal processors. The dual camera development framework is able to collect image parameters and markers from one image and provide these to another image. The dual camera development framework manages the parameter mediation by writing and reading XML files. Few dual camera algorithms were also researched in order to proof the dual camera framework concept. It turned out that the dual camera is able to improve image quality and the automatic camera settings. The automatic exposure control can be improved by collecting additional data from the histogram and by using this information when producing the final image. Motion blur can also be reduced by dual camera algorithm that compares edge data from two distinct images with each other and then provides valuable information about the motion to the exposure control algorithm. The automatic exposure control algorithm can thus reduce exposure time and increase the sensor sensitivity in order to produce sharp images. Automatic white balance can be improved with dual camera system by estimating illumination source with dual camera system. Even though the dual camera seems to provide some advantages to the algorithms, and the concept that the dual camera development framework introduces, seems valid, addition of second camera to mobile phone should be considered carefully. Many of the dual camera algorithms could be transformed to work with only one camera. The dual camera algorithm development takes also a large amount of time and effort, given that the dual camera is not going to be applied to all future camera phones

Automatic image quality enhancement using deep neural networks

Abstract. Photo retouching can significantly improve image quality and it is considered an essential part of photography. Traditionally this task has been completed manually with special image enhancement software. However, recent research utilizing neural networks has been proven to perform better in the automated image enhancement task compared to traditional methods. During the literature review of this thesis, multiple automatic neural-network-based image enhancement methods were studied, and one of these methods was chosen for closer examination and evaluation. The chosen network design has several appealing qualities such as the ability to learn both local and global enhancements, and its simple architecture constructed for efficient computational speed. This research proposes a novel dataset generation method for automated image enhancement research, and tests its usefulness with the chosen network design. This dataset generation method simulates commonly occurring photographic errors, and the original high-quality images can be used as the target data. This dataset design allows studying fixes for individual and combined aberrations. The underlying idea of this design choice is that the network would learn to fix these aberrations while producing aesthetically pleasing and consistent results. The quantitative evaluation proved that the network can learn to counter these errors, and with greater effort, it could also learn to enhance all of these aspects simultaneously. Additionally, the network’s capability of learning local and portrait specific enhancement tasks were evaluated. The models can apply the effect successfully, but the results did not gain the same level of accuracy as with global enhancement tasks. According to the completed qualitative survey, the images enhanced by the proposed general enhancement model can successfully enhance the image quality, and it can perform better than some of the state-of-the-art image enhancement methods.Automaattinen kuvanlaadun parantaminen käyttämällä syviä neuroverkkoja. Tiivistelmä. Manuaalinen valokuvien käsittely voi parantaa kuvanlaatua huomattavasti ja sitä pidetään oleellisena osana valokuvausprosessia. Perinteisesti tätä tehtävää varten on käytetty erityisiä manuaalisesti operoitavia kuvankäsittelyohjelmia. Nykytutkimus on kuitenkin todistanut neuroverkkojen paremmuuden automaattisessa kuvanparannussovelluksissa perinteisiin menetelmiin verrattuna. Tämän diplomityön kirjallisuuskatsauksessa tutkittiin useita neuroverkkopohjaisia kuvanparannusmenetelmiä, ja yksi näistä valittiin tarkempaa tutkimusta ja arviointia varten. Valitulla verkkomallilla on useita vetoavia ominaisuuksia, kuten paikallisten sekä globaalien kuvanparannusten oppiminen ja sen yksinkertaistettu arkkitehtuuri, joka on rakennettu tehokasta suoritusnopeutta varten. Tämä tutkimus esittää uuden opetusdatan generointimenetelmän automaattisia kuvanparannusmetodeja varten, ja testaa sen soveltuvuutta käyttämällä valittua neuroverkkorakennetta. Tämä opetusdatan generointimenetelmä simuloi usein esiintyviä valokuvauksellisia virheitä, ja alkuperäisiä korkealaatuisia kuvia voi käyttää opetuksen tavoitedatana. Tämän generointitavan avulla voitiin tutkia erillisten valokuvausvirheiden, sekä näiden yhdistelmän korjausta. Tämän menetelmän tarkoitus oli opettaa verkkoa korjaamaan erilaisia virheitä sekä tuottamaan esteettisesti miellyttäviä ja yhtenäisiä tuloksia. Kvalitatiivinen arviointi todisti, että käytetty neuroverkko kykenee oppimaan erillisiä korjauksia näille virheille. Neuroverkko pystyy oppimaan myös mallin, joka korjaa kaikkia ennalta määrättyjä virheitä samanaikaisesti, mutta alhaisemmalla tarkkuudella. Lisäksi neuroverkon kyvykkyyttä oppia paikallisia muotokuvakohtaisia kuvanparannuksia arvioitiin. Koulutetut mallit pystyvät myös toteuttamaan paikallisen kuvanparannuksen onnistuneesti, mutta nämä mallit eivät yltäneet globaalien parannusten tasolle. Toteutetun kyselytutkimuksen mukaan esitetty yleisen kuvanparannuksen malli pystyy parantamaan kuvanlaatua onnistuneesti, sekä tuottaa parempia tuloksia kuin osa vertailluista kuvanparannustekniikoista

CEL-Net: Continuous Exposure for Extreme Low-Light Imaging

Deep learning methods for enhancing dark images learn a mapping from input images to output images with pre-determined discrete exposure levels. Often, at inference time the input and optimal output exposure levels of the given image are different from the seen ones during training. As a result the enhanced image might suffer from visual distortions, such as low contrast or dark areas. We address this issue by introducing a deep learning model that can continuously generalize at inference time to unseen exposure levels without the need to retrain the model. To this end, we introduce a dataset of 1500 raw images captured in both outdoor and indoor scenes, with five different exposure levels and various camera parameters. Using the dataset, we develop a model for extreme low-light imaging that can continuously tune the input or output exposure level of the image to an unseen one. We investigate the properties of our model and validate its performance, showing promising results

Image Color Correction, Enhancement, and Editing

This thesis presents methods and approaches to image color correction, color enhancement, and color editing. To begin, we study the color correction problem from the standpoint of the camera's image signal processor (ISP). A camera's ISP is hardware that applies a series of in-camera image processing and color manipulation steps, many of which are nonlinear in nature, to render the initial sensor image to its final photo-finished representation saved in the 8-bit standard RGB (sRGB) color space. As white balance (WB) is one of the major procedures applied by the ISP for color correction, this thesis presents two different methods for ISP white balancing. Afterwards, we discuss another scenario of correcting and editing image colors, where we present a set of methods to correct and edit WB settings for images that have been improperly white-balanced by the ISP. Then, we explore another factor that has a significant impact on the quality of camera-rendered colors, in which we outline two different methods to correct exposure errors in camera-rendered images. Lastly, we discuss post-capture auto color editing and manipulation. In particular, we propose auto image recoloring methods to generate different realistic versions of the same camera-rendered image with new colors. Through extensive evaluations, we demonstrate that our methods provide superior solutions compared to existing alternatives targeting color correction, color enhancement, and color editing

Evolutionary Dynamics of Rapid, Microgeographic Adaptation in an Amphibian Metapopulation

Wild organisms can rapidly adapt to changing environments, even at fine spatial scales. This fact prompts hope that contemporary local adaptation may buffer some of the negative anthropogenic impacts to ecosystems. However, there are limits to the pace of adaptation. Understanding the adaptive potential—and limitations—of individual species at fine-resolution is an important task if we hope to accurately predict the repercussions of future climate and landscape change on biodiversity. My dissertation takes advantage of an uncommonly long-observed and closely-studied system to paint a comprehensive picture of evolution over time in association with shifts in ecological contexts. In this dissertation, I show evidence of rapid, microgeographic evolution in response to climate within a metapopulation of wood frogs (Rana sylvatica). Critically, I show that populations separated by tens to hundreds of meters—well within the dispersal ability of the species—exhibited considerable shifts in development rates over a period of two decades, or roughly 6-9 generations. Using historical climate data and new methods of assessing landscape change, I show that these changes were mainly a response to warming climates. The ecological contexts experienced by the metapopulation are associated with the evolution of physiological rates. Specifically, I show that climate change seems to have caused a counter-intuitive delay in spring breeding phenology while drought and warming later in the larval development period correspond with a shift toward earlier metamorphosis. The picture that emerges is of a contracting developmental window, which is expected to select for faster intrinsic development rates. Superimposed on the metapopulation-wide shift to faster development was a pattern of counter-gradient variation reflecting a similar pattern seen two decades prior. Furthermore, I empirically demonstrate a trade-off between faster development and a swimming performance trait that strongly contributes to fitness. This trade-off helps to explain why intrinsic development rates vary spatially with pond temperatures, but in the opposite direction of the relationship with temperature over time. Though the evidence for rapid adaptation to climate change presented in this dissertation reveals that evolution can buffer populations from extinction, it also entreats caution. There is a clear trend of demographic decline among wood frog populations that experienced greater magnitudes of environmental change. In fact, the three populations that suffered local extinctions over the 20-year course of observations inhabited ponds characterized by the greatest change in temperature or canopy

Programmable Image-Based Light Capture for Previsualization

Previsualization is a class of techniques for creating approximate previews of a movie sequence in order to visualize a scene prior to shooting it on the set. Often these techniques are used to convey the artistic direction of the story in terms of cinematic elements, such as camera movement, angle, lighting, dialogue, and character motion. Essentially, a movie director uses previsualization (previs) to convey movie visuals as he sees them in his minds-eye . Traditional methods for previs include hand-drawn sketches, Storyboards, scaled models, and photographs, which are created by artists to convey how a scene or character might look or move. A recent trend has been to use 3D graphics applications such as video game engines to perform previs, which is called 3D previs. This type of previs is generally used prior to shooting a scene in order to choreograph camera or character movements. To visualize a scene while being recorded on-set, directors and cinematographers use a technique called On-set previs, which provides a real-time view with little to no processing. Other types of previs, such as Technical previs, emphasize accurately capturing scene properties but lack any interactive manipulation and are usually employed by visual effects crews and not for cinematographers or directors. This dissertation\u27s focus is on creating a new method for interactive visualization that will automatically capture the on-set lighting and provide interactive manipulation of cinematic elements to facilitate the movie maker\u27s artistic expression, validate cinematic choices, and provide guidance to production crews. Our method will overcome the drawbacks of the all previous previs methods by combining photorealistic rendering with accurately captured scene details, which is interactively displayed on a mobile capture and rendering platform. This dissertation describes a new hardware and software previs framework that enables interactive visualization of on-set post-production elements. A three-tiered framework, which is the main contribution of this dissertation is; 1) a novel programmable camera architecture that provides programmability to low-level features and a visual programming interface, 2) new algorithms that analyzes and decomposes the scene photometrically, and 3) a previs interface that leverages the previous to perform interactive rendering and manipulation of the photometric and computer generated elements. For this dissertation we implemented a programmable camera with a novel visual programming interface. We developed the photometric theory and implementation of our novel relighting technique called Symmetric lighting, which can be used to relight a scene with multiple illuminants with respect to color, intensity and location on our programmable camera. We analyzed the performance of Symmetric lighting on synthetic and real scenes to evaluate the benefits and limitations with respect to the reflectance composition of the scene and the number and color of lights within the scene. We found that, since our method is based on a Lambertian reflectance assumption, our method works well under this assumption but that scenes with high amounts of specular reflections can have higher errors in terms of relighting accuracy and additional steps are required to mitigate this limitation. Also, scenes which contain lights whose colors are a too similar can lead to degenerate cases in terms of relighting. Despite these limitations, an important contribution of our work is that Symmetric lighting can also be leveraged as a solution for performing multi-illuminant white balancing and light color estimation within a scene with multiple illuminants without limits on the color range or number of lights. We compared our method to other white balance methods and show that our method is superior when at least one of the light colors is known a priori

Rehabilitering av sekundär tropisk regnskog på Malaysiska Borneo : tidiga effekter av krontakets sammansättning på ljusförhållanden vid marken

Tropisk regnskog i Sydostasien är ett av de områden som hyser störst biodiversitet i världen, av vilken stora ytor är hotat. Ön Borneo drabbades av en katastrof åren 1982-1983 efter att väderfenomenet El Niño orsakat torka med vidsträckta skogsbränder som följd. Detta lämnade stora ytor av Borneos skogar i ett undermåligt, sekundärt tillstånd. På grund av detta startades INIKEA projektet med syfte att rehabilitera skogar i regionen kring Tawau vid östkusten av delstaten Sabah i Malaysia. I denna studie undersökte jag resultatet av rehabiliteringsarbetet på krontaket i tre olika skogstyper genom att ta hemisfäriska foton med en digital systemkamera (DSLR). Vidare undersökte jag ytterligare tre metoder för att se deras lämplighet i denna typ av skog. Dessa var en sfärisk densiometer, det så kallade Crown Illumination Index (CII) och hemisfäriska foton tagna av en smartphone-kamera. Resultatet visade att skogstypen hade signifikant effekt på variablerna visible sky och Leaf Area Index (LAI) medan rehabiliteringsmetoden hade signifikant effekt på variablerna Direct Site Factor (DSF) och Global Site Factor (GSF). Smartphone fotona hade signifikant korrelation med sina motsvarande DSLR foton samt visade statistisk signifikans på variablerna visible sky och LAI. Densiometern och CII har sina styrkor i att de är enkla, snabba och lätta att bära, de är dock föremål för subjektivitet. Densiometern visade störst likhet med DSF och GSF medan CII visade signifikant korrelation med alla variabler förutom LAI.Tropical rainforests of South East Asia holds some of the biggest biodiversity in the world, with big parts of it being under threat. The island of Borneo was stricken by disaster in 1982-83, when following a weather phenomenon the El Niño Southern Oscillation, vast droughts and subsequent fires ravaged its forests. This left major parts of Borneo’s forest in a secondary state and as a result the INIKEA rehabilitation project was initiated in the Tawau region on the east coast in the state of Sabah, Malaysia. In this study I examined the outcome of the rehabilitation work on the forest canopy in three different forest types. This was done by means of hemispherical photographing using a DSLR camera. Further, three other methods was examined to see their suitability in this type of forest. These were a spherical densiometer, the Crown Illumination Index (CII), and hemispherical photographs taken by a smartphone camera. The results showed that forest type had a significant effect on the amount of visible sky and Leaf Area Index (LAI) while type of rehabilitation treatment had a significant effect on Direct Site Factor (DSF) and Global Site Factor (GSF). The smartphone photographs had a significant correlation with their DSLR counterparts, and also showed a statistical significance for visible sky and LAI signifying that it may be a tool in the future, should certain considerations be made. The densiometer and CII holds advantages in being easy, quick and non-cumbersome but are instead prone to subjectivity. The densiometer had best similarities with DSF and GSF and the CII showed significant correlation with all variables except for LAI

Machine vision detection of crop diseases

The production of agricultural crops such as wheat is a multi-billion dollar industry. Each year, diseases such as fungal infections can potentially destroy the entire production in a region if the conditions are right. Traditional mitigation has involved the grower's own judgement as the first option, followed by use of trained professionals for confirmation of diagnosis and advice. Often the onset of infection is rapid, and the professionals may not always be available to assess the situation before the infection spreads. This reliance on individual judgement and off-site experts highlights a need to develop a reliable, automated software solution which can provide an accurate and immediate software diagnosis at the first sign of infection. This research has developed into two solutions: a system to help agronomists by using professional camera equipment, as well as the outline for a mobile solution which can operate on a `smart' device to provide growers with an on-hand diagnosis tool. By investigating the way the human mind and eye works, a software emulation of the human visual system was constructed, with artificial intelligence approaches used for final interpretation of the optical response. This use of artificial intelligence has allowed for the design of a robust system which can `self-learn' to recognise any new disease samples. Research involved investigation of a number of camera and hardware options. Final system validation was conducted on both `stock' disease images provided by agronomists, and on actual plant samples, which proved that the system could function across a broad range of diseases and crops with a degree of accuracy between 95-99%. This research indicates that it is possible to develop tools which can give an immediate analysis at all stages of infection, and be robust enough to work over a range of diseases and crops. Further development and refinement would provide a useful diagnosis tool for both growers and experts

Source Camera Verification from Strongly Stabilized Videos

Image stabilization performed during imaging and/or post-processing poses one of the most significant challenges to photo-response non-uniformity based source camera attribution from videos. When performed digitally, stabilization involves cropping, warping, and inpainting of video frames to eliminate unwanted camera motion. Hence, successful attribution requires the inversion of these transformations in a blind manner. To address this challenge, we introduce a source camera verification method for videos that takes into account the spatially variant nature of stabilization transformations and assumes a larger degree of freedom in their search. Our method identifies transformations at a sub-frame level, incorporates a number of constraints to validate their correctness, and offers computational flexibility in the search for the correct transformation. The method also adopts a holistic approach in countering disruptive effects of other video generation steps, such as video coding and downsizing, for more reliable attribution. Tests performed on one public and two custom datasets show that the proposed method is able to verify the source of 23-30% of all videos that underwent stronger stabilization, depending on computation load, without a significant impact on false attribution

Visualising lighting simulations for automotive design evaluations using emerging technologies

Automotive design visualisation is at a turning point with the commercial development of immersive technologies such as virtual reality, among other displays and visual interfaces. A fundamental objective of this research is to assess how seamlessly the integration of emerging visualisation technologies can be implemented into the new product development methodologies, with the use of lighting simulation, design review applications and the use of immersive hardware and software. Optical automotive considerations such as display legibility, veiling glare, and perceived quality among other current processes of Systemic Optical Failure (SOF) modes are analysed, to determine how the application of new immersive visualisation technologies could improve the efficiency of new product development, in particular reducing time and cost in early stages while improving decision making and quality. Different hardware and software combinations were investigated in terms of their ability to realistically represent design intent. Following on from this investigation, a user study was carried out with subjects from various automotive engineering disciplines, to evaluate a range of potential solutions. Recommendations are then made as to how these solutions could be deployed within the automotive new product development process to deliver maximum value