Yleiskäyttöisen kuvankäsittelyliukuhihnan suunnittelu ja toteutus kamerapuhelimiin

Valtaosa nykyisistä edullisen hintaluokan matkapuhelimista sisältää kameran, joka käyttää yksinkertaista optiikkaa ja halpaa kamerasensoria. Riittävän kuvanlaadun takaamiseksi kuvaa käsitellään kuvankäsittelyalgoritmeilla, jotka yhdessä muodostavat kuvankäsittelyliukuhihnan. Paras suorituskyky saavutetaan yleensä rautapohjaisella liukuhihnalla. Ohjelmistopohjaisia ratkaisuja voidaan kuitenkin suosia tuotantokustannusten minimoimiseksi ja liukuhihnan joustavuuden sekä laajennettavuuden parantamiseksi. Jotta liukuhihnan muistinkulutus voidaan minimoida vähämuistisessa ympäristössä, on järkevää toteuttaa ohjelmistopohjainen kuvankäsittelyliukuhihna käyttäen juovapuskureita. Juovapuskurit monimutkaistavat liukuhihnan hallintaa, mikä kuitenkin on ratkaistavissa automaation avulla. Tämä diplomityö esittelee yleiskäyttöisen ohjelmistokehyksen juovapuskuripohjaiselle kuvankäsittelyliukuhihnalle. Ohjelmistokehys soveltuu vähämuistisiin ympäristöihin ja helpottaa kuvankäsittelyalgoritmien lisäämistä, poistamista ja muuttamista sekä muita liukuhihnan hallintatehtäviä merkittävästi. Säästöt kehitysajassa voivat olla jopa kuukausia. Ohjelmistokehyksen suorituskykyä ja muistinkäyttöä verrataan nykyisiin toteutuksiin käyttäen todellista kuvankäsittelyliukuhihnaa testitapauksena. Työssä pohditaan myös, kuinka juovapuskuripohjainen liukuhihna voitaisiin rinnakkaistaa entistä paremman suorituskyvyn saavuttamiseksi moniydinpuhelimissa. Rinnakkaistetun liukuhihan toteuttamiseksi esitellään kaksi lähestymistapaa: dataviipaloitu liukuhihna ja työjonopohjainen liukuhihna. Tutkimus osoittaa, että ohjelmistokehys säästää muistia yli 99% verrattuna perinteisiin toteutuksiin, jotka käyttävät ping-pong puskurointia täyden koon kuvapuskureilla. Toteutettu ohjelmistokehys parantaa myös suorituskykyä paremman välimuistin käytön ansiosta ja lisää kuvankäsittelyliukuhihnan joustavuutta useilla erilaisilla konfiguraatioilla.The majority of the current affordable mobile devices contain a camera with simple optics and a low-cost camera sensor. In these devices, the quality of the captured images is made acceptable with various image processing algorithms that together form an image reconstruction pipeline. The best performance is often achieved with a hardware pipeline, but software implementations can be preferred to minimize production costs and to maximize flexibility. To minimize memory consumption in such a limited-resource environment, it is reasonable to implement a software-based image reconstruction pipeline using line buffers. The line buffers complicate the management of the pipeline, which, however, can be solved by increasing development tool automatization. This Thesis presents a generic software framework for a line-buffer-based image reconstruction pipeline. The presented framework is capable of operating in low-memory environments and significantly eases algorithm insertions, changes of processing order, and other pipeline management tasks. The savings in development time can be even months. The performance and memory usage of the software framework is compared to contemporary implementations by using a real image reconstruction pipeline as the test case. The Thesis also discusses how the line-buffer-based pipeline could be parallelized to achieve improved performance on multi-core devices. Two promising approaches are considered: slice-based parallelization and work-queue-based parallelization. The experiments show that the software framework offers over 99% memory savings compared with traditional implementations using a ping-pong buffer scheme with full-sized image buffers. The implemented framework also enhances processing performance due to better cache usage and increases flexibility with various pipeline configurations

Automaattisten kuvantamisalgoritmien parantaminen kahden kameran järjestelmä avulla

Mobile camera users trust widely to the automatic camera settings, but the automatic settings do not perform equally well in all situations. All of the automatic settings have their own algorithms, which their operations are based on. Automatic white balance, automatic exposure control and automatic focus are known automatic camera settings. Dual camera system can improve the automatic settings in mobile cameras. In this thesis dual camera algorithms were researched and dual camera algorithm development framework was created. Instead of creating the entire algorithm development framework, the dual camera functionality was added to the existing single camera algorithm development framework. The existing algorithm development framework was a PC application, which could perform the same image processing steps as mobile phone camera image signal processors. The dual camera development framework is able to collect image parameters and markers from one image and provide these to another image. The dual camera development framework manages the parameter mediation by writing and reading XML files. Few dual camera algorithms were also researched in order to proof the dual camera framework concept. It turned out that the dual camera is able to improve image quality and the automatic camera settings. The automatic exposure control can be improved by collecting additional data from the histogram and by using this information when producing the final image. Motion blur can also be reduced by dual camera algorithm that compares edge data from two distinct images with each other and then provides valuable information about the motion to the exposure control algorithm. The automatic exposure control algorithm can thus reduce exposure time and increase the sensor sensitivity in order to produce sharp images. Automatic white balance can be improved with dual camera system by estimating illumination source with dual camera system. Even though the dual camera seems to provide some advantages to the algorithms, and the concept that the dual camera development framework introduces, seems valid, addition of second camera to mobile phone should be considered carefully. Many of the dual camera algorithms could be transformed to work with only one camera. The dual camera algorithm development takes also a large amount of time and effort, given that the dual camera is not going to be applied to all future camera phones

Virtual Cinematography: Beyond Big Studio Production

In the current production environment, the ability to previsualize shots utilizing a virtual camera system requires expensive hardware and large motion capture spaces only available to large studio environments. By leveraging consumer-level technologies such as tablets and motion gaming controllers as well as merging the cinematic techniques of film with the real-time benefits of game engines, it is possible to develop a hybrid interface that would lower the barrier of entry for virtual production. Utilizing affordable hardware, an intuitive user interface, and an intelligent camera system, the SmartVCS is a new virtual inematography platform that provides professional directors as well as a new market of amateur filmmakers the ability to previsualize their films or game cinematics with familiar and accessible technology. This system has potential applications to other areas including game level design, real-time compositing & post-production, and architectural visualization. In addition, this system has the ability to expand as a human-computer interface for video games, robotics, and medicine as a functional hybrid freespace input device.M.S., Digital Media -- Drexel University, 201

Camera functionality in modern mobile terminals : software FIR filter for demosaicing with RISC

Masteroppgave i informasjons- og kommunikasjonsteknologi 2001 - Høgskolen i Agder, GrimstadThis thesis discusses implementation and optimizing of the image pre-processing operation demosaicing on a typical hardware architecture for mobile terminals. Work has shown that the demosaicing operation is equivalent to the mathematical operation of interpolation, and that optimizing potential lies within the implementation of the anti-imaging filter for the interpolator. This filter can be realized as an FIR filter with the filter coefficients obtained from a suitable interpolation function (kernel). Through literature review, nearest neighbor replication, bilinear and cubic convolution kernels were implemented and optimized according to a typical hardware architecture found in mobile terminals. Typical hardware architecture used consisted of ARM710T RISC processor with 8 KB on-chip cache and external SDRAM memory. The work has shown that the cubic convolution interpolation kernel offers the best image quality requiring only moderate computational effort. Though, dependent the application area, and taking the whole camera system into consideration, bilinear interpolation might be better suited as interpolation kernel for the demosaicing operation in the mobile terminal. The bilinear interpolation offers a good trade-off between the visual result and computational effort. The nearest neighbor replication interpolation kernel offers the poorest image quality, but is the most computational efficient of the kernels

Enabling Real-Time Shared Environments on Mobile Head-Mounted Displays

Head-Mounted Displays (HMDs) are becoming more prevalent consumer devices, allowing users to experience scenes and environments from a point of view naturally controlled by their movement. However there is limited application of this experiential paradigm to telecommunications -- that is, where a HMD user can 'call' a mobile phone user and begin to look around in their environment. In this thesis we present a telepresence system for connecting mobile phone users with people wearing HMDs, allowing the HMD user to experience the environment of the mobile user in real-time. We developed an Android application that supports generating and transmitting high quality spherical panorama based environments in real-time, and a companion application for HMDs to view those environments live. This thesis focusses on the technical challenges involved with creating panoramic environments of sufficient quality to be suitable for viewing inside a HMD, given the constraints that arise from using mobile phones. We present computer vision techniques optimised for these constrained conditions, justifying the trade-offs made between speed and quality. We conclude by comparing our solution to conceptually similar past research along the metrics of computation speed and output quality