Simple, Inexpensive Technique for High-Quality Smartphone Fundus Photography in Human and Animal Eyes

Purpose. We describe in detail a relatively simple technique of fundus photography in human and rabbit eyes using a smartphone, an inexpensive app for the smartphone, and instruments that are readily available in an ophthalmic practice. Methods:. Fundus images were captured with a smartphone and a 20D lens with or without a Koeppe lens. By using the coaxial light source of the phone, this system works as an indirect ophthalmoscope that creates a digital image of the fundus. The application whose software allows for independent control of focus, exposure, and light intensity during video filming was used. With this app, we recorded high-definition videos of the fundus and subsequently extracted high-quality, still images from the video clip. Results:. The described technique of smartphone fundus photography was able to capture excellent high-quality fundus images in both children under anesthesia and in awake adults. Excellent images were acquired with the 20D lens alone in the clinic, and the addition of the Koeppe lens in the operating room resulted in the best quality images. Successful photodocumentation of rabbit fundus was achieved in control and experimental eyes. Conclusion:. The currently described system was able to take consistently high-quality fundus photographs in patients and in animals using readily available instruments that are portable with simple power sources. It is relatively simple to master, is relatively inexpensive, and can take advantage of the expanding mobile-telephone networks for telemedicine

A Novel Approach for Neuromorphic Vision Data Compression based on Deep Belief Network

A neuromorphic camera is an image sensor that emulates the human eyes capturing only changes in local brightness levels. They are widely known as event cameras, silicon retinas or dynamic vision sensors (DVS). DVS records asynchronous per-pixel brightness changes, resulting in a stream of events that encode the brightness change's time, location, and polarity. DVS consumes little power and can capture a wider dynamic range with no motion blur and higher temporal resolution than conventional frame-based cameras. Although this method of event capture results in a lower bit rate than traditional video capture, it is further compressible. This paper proposes a novel deep learning-based compression scheme for event data. Using a deep belief network (DBN), the high dimensional event data is reduced into a latent representation and later encoded using an entropy-based coding technique. The proposed scheme is among the first to incorporate deep learning for event compression. It achieves a high compression ratio while maintaining good reconstruction quality outperforming state-of-the-art event data coders and other lossless benchmark techniques

PhotoApp: Photorealistic Appearance Editing of Head Portraits

Photorealistic editing of head portraits is a challenging task as humans are very sensitive to inconsistencies in faces. We present an approach for high-quality intuitive editing of the camera viewpoint and scene illumination (parameterised with an environment map) in a portrait image. This requires our method to capture and control the full reflectance field of the person in the image. Most editing approaches rely on supervised learning using training data captured with setups such as light and camera stages. Such datasets are expensive to acquire, not readily available and do not capture all the rich variations of in-the-wild portrait images. In addition, most supervised approaches only focus on relighting, and do not allow camera viewpoint editing. Thus, they only capture and control a subset of the reflectance field. Recently, portrait editing has been demonstrated by operating in the generative model space of StyleGAN. While such approaches do not require direct supervision, there is a significant loss of quality when compared to the supervised approaches. In this paper, we present a method which learns from limited supervised training data. The training images only include people in a fixed neutral expression with eyes closed, without much hair or background variations. Each person is captured under 150 one-light-at-a-time conditions and under 8 camera poses. Instead of training directly in the image space, we design a supervised problem which learns transformations in the latent space of StyleGAN. This combines the best of supervised learning and generative adversarial modeling. We show that the StyleGAN prior allows for generalisation to different expressions, hairstyles and backgrounds. This produces high-quality photorealistic results for in-the-wild images and significantly outperforms existing methods. Our approach can edit the illumination and pose simultaneously, and runs at interactive rates

PhotoApp: Photorealistic Appearance Editing of Head Portraits

Photorealistic editing of portraits is a challenging task as humans are very sensitive to inconsistencies in faces. We present an approach for high-quality intuitive editing of the camera viewpoint and scene illumination in a portrait image. This requires our method to capture and control the full reflectance field of the person in the image. Most editing approaches rely on supervised learning using training data captured with setups such as light and camera stages. Such datasets are expensive to acquire, not readily available and do not capture all the rich variations of in-the-wild portrait images. In addition, most supervised approaches only focus on relighting, and do not allow camera viewpoint editing. Thus, they only capture and control a subset of the reflectance field. Recently, portrait editing has been demonstrated by operating in the generative model space of StyleGAN. While such approaches do not require direct supervision, there is a significant loss of quality when compared to the supervised approaches. In this paper, we present a method which learns from limited supervised training data. The training images only include people in a fixed neutral expression with eyes closed, without much hair or background variations. Each person is captured under 150 one-light-at-a-time conditions and under 8 camera poses. Instead of training directly in the image space, we design a supervised problem which learns transformations in the latent space of StyleGAN. This combines the best of supervised learning and generative adversarial modeling. We show that the StyleGAN prior allows for generalisation to different expressions, hairstyles and backgrounds. This produces high-quality photorealistic results for in-the-wild images and significantly outperforms existing methods. Our approach can edit the illumination and pose simultaneously, and runs at interactive rates.Comment: http://gvv.mpi-inf.mpg.de/projects/PhotoApp

A Smartphone-Based Tool for Rapid, Portable, and Automated Wide-Field Retinal Imaging.

Purpose:High-quality, wide-field retinal imaging is a valuable method for screening preventable, vision-threatening diseases of the retina. Smartphone-based retinal cameras hold promise for increasing access to retinal imaging, but variable image quality and restricted field of view can limit their utility. We developed and clinically tested a smartphone-based system that addresses these challenges with automation-assisted imaging. Methods:The system was designed to improve smartphone retinal imaging by combining automated fixation guidance, photomontage, and multicolored illumination with optimized optics, user-tested ergonomics, and touch-screen interface. System performance was evaluated from images of ophthalmic patients taken by nonophthalmic personnel. Two masked ophthalmologists evaluated images for abnormalities and disease severity. Results:The system automatically generated 100° retinal photomontages from five overlapping images in under 1 minute at full resolution (52.3 pixels per retinal degree) fully on-phone, revealing numerous retinal abnormalities. Feasibility of the system for diabetic retinopathy (DR) screening using the retinal photomontages was performed in 71 diabetics by masked graders. DR grade matched perfectly with dilated clinical examination in 55.1% of eyes and within 1 severity level for 85.2% of eyes. For referral-warranted DR, average sensitivity was 93.3% and specificity 56.8%. Conclusions:Automation-assisted imaging produced high-quality, wide-field retinal images that demonstrate the potential of smartphone-based retinal cameras to be used for retinal disease screening. Translational Relevance:Enhancement of smartphone-based retinal imaging through automation and software intelligence holds great promise for increasing the accessibility of retinal screening

Performance tuning of a smartphone-based overtaking assistant

ITS solutions suffer from the slow pace of adoption by manufacturers despite the interest shown by both consumers and industry. Our goal is to develop ITS applications using already available technologies to make them affordable, quick to deploy, and easy to adopt. In this paper we introduce EYES, an overtaking assistance solution that provides drivers with a real-time video feed from the vehicle located just in front. Our application thus provides a better view of the road ahead, and of any vehicles travelling in the opposite direction, being especially useful when the front view of the driver is blocked by large vehicles. We evaluated our application using the MJPEG video encoding format, and have determined the most effective resolution and JPEG quality choice for our case. Experimental results from the tests performed with the application in both indoor and outdoor scenarios, allow us to be optimistic about the effectiveness and applicability of smartphones in providing overtaking assistance based on video streaming in vehicular networks

Using facial feature extraction to enhance the creation of 3D human models

The creation of personalised 3D characters has evolved to provide a high degree of realism in both appearance and animation. Further to the creation of generic characters the capabilities exist to create a personalised character from images of an individual. This provides the possibility of immersing an individual into a virtual world. Feature detection, particularly on the face, can be used to greatly enhance the realism of the model. To address this innovative contour based templates are used to extract an individual from four orthogonal views providing localisation of the face. Then adaptive facial feature extraction from multiple views is used to enhance the realism of the model

A joint motion & disparity motion estimation technique for 3D integral video compression using evolutionary strategy

3D imaging techniques have the potential to establish a future mass-market in the fields of entertainment and communications. Integral imaging, which can capture true 3D color images with only one camera, has been seen as the right technology to offer stress-free viewing to audiences of more than one person. Just like any digital video, 3D video sequences must also be compressed in order to make it suitable for consumer domain applications. However, ordinary compression techniques found in state-of-the-art video coding standards such as H.264, MPEG-4 and MPEG-2 are not capable of producing enough compression while preserving the 3D clues. Fortunately, a huge amount of redundancies can be found in an integral video sequence in terms of motion and disparity. This paper discusses a novel approach to use both motion and disparity information to compress 3D integral video sequences. We propose to decompose the integral video sequence down to viewpoint video sequences and jointly exploit motion and disparity redundancies to maximize the compression. We further propose an optimization technique based on evolutionary strategies to minimize the computational complexity of the joint motion disparity estimation. Experimental results demonstrate that Joint Motion and Disparity Estimation can achieve over 1 dB objective quality gain over normal motion estimation. Once combined with Evolutionary strategy, this can achieve up to 94% computational cost saving