A deep learning framework for quality assessment and restoration in video endoscopy

Endoscopy is a routine imaging technique used for both diagnosis and minimally invasive surgical treatment. Artifacts such as motion blur, bubbles, specular reflections, floating objects and pixel saturation impede the visual interpretation and the automated analysis of endoscopy videos. Given the widespread use of endoscopy in different clinical applications, we contend that the robust and reliable identification of such artifacts and the automated restoration of corrupted video frames is a fundamental medical imaging problem. Existing state-of-the-art methods only deal with the detection and restoration of selected artifacts. However, typically endoscopy videos contain numerous artifacts which motivates to establish a comprehensive solution. We propose a fully automatic framework that can: 1) detect and classify six different primary artifacts, 2) provide a quality score for each frame and 3) restore mildly corrupted frames. To detect different artifacts our framework exploits fast multi-scale, single stage convolutional neural network detector. We introduce a quality metric to assess frame quality and predict image restoration success. Generative adversarial networks with carefully chosen regularization are finally used to restore corrupted frames. Our detector yields the highest mean average precision (mAP at 5% threshold) of 49.0 and the lowest computational time of 88 ms allowing for accurate real-time processing. Our restoration models for blind deblurring, saturation correction and inpainting demonstrate significant improvements over previous methods. On a set of 10 test videos we show that our approach preserves an average of 68.7% which is 25% more frames than that retained from the raw videos.Comment: 14 page

Inkjet Printing of Functional Electronic Memory Cells: A Step Forward to Green Electronics

open access journalNowadays, the environmental issues surrounding the production of electronics, from the perspectives of both the materials used and the manufacturing process, are of major concern. The usage, storage, disposal protocol and volume of waste material continue to increase the environmental footprint of our increasingly “throw away society”. Almost ironically, society is increasingly involved in pollution prevention, resource consumption issues and post-consumer waste management. Clearly, a dichotomy between environmentally aware usage and consumerism exists. The current technology used to manufacture functional materials and electronic devices requires high temperatures for material deposition processes, which results in the generation of harmful chemicals and radiation. With such issues in mind, it is imperative to explore new electronic functional materials and new manufacturing pathways. Here, we explore the potential of additive layer manufacturing, inkjet printing technology which provides an innovative manufacturing pathway for functional materials (metal nanoparticles and polymers), and explore a fully printed two terminal electronic memory cell. In this work, inkjetable materials (silver (Ag) and poly(3,4-ethylenedioxythiophene)-poly(styrenesulfonate) (PEDOT:PSS)) were first printed by a piezoelectric Epson Stylus P50 inkjet printer as stand-alone layers, and secondly as part of a metal (Ag)/active layer (PEDOT:PSS)/metal (Ag) crossbar architecture. The quality of the individual multi-layers of the printed Ag and PEDOT:PSS was first evaluated via optical microscopy and scanning electron microscopy (SEM). Furthermore, an electrical characterisation of the printed memory elements was performed using an HP4140B picoammeter

DeepKey: Towards End-to-End Physical Key Replication From a Single Photograph

This paper describes DeepKey, an end-to-end deep neural architecture capable of taking a digital RGB image of an 'everyday' scene containing a pin tumbler key (e.g. lying on a table or carpet) and fully automatically inferring a printable 3D key model. We report on the key detection performance and describe how candidates can be transformed into physical prints. We show an example opening a real-world lock. Our system is described in detail, providing a breakdown of all components including key detection, pose normalisation, bitting segmentation and 3D model inference. We provide an in-depth evaluation and conclude by reflecting on limitations, applications, potential security risks and societal impact. We contribute the DeepKey Datasets of 5, 300+ images covering a few test keys with bounding boxes, pose and unaligned mask data.Comment: 14 pages, 12 figure

Single-shot layered reflectance separation using a polarized light field camera

We present a novel computational photography technique for single shot separation of diffuse/specular reflectance as well as novel angular domain separation of layered reflectance. Our solution consists of a two-way polarized light field (TPLF) camera which simultaneously captures two orthogonal states of polarization. A single photograph of a subject acquired with the TPLF camera under polarized illumination then enables standard separation of diffuse (depolarizing) and polarization preserving specular reflectance using light field sampling. We further demonstrate that the acquired data also enables novel angular separation of layered reflectance including separation of specular reflectance and single scattering in the polarization preserving component, and separation of shallow scattering from deep scattering in the depolarizing component. We apply our approach for efficient acquisition of facial reflectance including diffuse and specular normal maps, and novel separation of photometric normals into layered reflectance normals for layered facial renderings. We demonstrate our proposed single shot layered reflectance separation to be comparable to an existing multi-shot technique that relies on structured lighting while achieving separation results under a variety of illumination conditions