Real-time refocusing using an FPGA-based standard plenoptic camera

Plenoptic cameras are receiving increased attention in scientific and commercial applications because they capture the entire structure of light in a scene, enabling optical transforms (such as focusing) to be applied computationally after the fact, rather than once and for all at the time a picture is taken. In many settings, real-time inter active performance is also desired, which in turn requires significant computational power due to the large amount of data required to represent a plenoptic image. Although GPUs have been shown to provide acceptable performance for real-time plenoptic rendering, their cost and power requirements make them prohibitive for embedded uses (such as in-camera). On the other hand, the computation to accomplish plenoptic rendering is well structured, suggesting the use of specialized hardware. Accordingly, this paper presents an array of switch-driven finite impulse response filters, implemented with FPGA to accomplish high-throughput spatial-domain rendering. The proposed architecture provides a power-efficient rendering hardware design suitable for full-video applications as required in broadcasting or cinematography. A benchmark assessment of the proposed hardware implementation shows that real-time performance can readily be achieved, with a one order of magnitude performance improvement over a GPU implementation and three orders ofmagnitude performance improvement over a general-purpose CPU implementation

A Survey of Signal Processing Problems and Tools in Holographic Three-Dimensional Television

Cataloged from PDF version of article.Diffraction and holography are fertile areas for application of signal theory and processing. Recent work on 3DTV displays has posed particularly challenging signal processing problems. Various procedures to compute Rayleigh-Sommerfeld, Fresnel and Fraunhofer diffraction exist in the literature. Diffraction between parallel planes and tilted planes can be efficiently computed. Discretization and quantization of diffraction fields yield interesting theoretical and practical results, and allow efficient schemes compared to commonly used Nyquist sampling. The literature on computer-generated holography provides a good resource for holographic 3DTV related issues. Fast algorithms to compute Fourier, Walsh-Hadamard, fractional Fourier, linear canonical, Fresnel, and wavelet transforms, as well as optimization-based techniques such as best orthogonal basis, matching pursuit, basis pursuit etc., are especially relevant signal processing techniques for wave propagation, diffraction, holography, and related problems. Atomic decompositions, multiresolution techniques, Gabor functions, and Wigner distributions are among the signal processing techniques which have or may be applied to problems in optics. Research aimed at solving such problems at the intersection of wave optics and signal processing promises not only to facilitate the development of 3DTV systems, but also to contribute to fundamental advances in optics and signal processing theory. © 2007 IEEE

A review of snapshot multidimensional optical imaging: Measuring photon tags in parallel

Multidimensional optical imaging has seen remarkable growth in the past decade. Rather than measuring only the two-dimensional spatial distribution of light, as in conventional photography, multidimensional optical imaging captures light in up to nine dimensions, providing unprecedented information about incident photons’ spatial coordinates, emittance angles, wavelength, time, and polarization. Multidimensional optical imaging can be accomplished either by scanning or parallel acquisition. Compared with scanning-based imagers, parallel acquisition–also dubbed snapshot imaging–has a prominent advantage in maximizing optical throughput, particularly when measuring a datacube of high dimensions. Here, we first categorize snapshot multidimensional imagers based on their acquisition and image reconstruction strategies, then highlight the snapshot advantage in the context of optical throughput, and finally we discuss their state-of-the-art implementations and applications

What about computational super-resolution in fluorescence Fourier light field microscopy?

Recently, Fourier light field microscopy was proposed to overcome the limitations in conventional light field microscopy by placing a micro-lens array at the aperture stop of the microscope objective instead of the image plane. In this way, a collection of orthographic views from different perspectives are directly captured. When inspecting fluorescent samples, the sensitivity and noise of the sensors are a major concern and large sensor pixels are required to cope with low-light conditions, which implies under-sampling issues. In this context, we analyze the sampling patterns in Fourier light field microscopy to understand to what extent computational super-resolution can be triggered during deconvolution in order to improve the resolution of the 3D reconstruction of the imaged data

Computational Light Transport for Forward and Inverse Problems.

El transporte de luz computacional comprende todas las técnicas usadas para calcular el flujo de luz en una escena virtual. Su uso es ubicuo en distintas aplicaciones, desde entretenimiento y publicidad, hasta diseño de producto, ingeniería y arquitectura, incluyendo el generar datos validados para técnicas basadas en imagen por ordenador. Sin embargo, simular el transporte de luz de manera precisa es un proceso costoso. Como consecuencia, hay que establecer un balance entre la fidelidad de la simulación física y su coste computacional. Por ejemplo, es común asumir óptica geométrica o una velocidad de propagación de la luz infinita, o simplificar los modelos de reflectancia ignorando ciertos fenómenos. En esta tesis introducimos varias contribuciones a la simulación del transporte de luz, dirigidas tanto a mejorar la eficiencia del cálculo de la misma, como a expandir el rango de sus aplicaciones prácticas. Prestamos especial atención a remover la asunción de una velocidad de propagación infinita, generalizando el transporte de luz a su estado transitorio. Respecto a la mejora de eficiencia, presentamos un método para calcular el flujo de luz que incide directamente desde luminarias en un sistema de generación de imágenes por Monte Carlo, reduciendo significativamente la variancia de las imágenes resultantes usando el mismo tiempo de ejecución. Asimismo, introducimos una técnica basada en estimación de densidad en el estado transitorio, que permite reusar mejor las muestras temporales en un medio parcipativo. En el dominio de las aplicaciones, también introducimos dos nuevos usos del transporte de luz: Un modelo para simular un tipo especial de pigmentos gonicromáticos que exhiben apariencia perlescente, con el objetivo de proveer una forma de edición intuitiva para manufactura, y una técnica de imagen sin línea de visión directa usando información del tiempo de vuelo de la luz, construida sobre un modelo de propagación de la luz basado en ondas.<br /

The 1990 Johnson Space Center bibliography of scientific and technical papers

Abstracts are presented of scientific and technical papers written and/or presented by L. B. Johnson Space Center (JSC) authors, including civil servants, contractors, and grantees, during the calendar year of 1990. Citations include conference and symposium presentations, papers published in proceedings or other collective works, seminars, and workshop results, NASA formal report series (including contractually required final reports), and articles published in professional journals

The standard plenoptic camera: applications of a geometrical light field model

A thesis submitted to the University of Bedfordshire, in partial fulfilment of the requirements for the degree of Doctor of PhilosophyThe plenoptic camera is an emerging technology in computer vision able to capture a light field image from a single exposure which allows a computational change of the perspective view just as the optical focus, known as refocusing. Until now there was no general method to pinpoint object planes that have been brought to focus or stereo baselines of perspective views posed by a plenoptic camera. Previous research has presented simplified ray models to prove the concept of refocusing and to enhance image and depth map qualities, but lacked promising distance estimates and an efficient refocusing hardware implementation. In this thesis, a pair of light rays is treated as a system of linear functions whose solution yields ray intersections indicating distances to refocused object planes or positions of virtual cameras that project perspective views. A refocusing image synthesis is derived from the proposed ray model and further developed to an array of switch-controlled semi-systolic FIR convolution filters. Their real-time performance is verified through simulation and implementation by means of an FPGA using VHDL programming. A series of experiments is carried out with different lenses and focus settings, where prediction results are compared with those of a real ray simulation tool and processed light field photographs for which a blur metric has been considered. Predictions accurately match measurements in light field photographs and signify deviations of less than 0.35 % in real ray simulation. A benchmark assessment of the proposed refocusing hardware implementation suggests a computation time speed-up of 99.91 % in comparison with a state-of-the-art technique. It is expected that this research supports in the prototyping stage of plenoptic cameras and microscopes as it helps specifying depth sampling planes, thus localising objects and provides a power-efficient refocusing hardware design for full-video applications as in broadcasting or motion picture arts

SESS Report 2021 The State of Environmental Science in Svalbard - an annual report

Executive Summary The State of Environmental Science in Svalbard (SESS) report 2021 together with its predecessors contributes to the documentation of the state of the Arctic environment in and around Svalbard, and highlights research conducted within the Svalbard Integrated Arctic Earth Observing System (SIOS). Climate change is a global problem, but many of its impacts are being felt most strongly in the Arctic. Given its remote but accessible location, Svalbard constitutes an ideal place to study the Arctic environment in general, including, more specifically, the causes and consequences of climate change. The Arctic Climate Change Update (2021) emphasised the severity of global climate change for ecosystems across the Arctic. They are undergoing radical changes regarding their structure and functioning, affecting flora, fauna and livelihoods of Arctic communities. Oceanic ecosystems and food webs are directly and indirectly altered by the warming and freshening of the Arctic Ocean. A prolonged open water period and the expansion of open water areas caused by declining sea ice affect under-ice productivity and diversity. These changes have cascading effects through ecosystems and impact the distribution, abundance and seasonality of a variety of marine species. Svalbard is located at one of the key oceanic gateways to the Arctic. This land–ice–ocean transition zone is a system particularly vulnerable to environmental changes. Svalbard’s environment is influenced by maritime processes; thus extensive observation of the ocean system is nowadays necessary. The chapter on the iMOP project reports seawater temperature and salinity variability over the last decades and indicates changes of Svalbard fjord seawater properties. The chapter highlights the role of a collaborative and supportive network of observatory operators and encourages joint planning and maintenance of future marine observatories. Arctic vegetation plays a key role in land–atmosphere interactions. Alterations can lead to ecosystem–climate feedbacks and exacerbate climate change. Extreme precipitation events are already becoming more frequent. Together with an increasing rain-to-snow ratio they impact the structure and functioning of terrestrial ecosystems. Dynamics in Arctic tundra ecosystems are expected to undergo fundamental changes with increasing temperatures as predicted by climate models. To detect, document, understand and predict those changes, COAT Svalbard provides a long-term and real-time operational observation system through ecosystem-based terrestrial monitoring. The observation system consists of six modules comprising food web pathways as well as one climate-monitoring module and focuses on two contrasting regions in Svalbard to allow for intercomparison. To date, the project has done an initial assessment of tundra ecosystems in Norway and will now begin with the long-term ecosystembased monitoring. For remote regions such as the Svalbard archipelago, terrestrial photography is a crucial addition to satellite imagery, because land-based cameras offer high temporal resolution and insensitivity towards varying weather conditions. PASSES provides an overview of cameras operating in Svalbard managed by research institutions and private companies. The survey revealed difficulties and knowledge gaps preventing the full potential of the terrestrial photography network in Svalbard from being used. Therefore, PASSES recommends the creation of a Svalbard camera system network. The effects of climate change contributed to a specific anomaly of the springtime Arctic atmosphere, namely a pronounced depletion of stratospheric ozone during March and April 2020, which can be called an Arctic ozone hole. In Svalbard, the amount of ozone loss was recorded by ground-based dedicated spectroscopic instruments measuring the total ozone column as well as the UV irradiance (EXAODEP-2020, an update of UV Ozone). The latter is important for effects on the biota. Corresponding erythemal daily doses for spring 2020 show a doubling compared to previous years with less or no ozone depletion. While the correspondence between ozone loss and increase in UV doses follows a well-known relationship, the possible later consequences of the observed springtime increase of UV doses on Svalbard’s environment need to be further studied. A particular method to observe the Svalbard environment, which has seen a very strong increase in usage during recent years, is the application of unmanned airborne or marine vehicles. The update on recent publications using these devices (UAV Svalbard) reveals that especially conventional remotely operated aerial vehicles (drones) with camera equipment are now widely used. It is recommended to SIOS to foster interdisciplinary communication among the multitude of drone users to establish exchange of information and data. New EU regulations for drone operations are being put in place from 2022 onwards also in Svalbard. Climate services are receiving more and more attention from Arctic countries, because they translate data into relevant and timely information, thereby supporting governments, societies and industries in planning and decision-making processes. SIOS contributes to climate services by providing research infrastructure with an overarching goal to develop and maintain a regional observational system for long-term measurements in and around Svalbard. The SIOS Core Data (SCD) consists of a list of essential Earth System Science variables relevant to determine environmental change in the Arctic. SCD is developed to improve the relevance and availability of scientific information addressing ESS topics for decision-making. SIOS Core Data providers have committed to maintain the observations for at least five years, to make the data publicly available, and to follow advanced principles of scientific data management and stewardship. Arctic climate change is posing risks to the safety, health and well-being of Arctic communities and ecosystems. Still, there remain gaps in our understanding of physical processes and societal implications. The authors of the SESS chapters have highlighted some unanswered questions and suggested concrete actions that should be taken to address them. The editors would like to thank the authors for their valuable contributions to the SESS Report 2021. These chapters illustrate how SIOS projects contribute to ensure the future vitality and resilience of Arctic peoples, communities and ecosystems

A study to determine the optimum design of a photographic film for the lunar surface hand-held camera Final report

Design, and processing of photographic film for lunar surface hand operated camer