Learning Wavefront Coding for Extended Depth of Field Imaging

Depth of field is an important factor of imaging systems that highly affects the quality of the acquired spatial information. Extended depth of field (EDoF) imaging is a challenging ill-posed problem and has been extensively addressed in the literature. We propose a computational imaging approach for EDoF, where we employ wavefront coding via a diffractive optical element (DOE) and we achieve deblurring through a convolutional neural network. Thanks to the end-to-end differentiable modeling of optical image formation and computational post-processing, we jointly optimize the optical design, i.e., DOE, and the deblurring through standard gradient descent methods. Based on the properties of the underlying refractive lens and the desired EDoF range, we provide an analytical expression for the search space of the DOE, which is instrumental in the convergence of the end-to-end network. We achieve superior EDoF imaging performance compared to the state of the art, where we demonstrate results with minimal artifacts in various scenarios, including deep 3D scenes and broadband imaging

Optical modelling of accommodative light field display system and prediction of human eye responses

The spatio-angular resolution of a light field (LF) display is a crucial factor for delivering adequate spatial image quality and eliciting an accommodation response. Previous studies have modelled retinal image formation with an LF display and evaluated whether accommodation would be evoked correctly. The models were mostly based on ray-tracing and a schematic eye model, which pose computational complexity and inaccurately represent the human eye population's behaviour. We propose an efficient wave-optics-based framework to model the human eye and a general LF display. With the model, we simulated the retinal point spread function (PSF) of a point rendered by an LF display at various depths to characterise the retinal image quality. Additionally, accommodation responses to rendered LF images were estimated by computing the visual Strehl ratio based on the optical transfer function (VSOTF) from the PSFs. We assumed an ideal LF display that had an infinite spatial resolution and was free from optical aberrations in the simulation. We tested images rendered at 0--4 dioptres of depths having angular resolutions of up to 4x4 viewpoints within a pupil. The simulation predicted small and constant accommodation errors, which contradict the findings of previous studies. An evaluation of the optical resolution of the rendered retinal image suggested a trade-off between the maximum resolution achievable and the depth range of a rendered image where in-focus resolution is kept high. The proposed framework can be used to evaluate the upper bound of the optical performance of an LF display for realistically aberrated eyes, which may help to find an optimal spatio-angular resolution required to render a high quality 3D scene.Comment: 24 pages, 12 figures, submitted to Optics Expres

Optical modelling of an accommodative light field display system and prediction of human eye responses

publishedVersionPeer reviewe

Observed optical resolution of light field display: Empirical and simulation results

Light field (LF) displays are a promising 3D display technology to mitigate the vergence-accommodation conflict. Recently, we have proposed a simulation framework to model an LF display system. It has predicted that the in-focus optical resolution on the retina would drop as the relative depth of a rendered image to the display-specific optical reference depth grows. In this study, we examine the empirical optical resolution of a near-eye LF display prototype by capturing rendered test images and compare it to simulation results based on the previously developed computational model. We use an LF display prototype that employs a time-multiplexing technique and achieves a high angular resolution of 6-by-6 viewpoints in the eyebox. The test image is rendered at various depths ranging 0–3 diopters, and the optical resolution of the best-focus images is analyzed from images captured by a camera. Additionally, we compare the measurement results to the simulation results, discussing theoretical and practical limitations of LF displays.Peer reviewe

Computational Hyperspectral Imaging with Diffractive Optics and Deep Residual Network

Hyperspectral imaging critically serves for various fields such as remote sensing, biomedical and agriculture. Its potential can be exploited to a greater extent when combined with deep learning methods, which improve the reconstructed hyperspectral image quality and reduce the processing time. In this paper, we propose a novel snapshot hyperspectral imaging system using optimized diffractive optical element and color filter along with the residual dense network. We evaluate our method through simulations considering the effects of each optical element and noise. Simulation results demonstrate high-quality hyperspectral image reconstruction capabilities through the proposed computational hyperspectral camera.acceptedVersionPeer reviewe

A Framework for Assessing Rendering Techniques for Near-Eye Integral Imaging Displays

We address the problem of 3D scene rendering on near-eye integral imaging displays and evaluation of different rendering methods in terms of human perception. We compare three rendering techniques in terms of perceived spatial resolution at different focused depths, simulating the display in virtual environment and representing the eye through a thin-lens camera model.acceptedVersionPeer reviewe

Generalized Tensor Summation Compressive Sensing Network (GTSNET) : An Easy to Learn Compressive Sensing Operation

The efforts in compressive sensing (CS) literature can be divided into two groups: finding a measurement matrix that preserves the compressed information at its maximum level, and finding a robust reconstruction algorithm. In the traditional CS setup, the measurement matrices are selected as random matrices, and optimization-based iterative solutions are used to recover the signals. Using random matrices when handling large or multi-dimensional signals is cumbersome especially when it comes to iterative optimizations. Recent deep learning-based solutions increase reconstruction accuracy while speeding up recovery, but jointly learning the whole measurement matrix remains challenging. For this reason, state-of-the-art deep learning CS solutions such as convolutional compressive sensing network (CSNET) use block-wise CS schemes to facilitate learning. In this work, we introduce a separable multi-linear learning of the CS matrix by representing the measurement signal as the summation of the arbitrary number of tensors. As compared to block-wise CS, tensorial learning eases blocking artifacts and improves performance, especially at low measurement rates (MRs), such as {MRs} < 0.1. The software implementation of the proposed network is publicly shared at https://github.com/mehmetyamac/GTSNET.Peer reviewe

Drug Delivery Approaches for the Treatment of Cervical Cancer

Cervical cancer is a highly prevalent cancer that affects women around the world. With the availability of new technologies, researchers have increased their efforts to develop new drug delivery systems in cervical cancer chemotherapy. In this review, we summarized some of the recent research in systematic and localized drug delivery systems and compared the advantages and disadvantages of these methods

Characterizing ABC-Transporter Substrate-Likeness Using a Clean-Slate Genetic Background

Mutations in ATP Binding Cassette (ABC)-transporter genes can have major effects on the bioavailability and toxicity of the drugs that are ABC-transporter substrates. Consequently, methods to predict if a drug is an ABC-transporter substrate are useful for drug development. Such methods traditionally relied on literature curated collections of ABC-transporter dependent membrane transfer assays. Here, we used a single large-scale dataset of 376 drugs with relative efficacy on an engineered yeast strain with all ABC-transporter genes deleted (ABC-16), to explore the relationship between a drug’s chemical structure and ABC-transporter substrate-likeness. We represented a drug’s chemical structure by an array of substructure keys and explored several machine learning methods to predict the drug’s efficacy in an ABC-16 yeast strain. Gradient-Boosted Random Forest models outperformed all other methods with an AUC of 0.723. We prospectively validated the model using new experimental data and found significant agreement with predictions. Our analysis expands the previously reported chemical substructures associated with ABC-transporter substrates and provides an alternative means to investigate ABC-transporter substrate-likeness