Neural Spectro-polarimetric Fields

Modeling the spatial radiance distribution of light rays in a scene has been extensively explored for applications, including view synthesis. Spectrum and polarization, the wave properties of light, are often neglected due to their integration into three RGB spectral bands and their non-perceptibility to human vision. Despite this, these properties encompass substantial material and geometric information about a scene. In this work, we propose to model spectro-polarimetric fields, the spatial Stokes-vector distribution of any light ray at an arbitrary wavelength. We present Neural Spectro-polarimetric Fields (NeSpoF), a neural representation that models the physically-valid Stokes vector at given continuous variables of position, direction, and wavelength. NeSpoF manages inherently noisy raw measurements, showcases memory efficiency, and preserves physically vital signals, factors that are crucial for representing the high-dimensional signal of a spectro-polarimetric field. To validate NeSpoF, we introduce the first multi-view hyperspectral-polarimetric image dataset, comprised of both synthetic and real-world scenes. These were captured using our compact hyperspectral-polarimetric imaging system, which has been calibrated for robustness against system imperfections. We demonstrate the capabilities of NeSpoF on diverse scenes

UGPNet: Universal Generative Prior for Image Restoration

Recent image restoration methods can be broadly categorized into two classes: (1) regression methods that recover the rough structure of the original image without synthesizing high-frequency details and (2) generative methods that synthesize perceptually-realistic high-frequency details even though the resulting image deviates from the original structure of the input. While both directions have been extensively studied in isolation, merging their benefits with a single framework has been rarely studied. In this paper, we propose UGPNet, a universal image restoration framework that can effectively achieve the benefits of both approaches by simply adopting a pair of an existing regression model and a generative model. UGPNet first restores the image structure of a degraded input using a regression model and synthesizes a perceptually-realistic image with a generative model on top of the regressed output. UGPNet then combines the regressed output and the synthesized output, resulting in a final result that faithfully reconstructs the structure of the original image in addition to perceptually-realistic textures. Our extensive experiments on deblurring, denoising, and super-resolution demonstrate that UGPNet can successfully exploit both regression and generative methods for high-fidelity image restoration.Comment: Accepted to WACV 202

Neural 360∘^\circ Structured Light with Learned Metasurfaces

Structured light has proven instrumental in 3D imaging, LiDAR, and holographic light projection. Metasurfaces, comprised of sub-wavelength-sized nanostructures, facilitate 180$^\circ$ field-of-view (FoV) structured light, circumventing the restricted FoV inherent in traditional optics like diffractive optical elements. However, extant metasurface-facilitated structured light exhibits sub-optimal performance in downstream tasks, due to heuristic pattern designs such as periodic dots that do not consider the objectives of the end application. In this paper, we present neural 360$^\circ$ structured light, driven by learned metasurfaces. We propose a differentiable framework, that encompasses a computationally-efficient 180$^\circ$ wave propagation model and a task-specific reconstructor, and exploits both transmission and reflection channels of the metasurface. Leveraging a first-order optimizer within our differentiable framework, we optimize the metasurface design, thereby realizing neural 360$^\circ$ structured light. We have utilized neural 360$^\circ$ structured light for holographic light projection and 3D imaging. Specifically, we demonstrate the first 360$^\circ$ light projection of complex patterns, enabled by our propagation model that can be computationally evaluated 50,000$\times$ faster than the Rayleigh-Sommerfeld propagation. For 3D imaging, we improve depth-estimation accuracy by 5.09$\times$ in RMSE compared to the heuristically-designed structured light. Neural 360$^\circ$ structured light promises robust 360$^\circ$ imaging and display for robotics, extended-reality systems, and human-computer interactions

Spectral and Polarization Vision: Spectro-polarimetric Real-world Dataset

Image datasets are essential not only in validating existing methods in computer vision but also in developing new methods. Most existing image datasets focus on trichromatic intensity images to mimic human vision. However, polarization and spectrum, the wave properties of light that animals in harsh environments and with limited brain capacity often rely on, remain underrepresented in existing datasets. Although spectro-polarimetric datasets exist, these datasets have insufficient object diversity, limited illumination conditions, linear-only polarization data, and inadequate image count. Here, we introduce two spectro-polarimetric datasets: trichromatic Stokes images and hyperspectral Stokes images. These novel datasets encompass both linear and circular polarization; they introduce multiple spectral channels; and they feature a broad selection of real-world scenes. With our dataset in hand, we analyze the spectro-polarimetric image statistics, develop efficient representations of such high-dimensional data, and evaluate spectral dependency of shape-from-polarization methods. As such, the proposed dataset promises a foundation for data-driven spectro-polarimetric imaging and vision research. Dataset and code will be publicly available

Self-care use patterns in the UK, US, Australia, and Japan: a multinational web-based survey

AbstractBackgroundThe trend toward patient- or consumer-centered healthcare has been accelerated by advances in technology, consumer empowerment, and a shift from infectious to chronic diseases. The purpose of this study was to examine the growing self-care market by analyzing self-care patterns.MethodsWe conducted a cross-sectional, web-based survey involving adults from nine major cities in the UK, the USA, Australia, and Japan. This study examined the extent and frequency of self-care, self-care expenditure, sources of self-care information, and reasons for self-care in each country.ResultsThe results showed that the prevalence of self-care was highest in Japan (54.9%), followed by the UK (43.1%), the USA (42.5%), and Australia (40.4%). The primary reason for practicing self-care was “to manage my healthcare myself” (cited by 45.7%, 59.5%, 49.2%, and 4.1% of participants in Australia, Japan, the UK, and the USA, respectively). Significant linear associations were observed between age and the prevalence of self-care in all countries (p<0.05), indicating that self-care prevalence decreased with age in the UK, the USA, and Australia, and increased with age in Japan. The frequency with which self-care was practiced was positively correlated with age in the USA (p<0.05), Australia (p<0.01), and Japan (p<0.05). In addition to acquaintances, internet search engines and information obtained from pharmacies were considered reliable and widely used sources of self-care information.ConclusionWhen developing self-care products or services, healthcare providers and policymakers should consider self-care patterns

Unsupervised Grouping of Local Components for Object Segmentation

In this paper, we propose a novel object segmentation method for image understanding. Due to challenges such as variations in object size, orientation, illumination etc. object segmentation is extraordinarily difficult task in the domain of image understanding. It is well-founded concept that a small portion of the pixel set in an image contributes most in image description. Based on this concept, we hypothesize that an image consists of many components or parts each of which represent a small local area in the image and they are very meaningful in visual perception. For object segmentation, we propose spatial segmentation method on such prototypical components of images. Given an image this segmentation method acts as coarse to fine search for object(s) iteratively. The proposed method demonstrate its excellence in localizing objects in various complex backgrounds, multiple objects in a single image even if they have variation in size, orientation, lighting conditions etc. The detection efficiency of our object detector on our self-collected image set which consists of images from six different object categories climbs up to 93% in average.