Demultiplexing Colored Images for Multispectral Photometric Stereo via Deep Neural Networks

Recovering fine-scale surface shapes is a challenging task in computer vision. Multispectral photometric stereo is one of the popular methods as it can handle non-rigid/moving objects and produces per-pixel dense results. However, the colored images captured by practical multispectral photometric stereo setups are aliased in RGB channels. Existing solutions require prior information to calibrate few points and estimates whole surface normal by the calibration, while prior information is not always available and accurate. Differing from previous solutions which require calibration or other prior information, we first formulate the problem in a learning framework, which directly seeks the per-pixel mapping of the aliased and spectrum-multiplexed pixel response to the anti-aliased and demultiplexed counterpart. In this paper, we propose to use a novel deep neural networks framework as the “demultiplexer”. By using “demultiplexer” and classic photometric stereo, our method can reconstruct a dense and accurate surface normal from a single-frame colored image without any prior information nor extra information injected. We build an imaging device to collect images of different materials under colored lights and white lights. We conducted extensive experiments on our dataset and a public dataset. The results show that the proposed fully connected network successfully demultiplexes the colorful image and produces satisfactory surface estimation

Spearman Correlation coefficients of the association between HFRS cases and meteorological variables.

<p>Spearman Correlation coefficients of the association between HFRS cases and meteorological variables.</p

Geographical location of study area.

<p>Geographical location of study area.</p

The number of HFRS cases in Jiaonan from 1992 to 2014.

<p>The number of HFRS cases in Jiaonan from 1992 to 2014.</p

Comparisons of univariate SARIMA models.

<p>Comparisons of univariate SARIMA models.</p

Univariate SARIMA analyses of HFRS cases.

<p>(a) Autocorrelation (ACF) plot of HFRS cases; (b) Partial ACF plot of HFRS cases; (c) SARIMA model of HFRS cases; (d) ACF and PACF plots of residuals after applying a SARIMA model.</p

Multivariate SARIMA model integrating meteorological variables.

<p>Multivariate SARIMA model integrating meteorological variables.</p

HFRS cases classified by season.

<p>HFRS cases classified by season.</p

Cross-correlation analysis between HFRS cases and meteorological variables.

<p>Cross-correlation analysis between HFRS cases and meteorological variables.</p

Additional file 2: of How does the dengue vector mosquito Aedes albopictus respond to global warming?

Regression analysis of the warming effects. (DOCX 14 kb