6 research outputs found
CompenNet++: End-to-end Full Projector Compensation
Full projector compensation aims to modify a projector input image such that
it can compensate for both geometric and photometric disturbance of the
projection surface. Traditional methods usually solve the two parts separately,
although they are known to correlate with each other. In this paper, we propose
the first end-to-end solution, named CompenNet++, to solve the two problems
jointly. Our work non-trivially extends CompenNet, which was recently proposed
for photometric compensation with promising performance. First, we propose a
novel geometric correction subnet, which is designed with a cascaded
coarse-to-fine structure to learn the sampling grid directly from photometric
sampling images. Second, by concatenating the geometric correction subset with
CompenNet, CompenNet++ accomplishes full projector compensation and is
end-to-end trainable. Third, after training, we significantly simplify both
geometric and photometric compensation parts, and hence largely improves the
running time efficiency. Moreover, we construct the first setup-independent
full compensation benchmark to facilitate the study on this topic. In our
thorough experiments, our method shows clear advantages over previous arts with
promising compensation quality and meanwhile being practically convenient.Comment: To appear in ICCV 2019. High-res supplementary material:
https://www3.cs.stonybrook.edu/~hling/publication/CompenNet++_sup-high-res.pdf.
Code: https://github.com/BingyaoHuang/CompenNet-plusplu
Efficient Distortion-Free Neural Projector Deblurring in Dynamic Projection Mapping
Kageyama Y., Iwai D., Sato K.. Efficient Distortion-Free Neural Projector Deblurring in Dynamic Projection Mapping. IEEE Transactions on Visualization and Computer Graphics , (2024); https://doi.org/10.1109/TVCG.2024.3354957.Dynamic Projection Mapping (DPM) necessitates geometric compensation of the projection image based on the position and orientation of moving objects. Additionally, the projector's shallow depth of field results in pronounced defocus blur even with minimal object movement. Achieving delay-free DPM with high image quality requires real-time implementation of geometric compensation and projector deblurring. To meet this demand, we propose a framework comprising two neural components: one for geometric compensation and another for projector deblurring. The former component warps the image by detecting the optical flow of each pixel in both the projection and captured images. The latter component performs real-time sharpening as needed. Ideally, our network's parameters should be trained on data acquired in an actual environment. However, training the network from scratch while executing DPM, which demands real-time image generation, is impractical. Therefore, the network must undergo pre-training. Unfortunately, there are no publicly available large real datasets for DPM due to the diverse image quality degradation patterns. To address this challenge, we propose a realistic synthetic data generation method that numerically models geometric distortion and defocus blur in real-world DPM. Through exhaustive experiments, we have confirmed that the model trained on the proposed dataset achieves projector deblurring in the presence of geometric distortions with a quality comparable to state-of-the-art methods
CompenHR: Efficient Full Compensation for High-resolution Projector
Full projector compensation is a practical task of projector-camera systems.
It aims to find a projector input image, named compensation image, such that
when projected it cancels the geometric and photometric distortions due to the
physical environment and hardware. State-of-the-art methods use deep learning
to address this problem and show promising performance for low-resolution
setups. However, directly applying deep learning to high-resolution setups is
impractical due to the long training time and high memory cost. To address this
issue, this paper proposes a practical full compensation solution. Firstly, we
design an attention-based grid refinement network to improve geometric
correction quality. Secondly, we integrate a novel sampling scheme into an
end-to-end compensation network to alleviate computation and introduce
attention blocks to preserve key features. Finally, we construct a benchmark
dataset for high-resolution projector full compensation. In experiments, our
method demonstrates clear advantages in both efficiency and quality