High Dynamic Range Imaging with Context-aware Transformer

Avoiding the introduction of ghosts when synthesising LDR images as high dynamic range (HDR) images is a challenging task. Convolutional neural networks (CNNs) are effective for HDR ghost removal in general, but are challenging to deal with the LDR images if there are large movements or oversaturation/undersaturation. Existing dual-branch methods combining CNN and Transformer omit part of the information from non-reference images, while the features extracted by the CNN-based branch are bound to the kernel size with small receptive field, which are detrimental to the deblurring and the recovery of oversaturated/undersaturated regions. In this paper, we propose a novel hierarchical dual Transformer method for ghost-free HDR (HDT-HDR) images generation, which extracts global features and local features simultaneously. First, we use a CNN-based head with spatial attention mechanisms to extract features from all the LDR images. Second, the LDR features are delivered to the Hierarchical Dual Transformer (HDT). In each Dual Transformer (DT), the global features are extracted by the window-based Transformer, while the local details are extracted using the channel attention mechanism with deformable CNNs. Finally, the ghost free HDR image is obtained by dimensional mapping on the HDT output. Abundant experiments demonstrate that our HDT-HDR achieves the state-of-the-art performance among existing HDR ghost removal methods.Comment: 8 pages, 5 figure

Modeling and removal of optical ghosts in the PROBA-3/ASPIICS externally occulted solar coronagraph

Context: ASPIICS is a novel externally occulted solar coronagraph, which will be launched onboard the PROBA-3 mission of the European Space Agency. The external occulter will be placed on the first satellite approximately 150 m ahead of the second satellite that will carry an optical instrument. During 6 hours per orbit, the satellites will fly in a precise formation, constituting a giant externally occulted coronagraph. Large distance between the external occulter and the primary objective will allow observations of the white-light solar corona starting from extremely low heights 1.1RSun. Aims: To analyze influence of optical ghost images formed inside the telescope and develop an algorithm for their removal. Methods: We implement the optical layout of ASPIICS in Zemax and study the ghost behaviour in sequential and non-sequential regimes. We identify sources of the ghost contributions and analyze their geometrical behaviour. Finally we develop a mathematical model and software to calculate ghost images for any given input image. Results: We show that ghost light can be important in the outer part of the field of view, where the coronal signal is weak, since the energy of bright inner corona is redistributed to the outer corona. However the model allows to remove the ghost contribution. Due to a large distance between the external occulter and the primary objective, the primary objective does not produce a significant ghost. The use of the Lyot spot in ASPIICS is not necessary.Comment: 14 pages, 13 figure

Temporal phase unwrapping using deep learning

The multi-frequency temporal phase unwrapping (MF-TPU) method, as a classical phase unwrapping algorithm for fringe projection profilometry (FPP), is capable of eliminating the phase ambiguities even in the presence of surface discontinuities or spatially isolated objects. For the simplest and most efficient case, two sets of 3-step phase-shifting fringe patterns are used: the high-frequency one is for 3D measurement and the unit-frequency one is for unwrapping the phase obtained from the high-frequency pattern set. The final measurement precision or sensitivity is determined by the number of fringes used within the high-frequency pattern, under the precondition that the phase can be successfully unwrapped without triggering the fringe order error. Consequently, in order to guarantee a reasonable unwrapping success rate, the fringe number (or period number) of the high-frequency fringe patterns is generally restricted to about 16, resulting in limited measurement accuracy. On the other hand, using additional intermediate sets of fringe patterns can unwrap the phase with higher frequency, but at the expense of a prolonged pattern sequence. Inspired by recent successes of deep learning techniques for computer vision and computational imaging, in this work, we report that the deep neural networks can learn to perform TPU after appropriate training, as called deep-learning based temporal phase unwrapping (DL-TPU), which can substantially improve the unwrapping reliability compared with MF-TPU even in the presence of different types of error sources, e.g., intensity noise, low fringe modulation, and projector nonlinearity. We further experimentally demonstrate for the first time, to our knowledge, that the high-frequency phase obtained from 64-period 3-step phase-shifting fringe patterns can be directly and reliably unwrapped from one unit-frequency phase using DL-TPU