CoMER: Modeling Coverage for Transformer-based Handwritten Mathematical Expression Recognition

The Transformer-based encoder-decoder architecture has recently made significant advances in recognizing handwritten mathematical expressions. However, the transformer model still suffers from the lack of coverage problem, making its expression recognition rate (ExpRate) inferior to its RNN counterpart. Coverage information, which records the alignment information of the past steps, has proven effective in the RNN models. In this paper, we propose CoMER, a model that adopts the coverage information in the transformer decoder. Specifically, we propose a novel Attention Refinement Module (ARM) to refine the attention weights with past alignment information without hurting its parallelism. Furthermore, we take coverage information to the extreme by proposing self-coverage and cross-coverage, which utilize the past alignment information from the current and previous layers. Experiments show that CoMER improves the ExpRate by 0.61%/2.09%/1.59% compared to the current state-of-the-art model, and reaches 59.33%/59.81%/62.97% on the CROHME 2014/2016/2019 test sets.Comment: Accept by ECCV 202

Recognition-Guided Diffusion Model for Scene Text Image Super-Resolution

Scene Text Image Super-Resolution (STISR) aims to enhance the resolution and legibility of text within low-resolution (LR) images, consequently elevating recognition accuracy in Scene Text Recognition (STR). Previous methods predominantly employ discriminative Convolutional Neural Networks (CNNs) augmented with diverse forms of text guidance to address this issue. Nevertheless, they remain deficient when confronted with severely blurred images, due to their insufficient generation capability when little structural or semantic information can be extracted from original images. Therefore, we introduce RGDiffSR, a Recognition-Guided Diffusion model for scene text image Super-Resolution, which exhibits great generative diversity and fidelity even in challenging scenarios. Moreover, we propose a Recognition-Guided Denoising Network, to guide the diffusion model generating LR-consistent results through succinct semantic guidance. Experiments on the TextZoom dataset demonstrate the superiority of RGDiffSR over prior state-of-the-art methods in both text recognition accuracy and image fidelity

Iterative projection meets sparsity regularization: towards practical single-shot quantitative phase imaging with in-line holography

Holography provides access to the optical phase. The emerging compressive phase retrieval approach can achieve in-line holographic imaging beyond the information-theoretic limit or even from a single shot by exploring the signal priors. However, iterative projection methods based on physical knowledge of the wavefield suffer from poor imaging quality, whereas the regularization techniques sacrifice robustness for fidelity. In this work, we present a unified compressive phase retrieval framework for in-line holography that encapsulates the unique advantages of both physical constraints and sparsity priors. In particular, a constrained complex total variation (CCTV) regularizer is introduced that explores the well-known absorption and support constraints together with sparsity in the gradient domain, enabling practical high-quality in-line holographic imaging from a single intensity image. We developed efficient solvers based on the proximal gradient method for the non-smooth regularized inverse problem and the corresponding denoising subproblem. Theoretical analyses further guarantee the convergence of the algorithms with prespecified parameters, obviating the need for manual parameter tuning. As both simulated and optical experiments demonstrate, the proposed CCTV model can characterize complex natural scenes while utilizing physically tractable constraints for quality enhancement. This new compressive phase retrieval approach can be extended, with minor adjustments, to various imaging configurations, sparsifying operators, and physical knowledge. It may cast new light on both theoretical and empirical studies