Super-Resolution of License Plate Images Using Attention Modules and Sub-Pixel Convolution Layers

Recent years have seen significant developments in the field of License Plate Recognition (LPR) through the integration of deep learning techniques and the increasing availability of training data. Nevertheless, reconstructing license plates (LPs) from low-resolution (LR) surveillance footage remains challenging. To address this issue, we introduce a Single-Image Super-Resolution (SISR) approach that integrates attention and transformer modules to enhance the detection of structural and textural features in LR images. Our approach incorporates sub-pixel convolution layers (also known as PixelShuffle) and a loss function that uses an Optical Character Recognition (OCR) model for feature extraction. We trained the proposed architecture on synthetic images created by applying heavy Gaussian noise to high-resolution LP images from two public datasets, followed by bicubic downsampling. As a result, the generated images have a Structural Similarity Index Measure (SSIM) of less than 0.10. Our results show that our approach for reconstructing these low-resolution synthesized images outperforms existing ones in both quantitative and qualitative measures. Our code is publicly available at https://github.com/valfride/lpr-rsr-ext

Super-resolution towards license plate recognition

Orientador: David MenottiDissertação (mestrado) - Universidade Federal do Paraná, Setor de Ciências Exatas, Programa de Pós-Graduação em Informática. Defesa : Curitiba, 24/04/2023Inclui referências: p. 51-59Área de concentração: Ciência da ComputaçãoResumo: Nos últimos anos, houve avanços significativos no campo de Reconhecimento de placas de veiculares (LPR, do inglês License Plate Recognition) por meio da integração de técnicas de aprendizado profundo e do aumento da disponibilidade de dados para treinamento. No entanto, reconstruir placas veiculares a partir de imagens de sistemas de vigilância em baixa resolução ainda é um desafio. Para enfrentar essa dificuldade, apresentamos uma abordagem de Super Resolução de Imagem Única (SISR, do inglês Single-Image Super-Resolution) que integra módulos de atenção para aprimorar a detecção de característica estruturais e texturais em imagens de baixa resolução. Nossa abordagem utiliza camadas de convolução sub-pixel (também conhecidas como PixelShuffle) e uma função de perda que emprega um modelo de Reconhecimento Óptico de Caracteres (OCR, do inglês Optical Character Recognition) para extração de características. Treinamos a arquitetura proposta com imagens sintéticas criadas aplicando ruído gaussiano pesado à imagens de alta resolução de placas veiculares de dois conjuntos de dados públicos, seguido de redução de sua resolução com interpolação bicúbica. Como resultado, as imagens geradas têm um Índice de Similaridade Estrutural (SSIM, do inglês Structural Similarity Index Measure) inferior a 0,10. Nossos resultados experimentais mostram que a abordagem proposta para reconstruir essas imagens sintéticas de baixa resolução superou as existentes tanto em medidas quantitativas quanto qualitativas.Abstract: Recent years have seen significant developments in the field of License Plate Recognition (LPR) through the integration of deep learning techniques and the increasing availability of training data. Nevertheless, reconstructing license plates (LPs) from low-resolution (LR) surveillance footage remains challenging. To address this issue, we introduce a Single-Image Super-Resolution (SISR) approach that integrates attention and transformer modules to enhance the detection of structural and textural features in LR images. Our approach incorporates sub-pixel convolution layers (also known as PixelShuffle) and a loss function that uses an Optical Character Recognition (OCR) model for feature extraction. We trained the proposed architecture on synthetic images created by applying heavy Gaussian noise to high-resolution LP images from two public datasets, followed by bicubic downsampling. As a result, the generated images have a Structural Similarity Index Measure (SSIM) of less than 0.10. Our results show that our approach for reconstructing these low-resolution synthesized images outperforms existing ones in both quantitative and qualitative measures

Ghost imaging lidar via sparsity constraints

For remote sensing, high-resolution imaging techniques are helpful to catch more characteristic information of the target. We extend pseudo-thermal light ghost imaging to the area of remote imaging and propose a ghost imaging lidar system. For the first time, we demonstrate experimentally that the real-space image of a target at about 1.0 km range with 20 mm resolution is achieved by ghost imaging via sparsity constraints (GISC) technique. The characters of GISC technique compared to the existing lidar systems are also discussed.Comment: 4pages, 3figure

Improved split fluorescent proteins for endogenous protein labeling.

Self-complementing split fluorescent proteins (FPs) have been widely used for protein labeling, visualization of subcellular protein localization, and detection of cell-cell contact. To expand this toolset, we have developed a screening strategy for the direct engineering of self-complementing split FPs. Via this strategy, we have generated a yellow-green split-mNeonGreen21-10/11 that improves the ratio of complemented signal to the background of FP1-10-expressing cells compared to the commonly used split GFP1-10/11; as well as a 10-fold brighter red-colored split-sfCherry21-10/11. Based on split sfCherry2, we have engineered a photoactivatable variant that enables single-molecule localization-based super-resolution microscopy. We have demonstrated dual-color endogenous protein tagging with sfCherry211 and GFP11, revealing that endoplasmic reticulum translocon complex Sec61B has reduced abundance in certain peripheral tubules. These new split FPs not only offer multiple colors for imaging interaction networks of endogenous proteins, but also hold the potential to provide orthogonal handles for biochemical isolation of native protein complexes.Split fluorescent proteins (FPs) have been widely used to visualise proteins in cells. Here the authors develop a screen for engineering new split FPs, and report a yellow-green split-mNeonGreen2 with reduced background, a red split-sfCherry2 for multicolour labeling, and its photoactivatable variant for super-resolution use

Flat-top TIRF illumination boosts DNA-PAINT imaging and quantification

Super-resolution (SR) techniques have extended the optical resolution down to a few nanometers. However, quantitative treatment of SR data remains challenging due to its complex dependence on a manifold of experimental parameters. Among the different SR variants, DNA-PAINT is relatively straightforward to implement, since it achieves the necessary 'blinking' without the use of rather complex optical or chemical activation schemes. However, it still suffers from image and quantification artifacts caused by inhomogeneous optical excitation. Here we demonstrate that several experimental challenges can be alleviated by introducing a segment-wise analysis approach and ultimately overcome by implementing a flat-top illumination profile for TIRF microscopy using a commercially-available beam-shaping device. The improvements with regards to homogeneous spatial resolution and precise kinetic information over the whole field-of-view were quantitatively assayed using DNA origami and cell samples. Our findings open the door to high-throughput DNA-PAINT studies with thus far unprecedented accuracy for quantitative data interpretation