Self-supervised memory learning for scene text image super-resolution

Computerised recognition of low-resolution scene text images has been a persistent challenge. To improve the recognition performance, image quality enhancement via image super-resolution technology provides an intuitive solution. Typical deep learning-based scene text image super-resolution methods assume that the image quality degradation from high-resolution images to their corresponding low-resolution counterparts can be represented by mapping well-distributed samples, which limits their reconstruction performance in a practical text recognition system. For real-world scenarios this assumption typically does not hold since image degradations arise from multiple sources during image capture and processing. In this paper, to alleviate this problem, we propose a novel self-supervised end-to-end memory network model for scene text image super-resolution. In particular, after extracting enriched and finer representations from low-resolution text images via a spatial refinement block, we introduce a memory-based network to yield an improved super-resolution model that can handle complex degradation sources. Furthermore, to boost the effectiveness of our method, we design a multi-term loss to exploit textual structure information, where, in addition to the traditional reconstruction loss, we embed a character perceptual loss and a boundary enhancement loss. Extensive experiments on different datasets demonstrate that our proposed MNTSR method effectively improves the recognition accuracy for several scene text image recognition models and achieves state-of-the-art results. The source code is made available at https://github.com/xyzhu1/MNTSR</p

Recoverable facial identity protection via adaptive makeup transfer adversarial attacks

Unauthorised face recognition (FR) systems have posed significant threats to digital identity and privacy protection. To alleviate the risk of compromised identities, recent makeup transfer-based attack methods embed adversarial signals in order to confuse unauthorised FR systems. However, their major weakness is that they set up a fixed image unrelated to both the protected and the makeup reference images as the confusion identity, which in turn has a negative impact on both attack success rate and visual quality of transferred photos. In addition, the generated images cannot be recognised by authorised FR systems once attacks are triggered. To ad- dress these challenges, in this paper, we propose a Recoverable Makeup Transferred Generative Adversarial Network (RMT-GAN) which has the distinctive feature of improving its image-transfer quality by selecting a suitable transfer reference photo as the target identity. Moreover, our method offers a solution to recover the protected photos to their original counterparts that can be recognised by authorised systems. Experimental results demonstrate that our method provides significantly improved attack success rates while maintaining higher visual quality compared to state-of-the-art makeup transfer-based adversarial attack methods.</p