Extreme Learning Machine Based Non-Iterative and Iterative Nonlinearity Mitigation for LED Communications

This work concerns receiver design for light emitting diode (LED) communications where the LED nonlinearity can severely degrade the performance of communications. We propose extreme learning machine (ELM) based non-iterative receivers and iterative receivers to effectively handle the LED nonlinearity and memory effects. For the iterative receiver design, we also develop a data-aided receiver, where data is used as virtual training sequence in ELM training. It is shown that the ELM based receivers significantly outperform conventional polynomial based receivers; iterative receivers can achieve huge performance gain compared to non-iterative receivers; and the data-aided receiver can reduce training overhead considerably. This work can also be extended to radio frequency communications, e.g., to deal with the nonlinearity of power amplifiers

Compositional Exemplars for In-context Learning

Large pretrained language models (LMs) have shown impressive In-Context Learning (ICL) ability, where the model learns to do an unseen task via a prompt consisting of input-output examples as the demonstration, without any parameter updates. The performance of ICL is highly dominated by the quality of the selected in-context examples. However, previous selection methods are mostly based on simple heuristics, leading to sub-optimal performance. In this work, we formulate in-context example selection as a subset selection problem. We propose CEIL (Compositional Exemplars for In-context Learning), which is instantiated by Determinantal Point Processes (DPPs) to model the interaction between the given input and in-context examples, and optimized through a carefully-designed contrastive learning objective to obtain preference from LMs. We validate CEIL on 12 classification and generation datasets from 7 distinct NLP tasks, including sentiment analysis, paraphrase detection, natural language inference, commonsense reasoning, open-domain question answering, code generation, and semantic parsing. Extensive experiments demonstrate not only the state-of-the-art performance but also the transferability and compositionality of CEIL, shedding new light on effective and efficient in-context learning. Our code is released at https://github.com/HKUNLP/icl-ceil.Comment: Accepted in ICML 202

DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models

Recently, diffusion models have emerged as a new paradigm for generative models. Despite the success in domains using continuous signals such as vision and audio, adapting diffusion models to natural language is under-explored due to the discrete nature of texts, especially for conditional generation. We tackle this challenge by proposing DiffuSeq: a diffusion model designed for sequence-to-sequence (Seq2Seq) text generation tasks. Upon extensive evaluation over a wide range of Seq2Seq tasks, we find DiffuSeq achieving comparable or even better performance than six established baselines, including a state-of-the-art model that is based on pre-trained language models. Apart from quality, an intriguing property of DiffuSeq is its high diversity during generation, which is desired in many Seq2Seq tasks. We further include a theoretical analysis revealing the connection between DiffuSeq and autoregressive/non-autoregressive models. Bringing together theoretical analysis and empirical evidence, we demonstrate the great potential of diffusion models in complex conditional language generation tasks. Code is available at \url{https://github.com/Shark-NLP/DiffuSeq}Comment: ICLR 2023 camera read

Text detection and recognition based on a lensless imaging system

Lensless cameras are characterized by several advantages (e.g., miniaturization, ease of manufacture, and low cost) as compared with conventional cameras. However, they have not been extensively employed due to their poor image clarity and low image resolution, especially for tasks that have high requirements on image quality and details such as text detection and text recognition. To address the problem, a framework of deep-learning-based pipeline structure was built to recognize text with three steps from raw data captured by employing lensless cameras. This pipeline structure consisted of the lensless imaging model U-Net, the text detection model connectionist text proposal network (CTPN), and the text recognition model convolutional recurrent neural network (CRNN). Compared with the method focusing only on image reconstruction, UNet in the pipeline was able to supplement the imaging details by enhancing factors related to character categories in the reconstruction process, so the textual information can be more effectively detected and recognized by CTPN and CRNN with fewer artifacts and high-clarity reconstructed lensless images. By performing experiments on datasets of different complexities, the applicability to text detection and recognition on lensless cameras was verified. This study reasonably demonstrates text detection and recognition tasks in the lensless camera system,and develops a basic method for novel applications

Exploring Reionization-Era Quasars IV: Discovery of Six New z≳6.5z \gtrsim 6.5 Quasars with DES, VHS and unWISE Photometry

This is the fourth paper in a series of publications aiming at discovering quasars at the epoch of reionization. In this paper, we expand our search for $z\sim 7$ quasars to the footprint of the Dark Energy Survey (DES) Data Release One (DR1), covering $\sim 5000$ deg$^2$ of new area. We select $z\sim 7$ quasar candidates using deep optical, near-infrared (near-IR) and mid-IR photometric data from the DES DR1, the VISTA Hemisphere Survey (VHS), the VISTA Kilo-degree Infrared Galaxy (VIKING) survey, the UKIRT InfraRed Deep Sky Surveys -- Large Area Survey (ULAS) and the unblurred coadds from the Wide-field Infrared Survey Explore ($WISE$) images (unWISE). The inclusion of DES and unWISE photometry allows the search to reach $\sim$ 1 magnitude fainter, comparing to our $z \gtrsim 6.5$ quasar survey in the northern sky (Wang et al. 2018). We report the initial discovery and spectroscopic confirmation of six new luminous quasars at $z>6.4$, including an object at $z=7.02$, the fourth quasar yet known at $z>7$, from a small fraction of candidates observed thus far. Based on the recent measurement of $z \sim 6.7$ quasar luminosity function using the quasar sample from our survey in the northern sky, we estimate that there will be $\gtrsim$ 55 quasars at $z > 6.5$ at $M_{1450} < -24.5$ in the full DES footprint.Comment: 8 pages, 3 figures, submitted to A