BatchPrompt: Accomplish more with less

As the ever-increasing token limits of large language models (LLMs) have enabled long context as input, prompting with single data samples might no longer an efficient way. A straightforward strategy improving efficiency is to batch data within the token limit (e.g., 8k for gpt-3.5-turbo; 32k for GPT-4), which we call BatchPrompt. We have two initial observations for prompting with batched data. First, we find that prompting with batched data in longer contexts will inevitably lead to worse performance, compared to single-data prompting. Second, the performance of the language model is significantly correlated with the positions and order of the batched data, due to the corresponding change in decoder context. To retain efficiency and overcome performance loss, we propose Batch Permutation and Ensembling (BPE), and a novel Self-reflection-guided EArly Stopping (SEAS) technique. Our comprehensive experimental evaluation demonstrates that BPE can boost the performance of BatchPrompt with a striking margin on a range of popular NLP tasks, including question answering (Boolq), textual entailment (RTE), and duplicate questions identification (QQP). These performances are even competitive with/higher than single-data prompting(SinglePrompt), while BatchPrompt requires much fewer LLM calls and input tokens (For SinglePrompt v.s. BatchPrompt with batch size 32, using just 9%-16% the number of LLM calls, Boolq accuracy 90.6% to 90.9% with 27.4% tokens, QQP accuracy 87.2% to 88.4% with 18.6% tokens, RTE accuracy 91.5% to 91.1% with 30.8% tokens). To the best of our knowledge, this is the first work to technically improve prompting efficiency of large language models. We hope our simple yet effective approach will shed light on the future research of large language models. The code will be released.Comment: 20 pages, 5 figure

Testing a word is not a test of word learning

Although vocabulary acquisition requires children learn names for multiple things, many investigations of word learning mechanisms teach children the name for only one of the objects presented. This is problematic because it is unclear whether children's performance reflects recall of the correct name-object association or simply selection of the only object that was singled out by being the only object named. Children introduced to one novel name may perform at ceiling as they are not required to discriminate on the basis of the name per se, and appear to rapidly learn words following minimal exposure to a single word. We introduced children to four novel objects. For half the children, only one of the objects was named and for the other children, all four objects were named. Only children introduced to one word reliably selected the target object at test. This demonstration highlights the over-simplicity of one-word learning paradigms and the need for a shift in word learning paradigms where more than one word is taught to ensure children disambiguate objects on the basis of their names rather than their degree of salience

Context and perceptual salience influence the formation of novel stereotypes via cumulative cultural evolution

We use a transmission chain method to establish how context and category salience influence the formation of novel stereotypes through cumulative cultural evolution. We created novel alien targets by combining features from three category dimensions—color, movement, and shape—thereby creating social targets that were individually unique but that also shared category membership with other aliens (e.g., two aliens might be the same color and shape but move differently). At the start of the transmission chains each alien was randomly assigned attributes that described it (e.g., arrogant, caring, confident). Participants were given training on the alien-attribute assignments and were then tested on their memory for these. The alien-attribute assignments participants produced during test were used as the training materials for the next participant in the transmission chain. As information was repeatedly transmitted an increasingly simplified, learnable stereotype-like structure emerged for targets who shared the same color, such that by the end of the chains targets who shared the same color were more likely to share the same attributes (a reanalysis of data from Martin et al., 2014 which we term Experiment 1). The apparent bias toward the formation of novel stereotypes around the color category dimension was also found for objects (Experiment 2). However, when the category dimension of color was made less salient, it no longer dominated the formation of novel stereotypes (Experiment 3). The current findings suggest that context and category salience influence category dimension salience, which in turn influences the cumulative cultural evolution of information.<br/

Spin-induced asymmetry reaction - The formation of asymmetric carbon by electropolymerization

We describe the spin polarization–induced chirogenic electropolymerization of achiral 2-vinylpyridine, which forms a layer of enantioenhanced isotactic polymer on the electrode. The product formed is enantioenriched in asymmetric carbon polymer. To confirm the chirality of the polymer film formed on the electrode, we also measured its electron spin polarization properties as a function of its thickness. Two methods were used: First, spin polarization was measured by applying magnetic contact atomic force microscopy, and second, magnetoresistance was assessed in a sandwich-like four-point contact structure. We observed high spin-selective electron transmission, even for a layer thickness of 120 nm. A correlation exists between the change in the circular dichroism signal and the change in the spin polarization, as a function of thickness. The spin-filtering efficiency increases with temperature

Metacognitive developments in word learning:mutual exclusivity and theory of mind

This study examines the flexibility with which children can use pragmatic information to determine word reference. Extensive previous research shows that children choose an unfamiliar object as referent of a novel name: the disambiguation effect. We added a pragmatic cue indirectly indicating a familiar object as intended referent. In three experiments, preschool children’s ability to take this cue into account was specifically associated with false belief understanding and the ability to produce familiar alternative names (e.g., rabbit, animal) for a given referent. The association was predicted by the hypothesis that all three tasks require an understanding of perspective (linguistic or mental). The findings indicate that perspectival understanding is required to take into account indirect pragmatic information to suspend the disambiguation effect. Implications for lexical principles and socio-pragmatic theories of word learning are discussed

Digitization and search: A non-traditional use of HPC

We describe our efforts in developing an open source cyberinfrastructure to provide a form of automated search of handwritten content within large digitized document archives. Such collections are a treasure trove of data ranging from decades ago to as far as the present. The information contained in these collections is also very relevant to both researchers who might extract numerical or statistical data from such sources as well as the general public. With the push to digitize our paper archives we are, how-ever, faced with the fact that though these digital versions are easier to share, they are not trivially searchable as the digitiza-tion process produces image data and not text. This inability to find and/or identify contents within these collections makes this data largely unusable without a lengthy and costly manual transcription process carried out by human beings