Teaching Small Language Models to Reason

Chain of thought prompting successfully improves the reasoning capabilities of large language models, achieving state of the art results on a range of datasets. However, these reasoning capabilities only appear to emerge in models with a size of over 100 billion parameters. In this paper, we explore the transfer of such reasoning capabilities to models with less than 100 billion parameters via knowledge distillation. Specifically, we finetune a student model on the chain of thought outputs generated by a larger teacher model. Our experiments show that the proposed method improves task performance across arithmetic, commonsense and symbolic reasoning datasets. For example, the accuracy of T5 XXL on GSM8K improves from 8.11% to 21.99% when finetuned on PaLM-540B generated chains of thought

Small Language Models Improve Giants by Rewriting Their Outputs

Large language models (LLMs) have demonstrated impressive few-shot learning capabilities, but they often underperform compared to fine-tuned models on challenging tasks. Furthermore, their large size and restricted access only through APIs make task-specific fine-tuning impractical. Moreover, LLMs are sensitive to different aspects of prompts (e.g., the selection and order of demonstrations) and can thus require time-consuming prompt engineering. In this light, we propose a method to correct LLM outputs without relying on their weights. First, we generate a pool of candidates by few-shot prompting an LLM. Second, we refine the LLM-generated outputs using a smaller model, the LM-corrector (LMCor), which is trained to rank, combine and rewrite the candidates to produce the final target output. Our experiments demonstrate that even a small LMCor model (250M) substantially improves the few-shot performance of LLMs (62B) across diverse tasks. Moreover, we illustrate that the LMCor exhibits robustness against different prompts, thereby minimizing the need for extensive prompt engineering. Finally, we showcase that the LMCor can be seamlessly integrated with different LLMs at inference time, serving as a plug-and-play module to improve their performance

Special Issue “Organophosphorus Chemistry: A New Perspective”

The European Chemical Society (EuChemS) and the European Parliament (Science and Policy Workshop, 25 May 2023) recognize phosphorus as one of the key chemical elements in daily life [...

Efektywne pamięciowo trwałe struktury danych

Trwałe struktury danych mają zastosowanie w wielu dziedzinach, m.in. implementacji kompilatorów i algorytmów geometrycznych. Driscoll et al. pokazali jak wydajnie uczynić dowolną wskaźnikową strukturę danych strukturą trwałą. Wzbogacamy ich metody o możliwość efektywnego pamięciowo usuwania wersji. Pogarszamy przy tym złożoności czasowe jedynie o czynnik logarytmiczny.Persistent data structures are useful in many ares, e.g. compiler implementation and geometric algorithms. Driscoll et al. have shown how to make arbitrary linked data structures persistent efficiently. We augment their methods to allow memory efficient deletion of versions. The obtained time complexities are worse by only a logarithmic factor

Allobetulin

Allobetulin was synthesized at room temperature, starting from betulin by Wagner–Meerwein rearrangement in the presence of tetrafluoroboric acid diethyl ether complex. The structure of the compound obtained was confirmed by spectroscopic methods (1H, 13C NMR and IR)

Allobetulin

Allobetulin was synthesized at room temperature, starting from betulin by Wagner–Meerwein rearrangement in the presence of tetrafluoroboric acid diethyl ether complex. The structure of the compound obtained was confirmed by spectroscopic methods (1H, 13C NMR and IR)