42 research outputs found
Teaching Small Language Models to Reason
Chain of thought prompting successfully improves the reasoning capabilities
of large language models, achieving state of the art results on a range of
datasets. However, these reasoning capabilities only appear to emerge in models
with a size of over 100 billion parameters. In this paper, we explore the
transfer of such reasoning capabilities to models with less than 100 billion
parameters via knowledge distillation. Specifically, we finetune a student
model on the chain of thought outputs generated by a larger teacher model. Our
experiments show that the proposed method improves task performance across
arithmetic, commonsense and symbolic reasoning datasets. For example, the
accuracy of T5 XXL on GSM8K improves from 8.11% to 21.99% when finetuned on
PaLM-540B generated chains of thought
Small Language Models Improve Giants by Rewriting Their Outputs
Large language models (LLMs) have demonstrated impressive few-shot learning
capabilities, but they often underperform compared to fine-tuned models on
challenging tasks. Furthermore, their large size and restricted access only
through APIs make task-specific fine-tuning impractical. Moreover, LLMs are
sensitive to different aspects of prompts (e.g., the selection and order of
demonstrations) and can thus require time-consuming prompt engineering. In this
light, we propose a method to correct LLM outputs without relying on their
weights. First, we generate a pool of candidates by few-shot prompting an LLM.
Second, we refine the LLM-generated outputs using a smaller model, the
LM-corrector (LMCor), which is trained to rank, combine and rewrite the
candidates to produce the final target output. Our experiments demonstrate that
even a small LMCor model (250M) substantially improves the few-shot performance
of LLMs (62B) across diverse tasks. Moreover, we illustrate that the LMCor
exhibits robustness against different prompts, thereby minimizing the need for
extensive prompt engineering. Finally, we showcase that the LMCor can be
seamlessly integrated with different LLMs at inference time, serving as a
plug-and-play module to improve their performance
Special Issue “Organophosphorus Chemistry: A New Perspective”
The European Chemical Society (EuChemS) and the European Parliament (Science and Policy Workshop, 25 May 2023) recognize phosphorus as one of the key chemical elements in daily life [...
Efektywne pamięciowo trwałe struktury danych
Trwałe struktury danych mają zastosowanie w wielu dziedzinach, m.in. implementacji kompilatorów i algorytmów geometrycznych. Driscoll et al. pokazali jak wydajnie uczynić dowolną wskaźnikową strukturę danych strukturą trwałą. Wzbogacamy ich metody o możliwość efektywnego pamięciowo usuwania wersji. Pogarszamy przy tym złożoności czasowe jedynie o czynnik logarytmiczny.Persistent data structures are useful in many ares, e.g. compiler implementation and geometric algorithms. Driscoll et al. have shown how to make arbitrary linked data structures persistent efficiently. We augment their methods to allow memory efficient deletion of versions. The obtained time complexities are worse by only a logarithmic factor
Allobetulin
Allobetulin was synthesized at room temperature, starting from betulin by Wagner–Meerwein rearrangement in the presence of tetrafluoroboric acid diethyl ether complex. The structure of the compound obtained was confirmed by spectroscopic methods (1H, 13C NMR and IR)
Allobetulin
Allobetulin was synthesized at room temperature, starting from betulin by Wagner–Meerwein rearrangement in the presence of tetrafluoroboric acid diethyl ether complex. The structure of the compound obtained was confirmed by spectroscopic methods (1H, 13C NMR and IR)