Multitask Prompted Training Enables Zero-Shot Task Generalization

International audienceLarge language models have recently been shown to attain reasonable zero-shot generalization on a diverse set of tasks (Brown et al., 2020). It has been hypothesized that this is a consequence of implicit multitask learning in language models’ pretraining (Radford et al., 2019). Can zero-shot generalization instead be directly induced by explicit multitask learning? To test this question at scale, we develop a system for easily mapping any natural language tasks into a human-readable prompted form. We convert a large set of supervised datasets, each with multiple prompts with diverse wording. These prompted datasets allow for benchmarking the ability of a model to perform completely held-out tasks. We fine-tune a pre-trained encoder-decoder model (Raffel et al., 2020; Lester et al., 2021) on this multitask mixture covering a wide variety of tasks. The model attains strong zero-shot performance on several standard datasets, often outperforming models up to 16x its size. Further, our approach attains strong performance on a subset of tasks from the BIG-bench benchmark, outperforming models up to 6x its size. All trained models are available at https://github.com/bigscience-workshop/t-zero, and all prompts are available at https://github.com/bigscience-workshop/promptsource

BLOOM: A 176B-Parameter Open-Access Multilingual Language Model

Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions. While these capabilities have led to widespread adoption, most LLMs are developed by resource-rich organizations and are frequently kept from the public. As a step towards democratizing this powerful technology, we present BLOOM, a 176B-parameter open-access language model designed and built thanks to a collaboration of hundreds of researchers. BLOOM is a decoder-only Transformer language model that was trained on the ROOTS corpus, a dataset comprising hundreds of sources in 46 natural and 13 programming languages (59 in total). We find that BLOOM achieves competitive performance on a wide variety of benchmarks, with stronger results after undergoing multitask prompted finetuning. To facilitate future research and applications using LLMs, we publicly release our models and code under the Responsible AI License

Functional traits and phylogenetic analysis of top-soil inhabiting rhizobacteria associated with tea rhizospheres in North Bengal, India

Rhizobacteria associated with cultivated crops are known to stimulate plant growth through various indirect or direct mechanisms. In recent years, the host list of plant growth promotion/promoting rhizobacteria has expanded to include bean, barley, cotton, maize, rice, vegetables, peanut, rice, wheat, and several plantation crops. However, interaction of rhizobacteria with tea plants of organic and conventional tea gardens is poorly understood. In the present study, rhizobacterial species associated with tea rhizosphere were isolated from 14 tea gardens located in North Bengal, India. In total, 16 rhizobacterial isolates isolated from collected soil samples were assessed for antagonistic and plant growth promotion/promoting activity under laboratory conditions. Molecular characterization based on sequencing of 16S rRNA gene revealed dominance of Bacillus with five species followed by Pseudomonas with two species. Interestingly, only one isolate was affiliated with actinobacteria, i.e., Microbacterium barkeri. Out of 16 isolates, isolates Bacillus subtilis OKAKP01, B. subtilis BNLG01, B. paramycoides BOK01, M. barkeri BPATH02, and Stenotrophomonas maltophilia BSEY01 showed highest growth inhibition against Fusarium solani (68.2 to 72.8%), Pseudopestalotiopsis theae (71.1 to 85.6%), and Exobasidium vexans (67.4 to 78.3%) causing respective Fusarium dieback, gray blight, and blister blight diseases in tea crop. Further, these five isolates also possessed significantly greater antifungal (siderophore producer, protease, chitinase, and cellulase activity) and plant growth promotion/promoting (indole-3-acetic acid production, ACC deaminase, ammonia, and phosphate solubilization) traits over other eleven rhizobacterial isolates. Therefore, these five isolates of rhizobacteria were chosen for their plant growth promotion/promoting activity on tea plants in nursery conditions. Results from nursery experiments revealed that these five rhizobacteria significantly improved growth rates of tea plants compared with the control. Therefore, this study suggests that these rhizobacteria could be used to formulate biopesticides and biofertilizers, which could be applied to sustainable tea cultivation to improve crop health and reduce disease attack

Multitask Prompted Training Enables Zero-Shot Task Generalization

International audienceLarge language models have recently been shown to attain reasonable zero-shot generalization on a diverse set of tasks (Brown et al., 2020). It has been hypothesized that this is a consequence of implicit multitask learning in language model training (Radford et al., 2019). Can zero-shot generalization instead be directly induced by explicit multitask learning? To test this question at scale, we develop a system for easily mapping general natural language tasks into a human-readable prompted form. We convert a large set of supervised datasets, each with multiple prompts using varying natural language. These prompted datasets allow for benchmarking the ability of a model to perform completely unseen tasks specified in natural language. We fine-tune a pretrained encoder-decoder model (Raffel et al., 2020; Lester et al., 2021) on this multitask mixture covering a wide variety of tasks. The model attains strong zero-shot performance onseveral datasets, often outperforming models 16× its size. Further, our model attains strong performance on a subset of tasks from the BIG-Bench benchmark, out-performing models 6× its size. All prompts and trained models are available at https://github.com/bigscience-workshop/promptsource/ and https://huggingface.co/bigscience/T0pp