Textbooks Are All You Need II: phi-1.5 technical report

We continue the investigation into the power of smaller Transformer-based language models as initiated by \textbf{TinyStories} -- a 10 million parameter model that can produce coherent English -- and the follow-up work on \textbf{phi-1}, a 1.3 billion parameter model with Python coding performance close to the state-of-the-art. The latter work proposed to use existing Large Language Models (LLMs) to generate ``textbook quality" data as a way to enhance the learning process compared to traditional web data. We follow the ``Textbooks Are All You Need" approach, focusing this time on common sense reasoning in natural language, and create a new 1.3 billion parameter model named \textbf{phi-1.5}, with performance on natural language tasks comparable to models 5x larger, and surpassing most non-frontier LLMs on more complex reasoning tasks such as grade-school mathematics and basic coding. More generally, \textbf{phi-1.5} exhibits many of the traits of much larger LLMs, both good -- such as the ability to ``think step by step" or perform some rudimentary in-context learning -- and bad, including hallucinations and the potential for toxic and biased generations -- encouragingly though, we are seeing improvement on that front thanks to the absence of web data. We open-source \textbf{phi-1.5} to promote further research on these urgent topics

Becoming an Older Volunteer: A Grounded Theory Study

This Grounded Theory study describes the process by which older persons “become” volunteers. Forty interviews of older persons who volunteered for Habitat for Humanity were subjected to secondary content analysis to uncover the process of “becoming” a volunteer. “Helping out” (core category) for older volunteers occurs within the context of “continuity”, “commitment” and “connection” which provide motivation for volunteering. When a need arises, older volunteers “help out” physically and financially as health and resources permit. Benefits described as “blessings” of volunteering become motivators for future volunteering. Findings suggest that older volunteering is a developmental process and learned behavior which should be fostered in older persons by personally inviting them to volunteer. Intergenerational volunteering projects will allow older persons to pass on knowledge and skills and provide positive role modeling for younger volunteers

Population-based rare variant detection via pooled exome or custom hybridization capture with or without individual indexing

BACKGROUND: Rare genetic variation in the human population is a major source of pathophysiological variability and has been implicated in a host of complex phenotypes and diseases. Finding disease-related genes harboring disparate functional rare variants requires sequencing of many individuals across many genomic regions and comparing against unaffected cohorts. However, despite persistent declines in sequencing costs, population-based rare variant detection across large genomic target regions remains cost prohibitive for most investigators. In addition, DNA samples are often precious and hybridization methods typically require large amounts of input DNA. Pooled sample DNA sequencing is a cost and time-efficient strategy for surveying populations of individuals for rare variants. We set out to 1) create a scalable, multiplexing method for custom capture with or without individual DNA indexing that was amenable to low amounts of input DNA and 2) expand the functionality of the SPLINTER algorithm for calling substitutions, insertions and deletions across either candidate genes or the entire exome by integrating the variant calling algorithm with the dynamic programming aligner, Novoalign. RESULTS: We report methodology for pooled hybridization capture with pre-enrichment, indexed multiplexing of up to 48 individuals or non-indexed pooled sequencing of up to 92 individuals with as little as 70 ng of DNA per person. Modified solid phase reversible immobilization bead purification strategies enable no sample transfers from sonication in 96-well plates through adapter ligation, resulting in 50% less library preparation reagent consumption. Custom Y-shaped adapters containing novel 7 base pair index sequences with a Hamming distance of ≥2 were directly ligated onto fragmented source DNA eliminating the need for PCR to incorporate indexes, and was followed by a custom blocking strategy using a single oligonucleotide regardless of index sequence. These results were obtained aligning raw reads against the entire genome using Novoalign followed by variant calling of non-indexed pools using SPLINTER or SAMtools for indexed samples. With these pipelines, we find sensitivity and specificity of 99.4% and 99.7% for pooled exome sequencing. Sensitivity, and to a lesser degree specificity, proved to be a function of coverage. For rare variants (≤2% minor allele frequency), we achieved sensitivity and specificity of ≥94.9% and ≥99.99% for custom capture of 2.5 Mb in multiplexed libraries of 22–48 individuals with only ≥5-fold coverage/chromosome, but these parameters improved to ≥98.7 and 100% with 20-fold coverage/chromosome. CONCLUSIONS: This highly scalable methodology enables accurate rare variant detection, with or without individual DNA sample indexing, while reducing the amount of required source DNA and total costs through less hybridization reagent consumption, multi-sample sonication in a standard PCR plate, multiplexed pre-enrichment pooling with a single hybridization and lesser sequencing coverage required to obtain high sensitivity

The Grizzly, October 30, 2014

Board Announces National Search for New President • Residents Relocated After Fire • Clubs Host First Ever Festifall • Yik Yak Exposes Use of Fake IDs • Midterm Elections Approaching • Halloween Compared to Other International Holidays • UC Alumni Return • Viewers are Drawn to Colonial Theatre in Phoenixville • Student Interns at Disney World • Opinion: Must Halloween Costumes for Women be Sexy?; Occupy Movement in Hong Kong Persists • Local Athlete Stong Giving Field Hockey a Scoring Spark • Diving Into 2014-15 Slate • Seri-ously Goodhttps://digitalcommons.ursinus.edu/grizzlynews/1914/thumbnail.jp

Transcriptional Regulation of Arabidopsis Polycomb Repressive Complex 2 Coordinates Cell Type Proliferation and Differentiation

Spatiotemporal regulation of transcription is fine-tuned at multiple levels, including chromatin compaction. Polycomb Repressive Complex 2 (PRC2) catalyzes the trimethylation of Histone 3 at lysine 27 (H3K27me3), which is the hallmark of a repressive chromatin state. Multiple PRC2 complexes have been reported in Arabidopsis thaliana to control the expression of genes involved in developmental transitions and maintenance of organ identity. Here, we show that PRC2 member genes display complex spatiotemporal gene expression patterns and function in root meristem and vascular cell proliferation and specification. Furthermore, PRC2 gene expression patterns correspond with vascular and non-vascular tissue-specific H3K27me3-marked genes. This tissue-specific repression via H3K27me3 regulates the balance between cell proliferation and differentiation. Using enhanced yeast-one-hybrid analysis, upstream regulators of the PRC2 member genes are identified, and genetic analysis demonstrates that transcriptional regulation of some PRC2 genes plays an important role in determining PRC2 spatiotemporal activity within a developing organ

The Grizzly, November 20, 2014

Instruments Stolen From Bomberger • Memorial Honors Ambassador Melrose • Senate Calls Emergency Meeting • Religious Realizations • Compost Company Shut Down • Transitioning from Undergrad to Corson • YAL Spreads Philosophy of Freedom Around UC Campus • Sycamore Tree Remembered in New Logo Shield Designs • Opinion: People Aren\u27t Listening to Victims of Rape; UC Website Emphasizes Students Too Much • Sans Seniors, Women\u27s Basketball Hopes to Improve • An Ocean Away • Stellar Hockey Season Endshttps://digitalcommons.ursinus.edu/grizzlynews/1917/thumbnail.jp

Morphological characterization and immunohistochemical detection of the proinflammatory cytokines IL-1β, IL-17A, and TNF-α in lung lesions associated with contagious bovine pleuropneumonia

Contagious bovine pleuropneumonia (CBPP), a severe respiratory disease, is characterized by massive inflammation of the lung especially during the acute clinical stage of infection. Tissue samples from cattle, experimentally infected with Mycoplasma mycoides subsp. mycoides Afadé, were subjected to histopathological and immunohistochemical examination in order to provide insight into innate immune pathways that shape inflammatory host responses. Lung lesions were characterized by vasculitis, necrosis, and increased presence of macrophages and neutrophils, relative to uninfected animals. The presence of three cytokines associated with innate inflammatory immune responses, namely, IL-1β, IL-17A, and TNF-α, were qualitatively investigated in situ. Higher cytokine levels were detected in lung tissue samples from CBPP-affected cattle compared to samples derived from an uninfected control group. We therefore conclude that the cytokines TNF-α and IL-1β, which are prevalent in the acute phase of infections, play a role in the inflammatory response seen in the lung tissue in CBPP. IL-17A gets released by activated macrophages and attracts granulocytes that modulate the acute phase of the CBPP lesions. ELECTRONIC SUPPLEMENTARY MATERIAL: The online version of this article (doi:10.1007/s11250-016-0994-9) contains supplementary material, which is available to authorized users

A large genome-wide association study of age-related macular degeneration highlights contributions of rare and common variants.

This is the author accepted manuscript. The final version is available from Nature Publishing Group via http://dx.doi.org/10.1038/ng.3448Advanced age-related macular degeneration (AMD) is the leading cause of blindness in the elderly, with limited therapeutic options. Here we report on a study of >12 million variants, including 163,714 directly genotyped, mostly rare, protein-altering variants. Analyzing 16,144 patients and 17,832 controls, we identify 52 independently associated common and rare variants (P < 5 × 10(-8)) distributed across 34 loci. Although wet and dry AMD subtypes exhibit predominantly shared genetics, we identify the first genetic association signal specific to wet AMD, near MMP9 (difference P value = 4.1 × 10(-10)). Very rare coding variants (frequency <0.1%) in CFH, CFI and TIMP3 suggest causal roles for these genes, as does a splice variant in SLC16A8. Our results support the hypothesis that rare coding variants can pinpoint causal genes within known genetic loci and illustrate that applying the approach systematically to detect new loci requires extremely large sample sizes.We thank all participants of all the studies included for enabling this research by their participation in these studies. Computer resources for this project have been provided by the high-performance computing centers of the University of Michigan and the University of Regensburg. Group-specific acknowledgments can be found in the Supplementary Note. The Center for Inherited Diseases Research (CIDR) Program contract number is HHSN268201200008I. This and the main consortium work were predominantly funded by 1X01HG006934-01 to G.R.A. and R01 EY022310 to J.L.H