Giraffe: Adventures in Expanding Context Lengths in LLMs

Modern large language models (LLMs) that rely on attention mechanisms are typically trained with fixed context lengths which enforce upper limits on the length of input sequences that they can handle at evaluation time. To use these models on sequences longer than the train-time context length, one might employ techniques from the growing family of context length extrapolation methods -- most of which focus on modifying the system of positional encodings used in the attention mechanism to indicate where tokens or activations are located in the input sequence. We conduct a wide survey of existing methods of context length extrapolation on a base LLaMA or LLaMA 2 model, and introduce some of our own design as well -- in particular, a new truncation strategy for modifying the basis for the position encoding. We test these methods using three new evaluation tasks (FreeFormQA, AlteredNumericQA, and LongChat-Lines) as well as perplexity, which we find to be less fine-grained as a measure of long context performance of LLMs. We release the three tasks publicly as datasets on HuggingFace. We discover that linear scaling is the best method for extending context length, and show that further gains can be achieved by using longer scales at evaluation time. We also discover promising extrapolation capabilities in the truncated basis. To support further research in this area, we release three new 13B parameter long-context models which we call Giraffe: 4k and 16k context models trained from base LLaMA-13B, and a 32k context model trained from base LLaMA2-13B. We also release the code to replicate our results

SCAN: Learning Hierarchical Compositional Visual Concepts

The seemingly infinite diversity of the natural world arises from a relatively small set of coherent rules, such as the laws of physics or chemistry. We conjecture that these rules give rise to regularities that can be discovered through primarily unsupervised experiences and represented as abstract concepts. If such representations are compositional and hierarchical, they can be recombined into an exponentially large set of new concepts. This paper describes SCAN (Symbol-Concept Association Network), a new framework for learning such abstractions in the visual domain. SCAN learns concepts through fast symbol association, grounding them in disentangled visual primitives that are discovered in an unsupervised manner. Unlike state of the art multimodal generative model baselines, our approach requires very few pairings between symbols and images and makes no assumptions about the form of symbol representations. Once trained, SCAN is capable of multimodal bi-directional inference, generating a diverse set of image samples from symbolic descriptions and vice versa. It also allows for traversal and manipulation of the implicit hierarchy of visual concepts through symbolic instructions and learnt logical recombination operations. Such manipulations enable SCAN to break away from its training data distribution and imagine novel visual concepts through symbolically instructed recombination of previously learnt concepts

An Upper Lip Swelling: A Case Report, Differential Diagnosis and Review of Literature

Pleomorphic adenoma is the most common tumor of the salivary glands. About 90% of these tumors occur in the parotid gland and 10% in the minor salivary glands. The most common site for pleomorphic adenoma of the minor salivary glands is the palate followed by upper lip. Surgical removal with adequate margins is the principal treatment. Due to its microscopicprojections, this tumor requires a wide resection to avoid recurrence. We report a case of pleomorphic adenoma in the upper lip of an elderly female with emphasis on differentialdiagnosis and relevant review of literature

Fusarium Disease of Maize and Its Management through Sustainable Approach

Fusarium causing disease in maize is probably the one of the most serious diseases among the crop plants all over the world. It not only damages the maize plant, reduces its potential yield and its nutritional values but imposes threatening to the human life through the induction of mycotoxin development. F. graminearum and F. moniliforme syn. Fusarium verticillioides are two important maize pathogens that cause substantial damage to its ear, stalk and foliage, causing contamination of grains with mycotoxins. Since conventional methods of controlling the diseases including the chemical methods proved not enough for total control of the disease with creating situation even worse for our surroundings, the application of PGPR and PGPF can play significant role to control the damage caused by Fusarium

Search for post-merger gravitational waves from the remnant of the binary neutron star merger GW170817

In Advanced LIGO, detection and astrophysical source parameter estimation of the binary black hole merger GW150914 requires a calibrated estimate of the gravitational-wave strain sensed by the detectors. Producing an estimate from each detector's differential arm length control loop readout signals requires applying time domain filters, which are designed from a frequency domain model of the detector's gravitational-wave response. The gravitational-wave response model is determined by the detector's opto-mechanical response and the properties of its feedback control system. The measurements used to validate the model and characterize its uncertainty are derived primarily from a dedicated photon radiation pressure actuator, with cross-checks provided by optical and radio frequency references. We describe how the gravitational-wave readout signal is calibrated into equivalent gravitational-wave-induced strain and how the statistical uncertainties and systematic errors are assessed. Detector data collected over 38 calendar days, from September 12 to October 20, 2015, contain the event GW150914 and approximately 16 of coincident data used to estimate the event false alarm probability. The calibration uncertainty is less than 10% in magnitude and 10 degrees in phase across the relevant frequency band 20 Hz to 1 kHz

First narrow-band search for continuous gravitational waves from known pulsars in advanced detector data

In Advanced LIGO, detection and astrophysical source parameter estimation of the binary black hole merger GW150914 requires a calibrated estimate of the gravitational-wave strain sensed by the detectors. Producing an estimate from each detector's differential arm length control loop readout signals requires applying time domain filters, which are designed from a frequency domain model of the detector's gravitational-wave response. The gravitational-wave response model is determined by the detector's opto-mechanical response and the properties of its feedback control system. The measurements used to validate the model and characterize its uncertainty are derived primarily from a dedicated photon radiation pressure actuator, with cross-checks provided by optical and radio frequency references. We describe how the gravitational-wave readout signal is calibrated into equivalent gravitational-wave-induced strain and how the statistical uncertainties and systematic errors are assessed. Detector data collected over 38 calendar days, from September 12 to October 20, 2015, contain the event GW150914 and approximately 16 of coincident data used to estimate the event false alarm probability. The calibration uncertainty is less than 10% in magnitude and 10 degrees in phase across the relevant frequency band 20 Hz to 1 kHz

Saving Love for Valentine's Day

Saving Love for Valentine's day presents a quest of a college student attempting to rescue his significant other by his first Valentine. In his journey, the main character is in a time constraint and has various mental/physical obstacles that may affect the outcome of the game

The X Chromosome of Hemipteran Insects: Conservation, Dosage Compensation and Sex-Biased Expression

Insects of the order Hemiptera (true bugs) use a wide range of mechanisms of sex determination, including genetic sex determination, paternal genome elimination, and haplodiploidy. Genetic sex determination, the prevalent mode, is generally controlled by a pair of XY sex chromosomes or by an XX/XO system, but different configurations that include additional sex chromosomes are also present. Although this diversity of sex determining systems has been extensively studied at the cytogenetic level, only the X chromosome of the model pea aphid Acyrthosiphon pisum has been analyzed at the genomic level, and little is known about X chromosome biology in the rest of the order. In this study, we take advantage of published DNA- and RNA-seq data from three additional Hemiptera species to perform a comparative analysis of the gene content and expression of the X chromosome throughout this clade. We find that, despite showing evidence of dosage compensation, the X chromosomes of these species show female-biased expression, and a deficit of male-biased genes, in direct contrast to the pea aphid X. We further detect an excess of shared gene content between these very distant species, suggesting that despite the diversity of sex determining systems, the same chromosomal element is used as the X throughout a large portion of the order