Interpreting Context Look-ups in Transformers: Investigating Attention-MLP Interactions

In this paper, we investigate the interplay between attention heads and specialized "next-token" neurons in the Multilayer Perceptron that predict specific tokens. By prompting an LLM like GPT-4 to explain these model internals, we can elucidate attention mechanisms that activate certain next-token neurons. Our analysis identifies attention heads that recognize contexts relevant to predicting a particular token, activating the associated neuron through the residual connection. We focus specifically on heads in earlier layers consistently activating the same next-token neuron across similar prompts. Exploring these differential activation patterns reveals that heads that specialize for distinct linguistic contexts are tied to generating certain tokens. Overall, our method combines neural explanations and probing isolated components to illuminate how attention enables context-dependent, specialized processing in LLMs.Comment: 15 pages, 11 figure

Consumption-Based Conservation Targeting: Linking Biodiversity Loss to Upstream Demand through a Global Wildlife Footprint.

Although most conservation efforts address the direct, local causes of biodiversity loss, effective long-term conservation will require complementary efforts to reduce the upstream economic pressures, such as demands for food and forest products, which ultimately drive these downstream losses. Here, we present a wildlife footprint analysis that links global losses of wild birds to consumer purchases across 57 economic sectors in 129 regions. The United States, India, China, and Brazil have the largest regional wildlife footprints, while per-person footprints are highest in Mongolia, Australia, Botswana, and the United Arab Emirates. A US$100 purchase of bovine meat or rice products occupies approximately 0.1 km2 of wild bird ranges, displacing 1-2 individual birds, for 1 year. Globally significant importer regions, including Japan, the United Kingdom, Germany, Italy, and France, have large footprints that drive wildlife losses elsewhere in the world and represent important targets for consumption-focused conservation attention

Toksisitas Bahan Pengawet Boron-kromium Terhadap Serangga Dan Jamur Pelapuk Kayu

Penelitian ini bertujuan untuk menguji toksisitas bahan pengawet boron-kromium terhadap rayap dan jamur pelapuk kayu. Bahan yang digunakan merupakan bahan kimia dengan kualitas teknis. Pengawetan contoh uji dilakukan menggunakan proses vakum tekan dengan lima konsentrasi yang berbeda. Untuk pengujian terhadap rayap tanah, 200 ekor pekerja rayap tanah Coptotermes curvignathus Holmgren dimasukkan ke dalam jam pot yang berisi media pasir dan contoh uji berupa kayu yang berukuran 25 mm x 25 mm x 5 mm. Untuk rayap kayu kering, lima puluh ekor pekerja rayap kayu kering Cryptotermes cynocephalus Light dimasukkan ke dalam tabung gelas berdiameter 18 mm dengan tinggi 35 mm yang dipasang pada salah satu sisi terlebar contoh uji yang berukuran 50 mm x 25 mm x 20 mm. Sementara terhadap jamur pelapuk kayu, pengujian dilakukan dengan jamur Schizophyllum commune yang dibiakkan dalam mediapotato dextrose agar (PDA). Hasil pengujian menunjukkan bahwa bahan pengawet yang digunakan efektif mencegah serangan rayap tanah C. curvignathus Holmgren dan rayap kayu kering C. cynocephalus Light. Efektivitas bahan pengawet dengan menyebabkan mortalitas rayap sebesar 100% diperoleh pada konsentrasi 1% dengan retensi masing-masing 6,01 kg/m3dan 5,64 kg/m3. Namun demikian terhadap jamur pelapuk kayu S. commune konsentrasi efektif diperoleh pada larutan 7,5% dengan retensi 45,44 kg/m3

Keefektifan Campuran Garam Tembaga-khromium- Boron Terhadap Rayap Dan Jamur Perusak Kayu

Tujuan penelitian ini adalah mengevaluasi toksisitas formula garam tembaga-khromium-boron terhadap serangan rayap dan jamur Perusak kayu. Dalam penelitian ini digunakan bahan kimia kualitas teknis. Hasil pengujian menunjukkan bahwa bahan pengawet yang digunakan sangat efektif mencegah serangan rayap tanah Coptotermes curvignathus Holmgren dan rayap kayu kering Cryptotermes cynocephalus Light. Keefektifan ditunjukkan oleh tingkat mortalitas rayap 100% pada konsentrasi 1% dan retensi lebih dari 5,30 kg/m3. Namun demikian terhadap jamur pelapuk kayu Schizophyllum communekonsentrasi efektif diperoleh pada larutan 7,5% dan retensi di atas 30 kg/m3

Reliability and factor structure of the audit among male and female bar patrons in a rural area of South Africa

We assessed the reliability and dimensional structure of the Alcohol Use Disorders Identification Test (AUDIT) among bar patrons in a rural area of South Africa. In total, 406 bar patrons completed a questionnaire containing the AUDIT, and demographic and psychosocial measures. The participants consisted of 314 (77.3%) males and 92 (22.6%) females. Their combined mean age was 30.0 years (SD = 8.45). The data were analysed using Confirmatory Factor Analysis (CFA) and Cronbach’s alpha reliability analysis which were conducted separately for males and females. We found that Cronbach’s alpha for the AUDIT was 0.81 and 0.72 for the males and females, respectively. CFA supported a two-factor and three-factor model for the males but failed to support a one-factor, two-factor, or three-factor model for the females. The results suggest that the AUDIT is highly reliable, but that potential gender differences in its factor structure should be considered, particularly when applied in new contexts.Keywords: AUDIT; South Africa; Bar patron

A New Evolutionary Path to Type Ia Supernovae: Helium-Rich Super-Soft X-Ray Source Channel

We have found a new evolutionary path to Type Ia supernovae (SNe Ia) which has been overlooked in previous work. In this scenario, a carbon-oxygen white dwarf (C+O WD) is originated, not from an asymptotic giant branch star with a C+O core, but from a red-giant star with a helium core of $\sim 0.8-2.0 M_\odot$. The helium star, which is formed after the first common envelope evolution, evolves to form a C+O WD of $\sim 0.8-1.1 M_\odot$ with transferring a part of the helium envelope onto the secondary main-sequence star. This new evolutionary path, together with the optically thick wind from mass-accreting white dwarf, provides a much wider channel to SNe Ia than previous scenarios. A part of the progenitor systems are identified as the luminous supersoft X-ray sources or the recurrent novae like U Sco, which are characterized by the accretion of helium-rich matter. The white dwarf accretes hydrogen-rich, helium-enhanced matter from a lobe-filling, slightly evolved companion at a critical rate and blows excess matter in the wind. The white dwarf grows in mass to the Chandrasekhar mass limit and explodes as an SN Ia. A theoretical estimate indicates that this channel contributes a considerable part of the inferred rate of SNe Ia in our Galaxy, i.e., the rate is about ten times larger than the previous theoretical estimates for white dwarfs with slightly evolved companions.Comment: 19 pages including 12 figures, to be published in ApJ, 519, No.

High Performance PbS Quantum Dot/Graphene Hybrid Solar Cell with Efficient Charge Extraction.

Hybrid colloidal quantum dot (CQD) solar cells are fabricated from multilayer stacks of lead sulfide (PbS) CQD and single layer graphene (SG). The inclusion of graphene interlayers is shown to increase power conversion efficiency by 9.18%. It is shown that the inclusion of conductive graphene enhances charge extraction in devices. Photoluminescence shows that graphene quenches emission from the quantum dot suggesting spontaneous charge transfer to graphene. CQD photodetectors exhibit increased photoresponse and improved transport properties. We propose that the CQD/SG hybrid structure is a route to make CQD thin films with improved charge extraction, therefore resulting in improved solar cell efficiency