Towards interpretable sequence continuation: analyzing shared circuits in large language models

While transformer models exhibit strong capabilities on linguistic tasks, their complex architectures make them difficult to interpret. Recent work has aimed to reverse engineer transformer models into human-readable representations called circuits that implement algorithmic functions. We extend this research by analyzing and comparing circuits for similar sequence continuation tasks, which include increasing sequences of Arabic numerals, number words, and months. By applying circuit interpretability analysis, we identify a key sub-circuit in both GPT-2 Small and Llama-2-7B responsible for detecting sequence members and for predicting the next member in a sequence. Our analysis reveals that semantically related sequences rely on shared circuit subgraphs with analogous roles. Additionally, we show that this sub-circuit has effects on various math-related prompts, such as on intervaled circuits, Spanish number word and months continuation, and natural language word problems. This mechanistic understanding of transformers is a critical step towards building more robust, aligned, and interpretable language models

PENDIDIKAN KEPEMIMPINAN DALAM KEGIATAN EKSTRAKURIKULER DI PONDOK MODERN DARUSSALAM GONTOR KAMPUS 2 PERIODE 1437-1438

Krisis kepemimpinan menjadi permasalahan utama bangsa Indonesia pada saat ini, hal ini ditandai dengan semakin berkurangnya kepercayaan masyarakat terhadap para pemimpin-pemimpin negeri ini, hal ini dapat terlihat dengan berbagai kasus korupsi yang terjadi serta protes dan demo yang terjadi baik secara langsung maupun lewat media social. Krisis pemimpin ini bukan disebabkan tidak adanya orang yang ingin memimpin tetapi kreadibilitas mereka yang belum mumpuni untuk menjadi pemimpin yang baik dan mampu dicontoh oleh masyarakatnya disinilah kualitas pendidikan Indonesia dipertanyakan, apakah pendidikan yang ada belum mampu untuk melahirkan pemimpin yang ideal. Melihat pentingnya peran seorang pemimpin ini maka penulis berusaha untuk meneliti pendidikan yang ada khususnya pendidikan kepemimpinan yang terdapat dalam kegiatan ekstrakurikuler. Untuk itu penulis memfokuskan penelitiannya dengan judul “Pendidikan Kepemimpinan Dalam Kegiatan Ekstrakurikuler di Pondok Modern Darussalam Gontor Kampus 2 Peroode 1437-1438”. Adapun metode yang digunakan penulis adalah metode deskriptif. Dan untuk mencapai hasil tersebut peneliti menggunakan metode interview, observasi dan dokumentasi untuk mengumpulkan data. Serta penulis menggunakan metode miles dan Huberman, yaitu reduksi data, penyajian data dan konklusi, untuk mengalisa data tersebut. Setelah melakukan penelitian tentang pendidikan kepemimpinan ini, peneliti menemukan bahwa kegiatan yang terdapat dalam kegiatan ekstrakurikuler di Pondok Modern Darussalam Gontor Kampus 2 telah berjalan dengan baik sesuai dengan apa yang telah menjadi definisi pendidikan nasional. Selain itu peneliti juga menemukan bahwa terdapat banyak kegiatan dan aktivitas dalam klub-klub kegiatan ekstrakurikuler yang memiliki dampak penting dalam pendidikan kepemimpinan bagi para siswa yang mengikuti kegiatan ekstrakurikuler tersebut. Proses pendidikan tersebut adalah penanaman dasar iman dan moral, berkomunikasi, penigkatan kemampuan, latihan memimpin dan mencetak pemimpin, pemberian tugas serta pemberian contoh yang baik walaupun dengan beberapa kekurangan di dalamnya. Semua itu sesuai dengan apa yang dijelaskan oleh Thoriq Suwaidan, Brata Taruna dan juga M Natsir. Dari penelitian ini penulis berpendapat bahwa pendidikan kepemimpinan sangat mungkin untuk dikembangkan dan dioptimalkan terutama lewat kegiatan-kegiatan sehari-hari yang dilakukan para siswa. Untuk itu peneliti berharap kepada peneliti selanjutnya dapat melanjutkan penelitian ini dengan pembahasan yang lebih mendalam tentan pendidikan kepemimpinan khususnya, baik di lingkungan rumah, sekolah maupun masyarakat

Towards interpreting visual information processing in vision-language models

Vision-Language Models (VLMs) are powerful tools for processing and understanding text and images. We study the processing of visual tokens in the language model component of LLaVA, a prominent VLM. Our approach focuses on analyzing the localization of object information, the evolution of visual token representations across layers, and the mechanism of integrating visual information for predictions. Through ablation studies, we demonstrated that object identification accuracy drops by over 70% when object-specific tokens are removed. We observed that visual token representations become increasingly interpretable in the vocabulary space across layers, suggesting an alignment with textual tokens corresponding to image content. Finally, we found that the model extracts object information from these refined representations at the last token position for prediction, mirroring the process in text-only language models for factual association tasks. These findings provide crucial insights into how VLMs process and integrate visual information, bridging the gap between our understanding of language and vision models, and paving the way for more interpretable and controllable multimodal systems

Emergency Pancreaticoduodenectomy in Duodenal Paraganglioma: Case Report

Duodenal gangliocytic paraganglioma (DGP) is a rare tumor that characteristically occurs in the second part of duodenum. These appear as submucosal masses that protrude into the lumen of a duodenum. Gastrointestinal bleeding is the commonest manifestation of DGP. Metastatic spread to regional lymph nodes occurs rarely. Surgical resection is the treatment of choice for DGP. A case of a DGP is reported in young female who presented with a recurrent upper gastrointestinal bleeding. Upper gastrointestinal endoscopy (UGIE) documented a mass in the ampullary region with ulceration in its middle which was bleeding. Recurrent gastrointestinal bleeding necessitated an emergency pancreaticoduodenectomy. Histopathology of specimen documented gangliocytic paraganglioma

A Unified method for vibration analysis of moderately thick annular, circular plates and their sector counterparts subjected to arbitrary boundary conditions

The vibrations of circular, annular and sector plates are different boundary value problems due to different edge conditions and thus have been treated separately using different solution algorithms and procedures. In this paper, a unified method is proposed for vibration analysis of moderately thick annular, circular plates and their sector counterparts with arbitrary boundary conditions. The unification of these plates is physically achieved by applying the coupling spring’s technique at the radial edges to ensure appropriate continuity conditions. Irrespective of the shape of the plate and the type of boundary conditions, each of the displacement function is expressed as a new form of trigonometric expansion with high convergence rate. Unlike most of the previous studies the current method can be universally applied to a wide range of vibration problems involving different shapes, boundary conditions, varying materials and geometric properties without modifying the solution algorithms and procedure. Furthermore, the current method can easily be applied to sector plates with an arbitrary inclusion angle of 2π. The accuracy, reliability and versatility of the proposed method are fully demonstrated with several numerical examples for different shapes of plates and under different boundary conditions

An improved Fourier series method for vibration analysis of moderately thick annular and circular sector plates subjected to elastic boundary conditions

In this paper, an improved Fourier series method is presented for vibration analysis of moderately thick annular and circular sector plates subjected to general elastic boundary conditions along its edges. In literature, annular and circular sector plates subjected to classical boundary conditions have been studied in detail however in practical engineering applications the boundary conditions are not always classical in nature. Therefore, study of vibration response of these plates subjected to general elastic boundary conditions is far needed. In the method presented, artificial boundary spring technique has been employed to simulate the general elastic boundary conditions and first order shear deformation theory has been employed to formulate the theoretical model. Irrespective of the boundary conditions, each of the displacement function is expressed as a new form of trigonometric expansion with accelerated convergence. Rayleigh-Ritz method has been employed to determine the expansion coefficients. Unlike most of the studies on vibration analysis of moderately thick annular sector plates, the present method can be universally applied to a wide range of vibration problems involving different boundary conditions, varying material and geometric properties without modifying the solution algorithms and procedure. The effectiveness, reliability and accuracy of the present method is fully demonstrated and verified by several numerical examples. Bench mark solutions for moderately thick annular sector and circular plates under general elastic boundary conditions are also presented for future computational methods

Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training

Humans are capable of strategically deceptive behavior: behaving helpfully in most situations, but then behaving very differently in order to pursue alternative objectives when given the opportunity. If an AI system learned such a deceptive strategy, could we detect it and remove it using current state-of-the-art safety training techniques? To study this question, we construct proof-of-concept examples of deceptive behavior in large language models (LLMs). For example, we train models that write secure code when the prompt states that the year is 2023, but insert exploitable code when the stated year is 2024. We find that such backdoor behavior can be made persistent, so that it is not removed by standard safety training techniques, including supervised fine-tuning, reinforcement learning, and adversarial training (eliciting unsafe behavior and then training to remove it). The backdoor behavior is most persistent in the largest models and in models trained to produce chain-of-thought reasoning about deceiving the training process, with the persistence remaining even when the chain-of-thought is distilled away. Furthermore, rather than removing backdoors, we find that adversarial training can teach models to better recognize their backdoor triggers, effectively hiding the unsafe behavior. Our results suggest that, once a model exhibits deceptive behavior, standard techniques could fail to remove such deception and create a false impression of safety.Comment: updated to add missing acknowledgement

Reducing the environmental impact of surgery on a global scale: systematic review and co-prioritization with healthcare workers in 132 countries

Abstract Background Healthcare cannot achieve net-zero carbon without addressing operating theatres. The aim of this study was to prioritize feasible interventions to reduce the environmental impact of operating theatres. Methods This study adopted a four-phase Delphi consensus co-prioritization methodology. In phase 1, a systematic review of published interventions and global consultation of perioperative healthcare professionals were used to longlist interventions. In phase 2, iterative thematic analysis consolidated comparable interventions into a shortlist. In phase 3, the shortlist was co-prioritized based on patient and clinician views on acceptability, feasibility, and safety. In phase 4, ranked lists of interventions were presented by their relevance to high-income countries and low–middle-income countries. Results In phase 1, 43 interventions were identified, which had low uptake in practice according to 3042 professionals globally. In phase 2, a shortlist of 15 intervention domains was generated. In phase 3, interventions were deemed acceptable for more than 90 per cent of patients except for reducing general anaesthesia (84 per cent) and re-sterilization of ‘single-use’ consumables (86 per cent). In phase 4, the top three shortlisted interventions for high-income countries were: introducing recycling; reducing use of anaesthetic gases; and appropriate clinical waste processing. In phase 4, the top three shortlisted interventions for low–middle-income countries were: introducing reusable surgical devices; reducing use of consumables; and reducing the use of general anaesthesia. Conclusion This is a step toward environmentally sustainable operating environments with actionable interventions applicable to both high– and low–middle–income countries