Mechanistic Behavior of Single-Pass Instruction Sequences

Earlier work on program and thread algebra detailed the functional, observable behavior of programs under execution. In this article we add the modeling of unobservable, mechanistic processing, in particular processing due to jump instructions. We model mechanistic processing preceding some further behavior as a delay of that behavior; we borrow a unary delay operator from discrete time process algebra. We define a mechanistic improvement ordering on threads and observe that some threads do not have an optimal implementation.Comment: 12 page

Rapid Visual Categorization is not Guided by Early Salience-Based Selection

The current dominant visual processing paradigm in both human and machine research is the feedforward, layered hierarchy of neural-like processing elements. Within this paradigm, visual saliency is seen by many to have a specific role, namely that of early selection. Early selection is thought to enable very fast visual performance by limiting processing to only the most salient candidate portions of an image. This strategy has led to a plethora of saliency algorithms that have indeed improved processing time efficiency in machine algorithms, which in turn have strengthened the suggestion that human vision also employs a similar early selection strategy. However, at least one set of critical tests of this idea has never been performed with respect to the role of early selection in human vision. How would the best of the current saliency models perform on the stimuli used by experimentalists who first provided evidence for this visual processing paradigm? Would the algorithms really provide correct candidate sub-images to enable fast categorization on those same images? Do humans really need this early selection for their impressive performance? Here, we report on a new series of tests of these questions whose results suggest that it is quite unlikely that such an early selection process has any role in human rapid visual categorization.Comment: 22 pages, 9 figure

Indirect jumps improve instruction sequence performance

Instruction sequences with direct and indirect jump instructions are as expressive as instruction sequences with direct jump instructions only. We show that, in the case where the number of instructions is not bounded, we are faced with increases of the maximal internal delays of instruction sequences on execution that are not bounded by a linear function if we strive for acceptable increases of the lengths of instruction sequences on elimination of indirect jump instructions.Comment: 10 pages, definition of maximal internal delay and theorem 1 are stated more precise; presentation improve

The elemental mechanism of transcriptional pausing.

Transcriptional pausing underlies regulation of cellular RNA biogenesis. A consensus pause sequence that acts on RNA polymerases (RNAPs) from bacteria to mammals halts RNAP in an elemental paused state from which longer-lived pauses can arise. Although the structural foundations of pauses prolonged by backtracking or nascent RNA hairpins are recognized, the fundamental mechanism of the elemental pause is less well-defined. Here we report a mechanistic dissection that establishes the elemental pause signal (i) is multipartite; (ii) causes a modest conformational shift that puts γ-proteobacterial RNAP in an off-pathway state in which template base loading but not RNA translocation is inhibited; and (iii) allows RNAP to enter pretranslocated and one-base-pair backtracked states easily even though the half-translocated state observed in paused cryo-EM structures rate-limits pause escape. Our findings provide a mechanistic basis for the elemental pause and a framework to understand how pausing is modulated by sequence, cellular conditions, and regulators

Exposing Attention Glitches with Flip-Flop Language Modeling

Why do large language models sometimes output factual inaccuracies and exhibit erroneous reasoning? The brittleness of these models, particularly when executing long chains of reasoning, currently seems to be an inevitable price to pay for their advanced capabilities of coherently synthesizing knowledge, pragmatics, and abstract thought. Towards making sense of this fundamentally unsolved problem, this work identifies and analyzes the phenomenon of attention glitches, in which the Transformer architecture's inductive biases intermittently fail to capture robust reasoning. To isolate the issue, we introduce flip-flop language modeling (FFLM), a parametric family of synthetic benchmarks designed to probe the extrapolative behavior of neural language models. This simple generative task requires a model to copy binary symbols over long-range dependencies, ignoring the tokens in between. We find that Transformer FFLMs suffer from a long tail of sporadic reasoning errors, some of which we can eliminate using various regularization techniques. Our preliminary mechanistic analyses show why the remaining errors may be very difficult to diagnose and resolve. We hypothesize that attention glitches account for (some of) the closed-domain hallucinations in natural LLMs.Comment: v2: NeurIPS 2023 camera-ready + data releas

Fast simulation techniques for microprocessor design space exploration

Designing a microprocessor is extremely time-consuming. Computer architects heavily rely on architectural simulators, e.g., to drive high-level design decisions during early stage design space exploration. The benefit of architectural simulators is that they yield relatively accurate performance results, are highly parameterizable and are very flexible to use. The downside, however, is that they are at least three or four orders of magnitude slower than real hardware execution. The current trend towards multicore processors exacerbates the problem; as the number of cores on a multicore processor increases, simulation speed has become a major concern in computer architecture research and development. In this dissertation, we propose and evaluate two simulation techniques that reduce the simulation time significantly: statistical simulation and interval simulation. Statistical simulation speeds up the simulation by reducing the number of dynamically executed instructions. First, we collect a number of program execution characteristics into a statistical profile. From this profile we can generate a synthetic trace that exhibits the same execution behavior but which has a much shorter trace length as compared to the original trace. Simulating this synthetic trace then yields a performance estimate. Interval simulation raises the level of abstraction in architectural simulation; it replaces the core-level cycle-accurate simulation model by a mechanistic analytical model. The analytical model builds on insights from interval analysis: miss events divide the smooth streaming of instructions into so called intervals. The model drives the timing by analyzing the type of the miss events and their latencies, instead of tracking the individual instructions as they propagate through the pipeline stages

About Instruction Sequence Testing

Software testing is presented as a so-called theme within which different authors and groups have defined different subjects each of these subjects having a different focus on testing. A uniform concept of software testing is non-existent and the space of possible coherent perspectives on software testing, each fitting within the theme, is viewed as being spanned by five dimensions, each dimension representing two opposite views with a variety of intermediate views in between. Instruction sequences are used as a simple theoretical conceptualization of computer programs. A theory of instruction sequence testing may serve as a model for a theory of software testing. Instruction sequences testing is considered a new topic for which definitions may be freely contemplated without being restricted by existing views on software testing. The problem of developing a theory of instruction sequence testing is posed. A survey is given of motivations and scenarios for developing a theory of instruction sequence testing