R-Charon, a Modeling Language for Reconfigurable Hybrid Systems

This paper describes the modeling language as an extension for architectural reconfiguration to the existing distributed hybrid system modeling language Charon. The target application domain of R-Charon includes but is not limited to modular reconfigurable robots and large-scale transportation systems. While largely leaving the Charon syntax and semantics intact, R-Charon allows dynamic creation and destruction of components (agents) as well as of links (references) between the agents. As such, R-Charon is the first formal, hybrid automata based modeling language which also addresses dynamic reconfiguration. We develop and present the syntax and operational semantics for R-Charon on three levels: behavior (modes), structure (agents) and configuration (system)

Asynchronous adaptive time step in quantitative cellular automata modeling

BACKGROUND: The behaviors of cells in metazoans are context dependent, thus large-scale multi-cellular modeling is often necessary, for which cellular automata are natural candidates. Two related issues are involved in cellular automata based multi-cellular modeling: how to introduce differential equation based quantitative computing to precisely describe cellular activity, and upon it, how to solve the heavy time consumption issue in simulation. RESULTS: Based on a modified, language based cellular automata system we extended that allows ordinary differential equations in models, we introduce a method implementing asynchronous adaptive time step in simulation that can considerably improve efficiency yet without a significant sacrifice of accuracy. An average speedup rate of 4–5 is achieved in the given example. CONCLUSIONS: Strategies for reducing time consumption in simulation are indispensable for large-scale, quantitative multi-cellular models, because even a small 100 × 100 × 100 tissue slab contains one million cells. Distributed and adaptive time step is a practical solution in cellular automata environment

FORMAL SPECIFICATIONS AND COMMAND MODELING IN SOFTWARE SYSTEMS WITH A COMPLEX COMMAND STRUCTURE

Commands are an important part of large scale industrial software specifications, especially where the specification is separated from its implementation as in open software standards. Commands can be complex because of large numbers of parameters, dependencies among parameters, subtle side effects, and lack of abstraction. We present a formal approach for command modeling and apply it to IBM\u27s Distributed Data Management Architecture (DDM), a complex, large scale specification of data access on remote and heterogeneous IBM systems. Our approach consists of three parts: a declarative, executable command specification language, an incremental specification technique, and automated reasoning tools. The command specification language provides a formal interpretation of the structural (input-output) and behavioral properties (state constraints/change) of commands. To manage the details of complex commands with numerous inter-dependent arguments, a novel incremental specification technique and several tools for incremental definition and browsing are presented. Two forms of automated reasoning are also demonstrated: type checking to ensure well-typed expressions and target system tracing to simulate command execution. Lessons learned from our experience with the DDM are also discussed

Hierarchical Distributed Representations for Statistical Language Modeling

Statistical language models estimate the probability of a word occurring in a given context. The most common language models rely on a discrete enumeration of predictive contexts (e.g., n-grams) and consequently fail to capture and exploit statistical regularities across these contexts. In this paper, we show how to learn hierarchical, distributed representations of word contexts that maximize the predictive value of a statistical language model. The representations are initialized by unsupervised algorithms for linear and nonlinear dimensionality reduction [14], then fed as input into a hierarchical mixture of experts, where each expert is a multinomial distribution over predicted words [12]. While the distributed representations in our model are inspired by the neural probabilistic language model of Bengio et al. [2, 3], our particular architecture enables us to work with significantly larger vocabularies and training corpora. For example, on a large-scale bigram modeling task involving a sixty thousand word vocabulary and a training corpus of three million sentences, we demonstrate consistent improvement over class-based bigram models [10, 13]. We also discuss extensions of our approach to longer multiword contexts

A Critique of the Telecommunications Description Language (TeD)

TeD is an object-oriented description language designed to facilitate the modeling of large scale telecommunication networks, with simulation on parallel and distributed platforms. TeD models are mapped to the Georgia Tech Time Warp engine (GTW) for execution. In this paper we outline the features of TeD, pointing out its strengths and identifying characteristics that gave us trouble as we used TeD to model detailed networks. Our issues are motivated specifically by a model of TCP and a model of multicast resource allocation. Our intention is to illustrate by example what TeD can do, and characteristics that a potential TeD user should be aware of

Test-Time Training on Nearest Neighbors for Large Language Models

Many recent efforts aim to augment language models with relevant information retrieved from a database at test time. We avoid the need for prompt engineering by directly fine-tuning the model on data retrieved at test time using its standard training setup. For this purpose, we build a large-scale distributed nearest neighbor index based on text embeddings of the Pile dataset. Given a query to a language model, our system retrieves the neighbors of the query and fine-tunes the model on the text data corresponding to those neighbors. Surprisingly, retrieving and training on as few as 20 neighbors, each for only one gradient iteration, drastically improves performance across more than twenty language modeling tasks in the Pile benchmark. For example, test-time training significantly narrows the performance gap between a small GPT2 model and a GPTNeo model, more than ten times larger, that was specifically trained to convergence on the Pile. Sufficient index quality and size, however, are important. Our work establishes a valuable first baseline for implementing test-time training in the context of large language models, opening the door to numerous promising research avenues.Comment: https://github.com/socialfoundations/tttl

A ProActive Backend for ABS: from Modelling to Deployment

ABS is an object-oriented modeling language that is based on a concurrent object group model, derived itself from the active object model. Its goal is to describe distributed and concurrent applications in order to verify their properties and make them safer. Thanks to the ABS Tool Suite, ABS programs can be translated into the Java programming language (among others), and executed in the JVM. This paper presents a new ABS backend that translates ABS programs into ProActive programs. ProActive is a well known active object Java library that provides support for distribution of applications across clusters or grids. The benefit of this work is to be able to easily distribute ABS programs, so that ABS models can also be experimented in a large scale setting. Our contribution includes the ProActive backend itself, the complete description of our translation strategy, and a realistic experiment that shows the benefits of the ProActive backend