Faithful reproduction of network experiments

The proliferation of cloud computing has compelled the research community to rethink fundamental aspects of network systems and architectures. However, the tools commonly used to evaluate new ideas have not kept abreast of the latest developments. Common simulation and emulation frameworks fail to provide scalability, fidelity, reproducibility and execute unmodified code, all at the same time. We present SELENA, a Xen-based network emulation framework that offers fully reproducible experiments via its automation interface and supports the use of unmodified guest operating systems. This allows out-of-the-box compatibility with common applications and OS components, such as network stacks and filesystems. In order to faithfully emulate faster and larger networks, SELENA adopts the technique of time-dilation and transparently slows down the passage of time for guest operating systems. This technique effectively virtualizes the availability of host’s hardware resources and allows the replication of scenarios with increased I/O and computational demands. Users can directly control the tradeoff between fidelity and running-times via intuitive tuning knobs. We evaluate the ability of SELENA to faithfully replicate the behaviour of real systems and compare it against existing popular experimentation platforms. Our results suggest that SELENA can accurately model networks with aggregate link speeds of 44 Gbps or more, while improving by four times the execution time in comparison to ns3 and exhibits near-linear scaling properties.This is the author accepted manuscript. The final version is available from ACM via http://dx.doi.org/10.1145/2658260.265827

Revisiting the Hierarchical Multiscale LSTM

Hierarchical Multiscale LSTM (Chung et al., 2016a) is a state-of-the-art language model that learns interpretable structure from character-level input. Such models can provide fertile ground for (cognitive) computational linguistics studies. However, the high complexity of the architecture, training procedure and implementations might hinder its applicability. We provide a detailed reproduction and ablation study of the architecture, shedding light on some of the potential caveats of re-purposing complex deep-learning architectures. We further show that simplifying certain aspects of the architecture can in fact improve its performance. We also investigate the linguistic units (segments) learned by various levels of the model, and argue that their quality does not correlate with the overall performance of the model on language modeling.Comment: To appear in COLING 2018 (reproduction track

Developing reproducible and comprehensible computational models

Quantitative predictions for complex scientific theories are often obtained by running simulations on computational models. In order for a theory to meet with wide-spread acceptance, it is important that the model be reproducible and comprehensible by independent researchers. However, the complexity of computational models can make the task of replication all but impossible. Previous authors have suggested that computer models should be developed using high-level specification languages or large amounts of documentation. We argue that neither suggestion is sufficient, as each deals with the prescriptive definition of the model, and does not aid in generalising the use of the model to new contexts. Instead, we argue that a computational model should be released as three components: (a) a well-documented implementation; (b) a set of tests illustrating each of the key processes within the model; and (c) a set of canonical results, for reproducing the model’s predictions in important experiments. The included tests and experiments would provide the concrete exemplars required for easier comprehension of the model, as well as a confirmation that independent implementations and later versions reproduce the theory’s canonical results

Accurate robot simulation through system identification

Robot simulators are useful tools for developing robot behaviours. They provide a fast and efficient means to test robot control code at the convenience of the office desk. In all but the simplest cases though, due to the complexities of the physical systems modelled in the simulator, there are considerable differences between the behaviour of the robot in the simulator and that in the real world environment. In this paper we present a novel method to create a robot simulator using real sensor data. Logged sensor data is used to construct a mathematically explicit model(in the form of a NARMAX polynomial) of the robot’s environment. The advantage of such a transparent model — in contrast to opaque modelling methods such as artificial neural networks — is that it can be analysed to characterise the modelled system, using established mathematical methods In this paper we compare the behaviour of the robot running a particular task in both the simulator and the real-world using qualitative and quantitative measures including statistical methods to investigate the faithfulness of the simulator

Zipf's Law in Gene Expression

Using data from gene expression databases on various organisms and tissues, including yeast, nematodes, human normal and cancer tissues, and embryonic stem cells, we found that the abundances of expressed genes exhibit a power-law distribution with an exponent close to -1, i.e., they obey Zipf's law. Furthermore, by simulations of a simple model with an intra-cellular reaction network, we found that Zipf's law of chemical abundance is a universal feature of cells where such a network optimizes the efficiency and faithfulness of self-reproduction. These findings provide novel insights into the nature of the organization of reaction dynamics in living cells.Comment: revtex, 11 pages, 3 figures, submitted to Phys. Rev. Let