Modelling Knowledge Systems using Relation Nets and Hypernets

In the book "Knowledge Representation and Relation Nets" we introduced a structural model called a Relation Net and a special relation net called a Concept Relationship Knowledge Structure (CRKS). In this work we broaden the notion of a relation net to produce a new but associated structural model, a Hypernet. We show that the general theory of hypernets has applications in the acquisition/learning, representation, retrieval, accommodation and assimilation, management and communication/teaching of knowledge, and also in problem representation and solution and in modelling the various modes of reasoning. This report is a revised and extended version of Relation Nets and Hypernets, Technical Report TR-01-020, Department of Mathematics and Computer Science , University of Mannheim, 2001

Relation nets and hypernets

In many respects this report is a companion work of our book "Knowledge representation and relation nets. Kluwer Academic Publishers, Boston, 1999". In some senses it runs parallel to it, while in others it is a sequel to that book. Readers not familiar with the book will find themselves refering back to it several instances to follow some of the subtleties of this work, particularly in the case of concept-relationship knowledge stuctures, abbreviated CRKS in what follows. The main application of CRKS's - namely modelling study material - is not explicitly transscribed to this paper, but that whole notion is abstracted and made independent of any specific teaching/learning metalanguage through the implications of this abstraction. Two key factors emerge from this paper on hypernets. First, unlike the case for CRKS's in which little of the general theory of relation nets applies to CRKS's, the broad theory of hypernets, as far as it is covered in this report, is often applicable to the hypernet equivalent of a CRKS. Second, we will show a link between relation net isomorphism and hypernet isomorphism which makes it considerably easier to deal with CRKS isomorphism and, thus, with structural analogy as used in a modelling based approach to teaching/learning/analogical reasoning. Finally, we must mention that it appears that the domain of potential practical applications of hypernets must inevitably be wider than that for relation nets

A formal representation of the method of learning

We show how the method of learning in science, education, and mathematics can be represented by a knowledge hypernet and its concept-relationship knowledge structure interpretations. We conclude that the method of learning is invariant over the three fields. The paper is particularly suited for teachers of science, particularly physics, and of mathematics, and in the philosophy of science, but is also relevant for educators at every level of instruction. Those working in the fields of cognitive science and knowledge representation can also benefit from this paper and its main references

Recent advances in petri nets and concurrency

CEUR Workshop Proceeding

SMASH: One-Shot Model Architecture Search through HyperNetworks

Designing architectures for deep neural networks requires expert knowledge and substantial computation time. We propose a technique to accelerate architecture selection by learning an auxiliary HyperNet that generates the weights of a main model conditioned on that model's architecture. By comparing the relative validation performance of networks with HyperNet-generated weights, we can effectively search over a wide range of architectures at the cost of a single training run. To facilitate this search, we develop a flexible mechanism based on memory read-writes that allows us to define a wide range of network connectivity patterns, with ResNet, DenseNet, and FractalNet blocks as special cases. We validate our method (SMASH) on CIFAR-10 and CIFAR-100, STL-10, ModelNet10, and Imagenet32x32, achieving competitive performance with similarly-sized hand-designed networks. Our code is available at https://github.com/ajbrock/SMAS

Hypernet semantics of programming languages

Comparison is common practice in programming, even regarding a single programming language. One would ask if two programs behave the same, if one program runs faster than another, or if one run-time system produces the outcome of a program faster than another system. To answer these questions, it is essential to have a formal specification of program execution, with measures such as result and resource usage. This thesis proposes a semantical framework based on abstract machines that enables analysis of program execution cost and direct proof of program equivalence. These abstract machines are inspired by Girard’s Geometry of Interaction, and model program execution as dynamic rewriting of graph representation of a program, guided and controlled by a dedicated object (token) of the graph. The graph representation yields fine control over resource usage, and moreover, the concept of locality in analysing program execution. As a result, this framework enjoys novel flexibility, with which various evaluation strategies and language features, whether they are effects or not, can be modelled and analysed in a uniform way

AI alignment and generalization in deep learning

This thesis covers a number of works in deep learning aimed at understanding and improving generalization abilities of deep neural networks (DNNs). DNNs achieve unrivaled performance in a growing range of tasks and domains, yet their behavior during learning and deployment remains poorly understood. They can also be surprisingly brittle: in-distribution generalization can be a poor predictor of behavior or performance under distributional shifts, which typically cannot be avoided in practice. While these limitations are not unique to DNNs -- and indeed are likely to be challenges facing any AI systems of sufficient complexity -- the prevalence and power of DNNs makes them particularly worthy of study. I frame these challenges within the broader context of "AI Alignment": a nascent field focused on ensuring that AI systems behave in accordance with their user's intentions. While making AI systems more intelligent or capable can help make them more aligned, it is neither necessary nor sufficient for alignment. However, being able to align state-of-the-art AI systems (e.g. DNNs) is of great social importance in order to avoid undesirable and unsafe behavior from advanced AI systems. Without progress in AI Alignment, advanced AI systems might pursue objectives at odds with human survival, posing an existential risk (``x-risk'') to humanity. A core tenet of this thesis is that the achieving high performance on machine learning benchmarks if often a good indicator of AI systems' capabilities, but not their alignment. This is because AI systems often achieve high performance in unexpected ways that reveal the limitations of our performance metrics, and more generally, our techniques for specifying our intentions. Learning about human intentions using DNNs shows some promise, but DNNs are still prone to learning to solve tasks using concepts of "features" very different from those which are salient to humans. Indeed, this is a major source of their poor generalization on out-of-distribution data. By better understanding the successes and failures of DNN generalization and current methods of specifying our intentions, we aim to make progress towards deep-learning based AI systems that are able to understand users' intentions and act accordingly.Cette thèse discute quelques travaux en apprentissage profond visant à comprendre et à améliorer les capacités de généralisation des réseaux de neurones profonds (DNN). Les DNNs atteignent des performances inégalées dans un éventail croissant de tâches et de domaines, mais leur comportement pendant l'apprentissage et le déploiement reste mal compris. Ils peuvent également être étonnamment fragiles: la généralisation dans la distribution peut être un mauvais prédicteur du comportement ou de la performance lors de changements de distribution, ce qui ne peut généralement pas être évité dans la pratique. Bien que ces limitations ne soient pas propres aux DNN - et sont en effet susceptibles de constituer des défis pour tout système d'IA suffisamment complexe - la prévalence et la puissance des DNN les rendent particulièrement dignes d'étude. J'encadre ces défis dans le contexte plus large de «l'alignement de l'IA»: un domaine naissant axé sur la garantie que les systèmes d'IA se comportent conformément aux intentions de leurs utilisateurs. Bien que rendre les systèmes d'IA plus intelligents ou capables puisse aider à les rendre plus alignés, cela n'est ni nécessaire ni suffisant pour l'alignement. Cependant, être capable d'aligner les systèmes d'IA de pointe (par exemple les DNN) est d'une grande importance sociale afin d'éviter les comportements indésirables et dangereux des systèmes d'IA avancés. Sans progrès dans l'alignement de l'IA, les systèmes d'IA avancés pourraient poursuivre des objectifs contraires à la survie humaine, posant un risque existentiel («x-risque») pour l'humanité. L'un des principes fondamentaux de cette thèse est que l'obtention de hautes performances sur les repères d'apprentissage automatique est souvent un bon indicateur des capacités des systèmes d'IA, mais pas de leur alignement. En effet, les systèmes d'IA atteignent souvent des performances élevées de manière inattendue, ce qui révèle les limites de nos mesures de performance et, plus généralement, de nos techniques pour spécifier nos intentions. L'apprentissage des intentions humaines à l'aide des DNN est quelque peu prometteur, mais les DNN sont toujours enclins à apprendre à résoudre des tâches en utilisant des concepts de «caractéristiques» très différents de ceux qui sont saillants pour les humains. En effet, c'est une source majeure de leur mauvaise généralisation sur les données hors distribution. En comprenant mieux les succès et les échecs de la généralisation DNN et les méthodes actuelles de spécification de nos intentions, nous visons à progresser vers des systèmes d'IA basés sur l'apprentissage en profondeur qui sont capables de comprendre les intentions des utilisateurs et d'agir en conséquence

A HyperNet Architecture

Network virtualization is becoming a fundamental building block of future Internet architectures. By adding networking resources into the “cloud”, it is possible for users to rent virtual routers from the underlying network infrastructure, connect them with virtual channels to form a virtual network, and tailor the virtual network (e.g., load application-specific networking protocols, libraries and software stacks on to the virtual routers) to carry out a specific task. In addition, network virtualization technology allows such special-purpose virtual networks to co-exist on the same set of network infrastructure without interfering with each other. Although the underlying network resources needed to support virtualized networks are rapidly becoming available, constructing a virtual network from the ground up and using the network is a challenging and labor-intensive task, one best left to experts. To tackle this problem, we introduce the concept of a HyperNet, a pre-built, pre-configured network package that a user can easily deploy or access a virtual network to carry out a specific task (e.g., multicast video conferencing). HyperNets package together the network topology configuration, software, and network services needed to create and deploy a custom virtual network. Users download HyperNets from HyperNet repositories and then “run” them on virtualized network infrastructure much like users download and run virtual appliances on a virtual machine. To support the HyperNet abstraction, we created a Network Hypervisor service that provides a set of APIs that can be called to create a virtual network with certain characteristics. To evaluate the HyperNet architecture, we implemented several example Hyper-Nets and ran them on our prototype implementation of the Network Hypervisor. Our experiments show that the Hypervisor API can be used to compose almost any special-purpose network – networks capable of carrying out functions that the current Internet does not provide. Moreover, the design of our HyperNet architecture is highly extensible, enabling developers to write high-level libraries (using the Network Hypervisor APIs) to achieve complicated tasks

Using Interval Constrained Petri Nets and Fuzzy Method for Regulation of Quality: The Case of Weight in Tobacco Factory

The existence of maximal durations drastically modifies the performance evaluation in Discrete Event Systems (DES). The same particularity may be found on systems where the associated constraints do not concern the time. For example weight measures, in chemical industry, are used in order to control the quantity of consumed raw materials. This parameter also takes a fundamental part in the product quality as the correct transformation process is based upon a given percentage of each essence. Weight regulation therefore increases the global productivity of the system by decreasing the quantity of rejected products. In this paper we present an approach based on mixing different characteristics theories, the fuzzy system and Petri net system to describe the behaviour. An industriel application on a tobacco manufacturing plant, where the critical parameter is the weight is presented as an illustration