Formalizing graphical notations

The thesis describes research into graphical notations for software engineering, with a principal interest in ways of formalizing them. The research seeks to provide a theoretical basis that will help in designing both notations and the software tools that process them. The work starts from a survey of literature on notation, followed by a review of techniques for formal description and for computational handling of notations. The survey concentrates on collecting views of the benefits and the problems attending notation use in software development; the review covers picture description languages, grammars and tools such as generic editors and visual programming environments. The main problem of notation is found to be a lack of any coherent, rigorous description methods. The current approaches to this problem are analysed as lacking in consensus on syntax specification and also lacking a clear focus on a defined concept of notated expression. To address these deficiencies, the thesis embarks upon an exploration of serniotic, linguistic and logical theory; this culminates in a proposed formalization of serniosis in notations, using categorial model theory as a mathematical foundation. An argument about the structure of sign systems leads to an analysis of notation into a layered system of tractable theories, spanning the gap between expressive pictorial medium and subject domain. This notion of 'tectonic' theory aims to treat both diagrams and formulae together. The research gives details of how syntactic structure can be sketched in a mathematical sense, with examples applying to software development diagrams, offering a new solution to the problem of notation specification. Based on these methods, the thesis discusses directions for resolving the harder problems of supporting notation design, processing and computer-aided generic editing. A number of future research areas are thereby opened up. For practical trial of the ideas, the work proceeds to the development and partial implementation of a system to aid the design of notations and editors. Finally the thesis is evaluated as a contribution to theory in an area which has not attracted a standard approach

Paradoxes of interactivity: perspectives for media theory, human-computer interaction, and artistic investigations

Current findings from anthropology, genetics, prehistory, cognitive and neuroscience indicate that human nature is grounded in a co-evolution of tool use, symbolic communication, social interaction and cultural transmission. Digital information technology has recently entered as a new tool in this co-evolution, and will probably have the strongest impact on shaping the human mind in the near future. A common effort from the humanities, the sciences, art and technology is necessary to understand this ongoing co- evolutionary process. Interactivity is a key for understanding the new relationships formed by humans with social robots as well as interactive environments and wearables underlying this process. Of special importance for understanding interactivity are human-computer and human-robot interaction, as well as media theory and New Media Art. "Paradoxes of Interactivity" brings together reflections on "interactivity" from different theoretical perspectives, the interplay of science and art, and recent technological developments for artistic applications, especially in the realm of sound

Paradoxes of Interactivity

Current findings from anthropology, genetics, prehistory, cognitive and neuroscience indicate that human nature is grounded in a co-evolution of tool use, symbolic communication, social interaction and cultural transmission. Digital information technology has recently entered as a new tool in this co-evolution, and will probably have the strongest impact on shaping the human mind in the near future. A common effort from the humanities, the sciences, art and technology is necessary to understand this ongoing co- evolutionary process. Interactivity is a key for understanding the new relationships formed by humans with social robots as well as interactive environments and wearables underlying this process. Of special importance for understanding interactivity are human-computer and human-robot interaction, as well as media theory and New Media Art. »Paradoxes of Interactivity« brings together reflections on »interactivity« from different theoretical perspectives, the interplay of science and art, and recent technological developments for artistic applications, especially in the realm of sound

Thought, Sign and Machine - the Idea of the Computer Reconsidered

Throughout what is now the more than 50-year history of the computer many theories have been advanced regarding the contribution this machine would make to changes both in the structure of society and in ways of thinking. Like other theories regarding the future, these should also be taken with a pinch of salt. The history of the development of computer technology contains many predictions which have failed to come true and many applications that have not been foreseen. While we must reserve judgment as to the question of the impact on the structure of society and human thought, there is no reason to wait for history when it comes to the question: what are the properties that could give the computer such far-reaching importance? The present book is intended as an answer to this question. The fact that this is a theoretical analysis is due to the nature of the subject. No other possibilities are available because such a description of the properties of the computer must be valid for any kind of application. An additional demand is that the description should be capable of providing an account of the properties which permit and limit these possible applications, just as it must make it possible to characterize a computer as distinct from a) other machines whether clocks, steam engines, thermostats, or mechanical and automatic calculating machines, b) other symbolic media whether printed, mechanical, or electronic and c) other symbolic languages whether ordinary languages, spoken or written or formal languages. This triple limitation, however, (with regard to other machines, symbolic media and symbolic languages) raises a theoretical question as it implies a meeting between concepts of mechanical-deterministic systems, which stem from mathematical physics, and concepts of symbolic systems which stem from the description of symbolic activities common to the humanities. The relationship between science and the humanities has traditionally been seen from a dualistic perspective, as a relationship between two clearly separate subject areas, each studied on its own set of premises and using its own methods. In the present case, however, this perspective cannot be maintained since there is both a common subject area and a new - and specific - kind of interaction between physical and symbolic processes

Computational externalism: The semantic picture of implementation.

The property of being the realization of a computational structure has been argued to be observer-relative. After contrasting the problematic individuation of states in computational systems with the unproblematic individuation of states in dynamical systems, a general diagnosis of the problem is put forward. It is argued that the unwanted proliferation of models for the relation of implementation cannot be blocked unless the labelling scheme is restricted to semantically evaluated items. The instantiation of mathematical dynamical systems, by contrast, is showed to be immune to analogous skeptical arguments due to the virtuous role of measurements in grounding the relevant abstractions. Naturalized semantic properties are proposed to serve as a surrogate for measurements in grounding the relevant abstractions from the physical to the computational level of description, thus making implementations objective. It is argued that a view of implementation that abandons the pervasive internalist view in favor of a view of implementation according to which inputs and outputs are individuated by their broad semantic properties allows us to accept the validity of observer-relativity arguments while preserving the satisfaction of the desiderata of a theory of implementation, as well as the explanatory power of computational- ism as a theory of the mind. The general idea is that of incorporating teleological theories of intentionality within the foundational heart of the notion of computation. An important corollary is that computational properties must be understood as broadly instantiated by relational properties of the implementing system and of its environment. The proposed understanding of implementation is then tested against a number of recalcitrant problems of computationalism. It is argued to be immune to standard objections

Chomskyan (R)evolutions

It is not unusual for contemporary linguists to claim that “Modern Linguistics began in 1957” (with the publication of Noam Chomsky’s Syntactic Structures). Some of the essays in Chomskyan (R)evolutions examine the sources, the nature and the extent of the theoretical changes Chomsky introduced in the 1950s. Other contributions explore the key concepts and disciplinary alliances that have evolved considerably over the past sixty years, such as the meanings given for “Universal Grammar”, the relationship of Chomskyan linguistics to other disciplines (Cognitive Science, Psychology, Evolutionary Biology), and the interactions between mainstream Chomskyan linguistics and other linguistic theories active in the late 20th century: Functionalism, Generative Semantics and Relational Grammar. The broad understanding of the recent history of linguistics points the way towards new directions and methods that linguistics can pursue in the future