Embodied neuromorphic intelligence

The design of robots that interact autonomously with the environment and exhibit complex behaviours is an open challenge that can benefit from understanding what makes living beings fit to act in the world. Neuromorphic engineering studies neural computational principles to develop technologies that can provide a computing substrate for building compact and low-power processing systems. We discuss why endowing robots with neuromorphic technologies – from perception to motor control – represents a promising approach for the creation of robots which can seamlessly integrate in society. We present initial attempts in this direction, highlight open challenges, and propose actions required to overcome current limitations

GPU Computing for Cognitive Robotics

This thesis presents the first investigation of the impact of GPU computing on cognitive robotics by providing a series of novel experiments in the area of action and language acquisition in humanoid robots and computer vision. Cognitive robotics is concerned with endowing robots with high-level cognitive capabilities to enable the achievement of complex goals in complex environments. Reaching the ultimate goal of developing cognitive robots will require tremendous amounts of computational power, which was until recently provided mostly by standard CPU processors. CPU cores are optimised for serial code execution at the expense of parallel execution, which renders them relatively inefficient when it comes to high-performance computing applications. The ever-increasing market demand for high-performance, real-time 3D graphics has evolved the GPU into a highly parallel, multithreaded, many-core processor extraordinary computational power and very high memory bandwidth. These vast computational resources of modern GPUs can now be used by the most of the cognitive robotics models as they tend to be inherently parallel. Various interesting and insightful cognitive models were developed and addressed important scientific questions concerning action-language acquisition and computer vision. While they have provided us with important scientific insights, their complexity and application has not improved much over the last years. The experimental tasks as well as the scale of these models are often minimised to avoid excessive training times that grow exponentially with the number of neurons and the training data. This impedes further progress and development of complex neurocontrollers that would be able to take the cognitive robotics research a step closer to reaching the ultimate goal of creating intelligent machines. This thesis presents several cases where the application of the GPU computing on cognitive robotics algorithms resulted in the development of large-scale neurocontrollers of previously unseen complexity enabling the conducting of the novel experiments described herein.European Commission Seventh Framework Programm

2022 roadmap on neuromorphic computing and engineering

Modern computation based on von Neumann architecture is now a mature cutting-edge science. In the von Neumann architecture, processing and memory units are implemented as separate blocks interchanging data intensively and continuously. This data transfer is responsible for a large part of the power consumption. The next generation computer technology is expected to solve problems at the exascale with 10$^{18}$ calculations each second. Even though these future computers will be incredibly powerful, if they are based on von Neumann type architectures, they will consume between 20 and 30 megawatts of power and will not have intrinsic physically built-in capabilities to learn or deal with complex data as our brain does. These needs can be addressed by neuromorphic computing systems which are inspired by the biological concepts of the human brain. This new generation of computers has the potential to be used for the storage and processing of large amounts of digital information with much lower power consumption than conventional processors. Among their potential future applications, an important niche is moving the control from data centers to edge devices. The aim of this roadmap is to present a snapshot of the present state of neuromorphic technology and provide an opinion on the challenges and opportunities that the future holds in the major areas of neuromorphic technology, namely materials, devices, neuromorphic circuits, neuromorphic algorithms, applications, and ethics. The roadmap is a collection of perspectives where leading researchers in the neuromorphic community provide their own view about the current state and the future challenges for each research area. We hope that this roadmap will be a useful resource by providing a concise yet comprehensive introduction to readers outside this field, for those who are just entering the field, as well as providing future perspectives for those who are well established in the neuromorphic computing community

Investigation and development of a tangible technology framework for highly complex and abstract concepts

The ubiquitous integration of computer-supported learning tools within the educational domain has led educators to continuously seek effective technological platforms for teaching and learning. Overcoming the inherent limitations of traditional educational approaches, interactive and tangible computing platforms have consequently garnered increased interest in the pursuit of embedding active learning pedagogies within curricula. However, whilst Tangible User Interface (TUI) systems have been successfully developed to edutain children in various research contexts, TUI architectures have seen limited deployment towards more advanced educational pursuits. Thus, in contrast to current domain research, this study investigates the effectiveness and suitability of adopting TUI systems for enhancing the learning experience of abstract and complex computational science and technology-based concepts within higher educational institutions (HEI)s. Based on the proposal of a contextually apt TUI architecture, the research describes the design and development of eight distinct TUI frameworks embodying innovate interactive paradigms through tabletop peripherals, graphical design factors, and active tangible manipulatives. These computationally coupled design elements are evaluated through summative and formative experimental methodologies for their ability to aid in the effective teaching and learning of diverse threshold concepts experienced in computational science. In addition, through the design and adoption of a technology acceptance model for educational technology (TAM4Edu), the suitability of TUI frameworks in HEI education is empirically evaluated across a myriad of determinants for modelling students’ behavioural intention. In light of the statistically significant results obtained in both academic knowledge gain (μ = 25.8%) and student satisfaction (μ = 12.7%), the study outlines the affordances provided through TUI design for various constituents of active learning theories and modalities. Thus, based on an empirical and pedagogical analyses, a set of design guidelines is defined within this research to direct the effective development of TUI design elements for teaching and learning abstract threshold concepts in HEI adaptations

Multisensory instrumental dynamics as an emergent paradigm for digital musical creation

The nature of human/instrument interaction is a long-standing area of study, drawing interest from fields as diverse as philosophy, cognitive sciences, anthropology, human–computer-interaction, and artistic creation. In particular, the case of the interaction between performer and musical instrument provides an enticing framework for studying the instrumental dynamics that allow for embodiment, skill acquisition and virtuosity with (electro-)acoustical instruments, and questioning how such notions may be transferred into the realm of digital music technologies and virtual instruments. This paper offers a study of concepts and technologies allowing for instrumental dynamics with Digital Musical Instruments, through an analysis of haptic-audio creation centred on (a) theoretical and conceptual frameworks, (b) technological components—namely physical modelling techniques for the design of virtual mechanical systems and force-feedback technologies allowing mechanical coupling with them, and (c) a corpus of artistic works based on this approach. Through this retrospective, we argue that artistic works created in this field over the last 20 years—and those yet to come—may be of significant importance to the haptics community as new objects that question physicality, tangibility, and creativity from a fresh and rather singular angle. Following which, we discuss the convergence of efforts in this field, challenges still ahead, and the possible emergence of a new transdisciplinary community focused on multisensory digital art forms

Opinions and Outlooks on Morphological Computation

Morphological Computation is based on the observation that biological systems seem to carry out relevant computations with their morphology (physical body) in order to successfully interact with their environments. This can be observed in a whole range of systems and at many different scales. It has been studied in animals – e.g., while running, the functionality of coping with impact and slight unevenness in the ground is "delivered" by the shape of the legs and the damped elasticity of the muscle-tendon system – and plants, but it has also been observed at the cellular and even at the molecular level – as seen, for example, in spontaneous self-assembly. The concept of morphological computation has served as an inspirational resource to build bio-inspired robots, design novel approaches for support systems in health care, implement computation with natural systems, but also in art and architecture. As a consequence, the field is highly interdisciplinary, which is also nicely reflected in the wide range of authors that are featured in this e-book. We have contributions from robotics, mechanical engineering, health, architecture, biology, philosophy, and others

Growing cybernetic ears: transduction and performativity in the analogue and digital what have you

At a time when digital technologies have become ubiquitous in music making, and where the majority of research into music technology happens at the computational ‘cutting edge’, this practice-based PhD explores analogue technologies deemed, in the main, obsolete, anachronistic, or as quaint nostalgic throwbacks, and asks how a combination of technological, historical and practice-based research, focused through commitment to artistic outputs in the domain of music technology, might shed new light on the terms analogue and digital, and on the nature of the analogue-digital relationship. Underlying much contemporary enthusiasm for ‘the digital’ are progress narratives that rely on both a succession logic (old analogue technology gets replaced by new digital technology) and an assumption of isomorphism (the digital technology does all the same things as the replaced technology, though often with ‘enhanced’ affordances). This thesis questions such assumptions along historical, philosophical and practice-based trajectories. Key to these research trajectories is the trans-discipline cybernetics, in particular the second-order cybernetics of Gordon Pask, whose self-designation ‘philosophical mechanic’ indicates the importance he placed on a cyclical, mutually accommodating thinking-designing-making. Pask presented a powerful practical methodology for the examination and creation of dynamical systems in flux, systems that evolve as a result of participant interaction, systems that can be seen to manifest self-organisation. Second-order cybernetics puts the emphasis on processes in interaction rather than positing pre-existing objects (including concepts) in a world ‘out there’. Cybernetics helps us to explore systems whose complexity and interdependence precludes the separation out into constituent parts, systems where control is shared across multiple mutually interacting dimensions, and where the observer is a committed participant whose actions, interests and biases cannot be divorced from the interactions therein. Two other key concepts are: (1) transduction, which relates energy, information, patterns of growth, or other dynamical processes across media or between domains; (2) performativity, an interventional act that brings forth a world. Transduction is essential to an understanding of recording studio processes and practices: the microphone, signal processing and recording itself all rely on transduction. When viewed from a performative perspective, actions such as recording are found to be carried out very differently when the final stage of transduction is discrete (the case with the now ubiquitous digital audio workstation) or continuous (such as recording to tape). This difference is primarily due to the hyper-plasticity of digital audio, a taking of sound ‘out of time’. Rather than seeing this as an evolution of ‘precursor’ analogue technologies, as most accounts have it, this thesis takes the perspective that this is a difference in kind, rather than one of degree, and explores that difference with a particular focus on emergent and intertwined cultural, embodied and technological systems, rather than on end products. The second half of the thesis presents the compositional practice, ranging from experimental work on tape music composition and installation, through a series of modular synthesis live performances, to tape-based recording of pop music. The physical, gestural engagement with the resistant materiality of these technologies emphasises a very different cognitive engagement with processes of composition and production to that which happens with supposed ‘successor’ digital technologies; assumptions of isomorphism, buttressed by skeuomorphic emulation, tend to occlude this cognitive distinction. This thesis is offered as an act of cybernetic musicking – resolutely practical in orientation, with a wide-ranging, trans-disciplinary theoretical framework, and with the emphasis not on things but on ongoing processes in complex interaction with a world in constant becoming

The External Tape Hypothesis: a Turing machine based approach to cognitive computation

The symbol processing or "classical cognitivist" approach to mental computation suggests that the cognitive architecture operates rather like a digital computer. The components of the architecture are input, output and central systems. The input and output systems communicate with both the internal and external environments of the cognizer and transmit codes to and from the rule governed, central processing system which operates on structured representational expressions in the internal environment. The connectionist approach, by contrast, suggests that the cognitive architecture should be thought of as a network of interconnected neuron-like processing elements (nodes) which operates rather like a brain. Connectionism distinguishes input, output and central or "hidden" layers of nodes. Connectionists claim that internal processing consists not of the rule governed manipulation of structured symbolic expressions, but of the excitation and inhibition of activity and the alteration of connection strengths via message passing within and between layers of nodes in the network. A central claim of the thesis is that neither symbol processing nor connectionism provides an adequate characterization of the role of the external environment in cognitive computation. An alternative approach, called the External Tape Hypothesis (ETH), is developed which claims, on the basis of Turing's analysis of routine computation, that the Turing machine model can be used as the basis for a theory which includes the environment as an essential part of the cognitive architecture. The environment is thought of as the tape, and the brain as the control of a Turing machine. Finite state automata, Turing machines, and universal Turing machines are described, including details of Turing's original universal machine construction. A short account of relevant aspects of the history of digital computation is followed by a critique of the symbol processing approach as it is construed by influential proponents such as Allen Newell and Zenon Pylyshyn among others. The External Tape Hypothesis is then developed as an alternative theoretical basis. In the final chapter, the ETH is combined with the notion of a self-describing Turing machine to provide the basis for an account of thinking and the development of internal representations

An agent-based reinforcement learning approach to improve human-robot-interaction in manufacturing

This work is aimed at the understanding and application of several emerging technologies as they relate to improving the interactions which occur between robotic operators and their human colleagues across a range of manufacturing processes. These interactions are problematic, as variation in performance of human beings remains one of the largest sources of disturbances within such systems, with potentially significant implications for productivity if it continues unmitigated. The problem remains for the most part unaddressed, despite these interactions becoming increasingly prevalent as the rate of adoption of automation technologies increases. By reconciling multiple areas encompassed by the wider domain of intelligent manufacturing, the presented work identifies a methodology and a set of software tools which leverage the strengths of neural-network-based reinforcement learning to develop intelligent software agents capable of adaptable behaviour in response to observed environmental changes. The methodology further focuses on developing representative simulation models for these interactions following a pattern of generalisation, to effectively represent both human and robotic elements, and facilitate implementation. By learning through their interaction with the simulated manufacturing environment, these agents can determine an appropriate policy, by which to autonomously adjust their operating parameters, as a response to changes in their human colleagues. This adaptability is demonstrated to enable the intelligent agents to determine an action policy which results in less observed idle time, along with improved leanness and overall productivity, over multiple scenarios. The findings of the work suggest that software agents that make use of a reinforcement based learning approach are well suited to the task of enabling robotic adaptability in such a way, and the developed methodology provides a platform for further development and exploration, along with numerous insights into the effective development of these agents