Towards Anchoring Self-Learned Representations to Those of Other Agents

In the future, robots will support humans in their every day activities. One particular challenge that robots will face is understanding and reasoning about the actions of other agents in order to cooperate effectively with humans. We propose to tackle this using a developmental framework, where the robot incrementally acquires knowledge, and in particular 1) self-learns a mapping between motor commands and sensory consequences, 2) rapidly acquires primitives and complex actions by verbal descriptions and instructions from a human partner, 3) discovers correspondences between the robots body and other articulated objects and agents, and 4) employs these correspondences to transfer the knowledge acquired from the robots point of view to the viewpoint of the other agent. We show that our approach requires very little a-priori knowledge to achieve imitation learning, to find correspondent body parts of humans, and allows taking the perspective of another agent. This represents a step towards the emergence of a mirror neuron like system based on self-learned representations

Lifelong Augmentation of Multi-Modal Streaming Autobiographical Memories

Robot systems that interact with humans over extended periods of time will benefit from storing and recalling large amounts of accumulated sensorimotor and interaction data. We provide a principled framework for the cumulative organisation of streaming autobiographical data so that data can be continuously processed and augmented as the processing and reasoning abilities of the agent develop and further interactions with humans take place. As an example, we show how a kinematic structure learning algorithm reasons a-posteriori about the skeleton of a human hand. A partner can be asked to provide feedback about the augmented memories, which can in turn be supplied to the reasoning processes in order to adapt their parameters. We employ active, multi-modal remembering, so the robot as well as humans can gain insights of both the original and augmented memories. Our framework is capable of storing discrete and continuous data in real-time. The data can cover multiple modalities and several layers of abstraction (e.g. from raw sound signals over sentences to extracted meanings). We show a typical interaction with a human partner using an iCub humanoid robot. The framework is implemented in a platform-independent manner. In particular, we validate its multi platform capabilities using the iCub, Baxter and NAO robots. We also provide an interface to cloud based services, which allow automatic annotation of episodes. Our framework is geared towards the developmental robotics community, as it 1) provides a variety of interfaces for other modules, 2) unifies previous works on autobiographical memory, and 3) is licensed as open source software

DAC-h3: A Proactive Robot Cognitive Architecture to Acquire and Express Knowledge About the World and the Self

This paper introduces a cognitive architecture for a humanoid robot to engage in a proactive, mixed-initiative exploration and manipulation of its environment, where the initiative can originate from both the human and the robot. The framework, based on a biologically-grounded theory of the brain and mind, integrates a reactive interaction engine, a number of state-of-the art perceptual and motor learning algorithms, as well as planning abilities and an autobiographical memory. The architecture as a whole drives the robot behavior to solve the symbol grounding problem, acquire language capabilities, execute goal-oriented behavior, and express a verbal narrative of its own experience in the world. We validate our approach in human-robot interaction experiments with the iCub humanoid robot, showing that the proposed cognitive architecture can be applied in real time within a realistic scenario and that it can be used with naive users

Towards Lifelong Reasoning with Sparse and Compressive Memory Systems

Humans have a remarkable ability to remember information over long time horizons. When reading a book, we build up a compressed representation of the past narrative, such as the characters and events that have built up the story so far. We can do this even if they are separated by thousands of words from the current text, or long stretches of time between readings. During our life, we build up and retain memories that tell us where we live, what we have experienced, and who we are. Adding memory to artificial neural networks has been transformative in machine learning, allowing models to extract structure from temporal data, and more accurately model the future. However the capacity for long-range reasoning in current memory-augmented neural networks is considerably limited, in comparison to humans, despite the access to powerful modern computers. This thesis explores two prominent approaches towards scaling artificial memories to lifelong capacity: sparse access and compressive memory structures. With sparse access, the inspection, retrieval, and updating of only a very small subset of pertinent memory is considered. It is found that sparse memory access is beneficial for learning, allowing for improved data-efficiency and improved generalisation. From a computational perspective - sparsity allows scaling to memories with millions of entities on a simple CPU-based machine. It is shown that memory systems that compress the past to a smaller set of representations reduce redundancy and can speed up the learning of rare classes and improve upon classical data-structures in database systems. Compressive memory architectures are also devised for sequence prediction tasks and are observed to significantly increase the state-of-the-art in modelling natural language

Digital media and storytelling in higher education

Anita Lanszki's book is about storytelling in the digital media environment. The enterprise is both classical in that it explores the nature of storytelling, which is found in all historical periods and human communities, and modern in that it undertakes a broad overview of contemporary digital culture from the perspective of storytelling. The book is also a methodological guide, illustrated with numerous examples, which has emerged organically from the author's many years of teaching experience. Although the title reflects a focus on the use of digital storytelling in various fields of higher education and research, this excellent work can also be used by professionals working in other spheres of education. Whatever our views on the digital space and age may be, we can probably all agree that we are witnessing a democratization of storytelling in our time. The insights in this book are therefore extremely useful for anyone who is interested in how the timeless practice of storytelling is adapting to the new media environment

Technology-enhanced learning and teaching: Narratives of secondary English teachers in Western Australia

This qualitative study explored teacher beliefs, decision-making, and pedagogical practices for using digital technologies in Western Australian secondary English classrooms. Driven by technology and the assumption that technology integration enhances student learning, the secondary school education landscape is undergoing a period of rapid change. While some stakeholders view teachers’ pedagogical practice as integrated with technology-enhanced learning (TEL), the literature identifies that the contrary is often true: technology use and deployment can be uneven and unsystematic. This study aimed to understand these perceptions via three research questions, that explored TEL's effects on secondary English teaching, both before and after the COVID-19 pandemic home learning experience. Six secondary English teachers were recruited via professional networking. Employing a narrative inquiry methodology, semi-structured interview questions were designed to encourage participants to share their pedagogical approaches and experiences with technology. The interview data were analysed using narrative thematic analysis and dominant themes were identified. The analysis found that teacher participants articulated stories of change and adaptation when using TEL, which gave insight into both personal and professional beliefs regarding education technology. Additional findings report that the home-learning experience during the pandemic has opened up possibilities for new digital pedagogies. However, this was counterbalanced by reinforcement of traditional beliefs about behaviour management, student learning, and teacher agency. The study concludes that if the knowledge gained from home learning in 2020 is utilised in positive ways to further integrate digital pedagogies into curricula, all educational stakeholders are likely to benefit. Furthermore, findings may have implications for twenty-first century in-service and pre-service teacher education to ensure that TEL functions as an enhancement to learning in its broadest sociocultural context

Digital life stories: Semi-automatic (auto)biographies within lifelog collections

Our life stories enable us to reflect upon and share our personal histories. Through emerging digital technologies the possibility of collecting life experiences digitally is increasingly feasible; consequently so is the potential to create a digital counterpart to our personal narratives. In this work, lifelogging tools are used to collect digital artifacts continuously and passively throughout our day. These include images, documents, emails and webpages accessed; texts messages and mobile activity. This range of data when brought together is known as a lifelog. Given the complexity, volume and multimodal nature of such collections, it is clear that there are significant challenges to be addressed in order to achieve coherent and meaningful digital narratives of our events from our life histories. This work investigates the construction of personal digital narratives from lifelog collections. It examines the underlying questions, issues and challenges relating to construction of personal digital narratives from lifelogs. Fundamentally, it addresses how to organize and transform data sampled from an individual’s day-to-day activities into a coherent narrative account. This enquiry is enabled by three 20-month long-term lifelogs collected by participants and produces a narrative system which enables the semi-automatic construction of digital stories from lifelog content. Inspired by probative studies conducted into current practices of curation, from which a set of fundamental requirements are established, this solution employs a 2-dimensional spatial framework for storytelling. It delivers integrated support for the structuring of lifelog content and its distillation into storyform through information retrieval approaches. We describe and contribute flexible algorithmic approaches to achieve both. Finally, this research inquiry yields qualitative and quantitative insights into such digital narratives and their generation, composition and construction. The opportunities for such personal narrative accounts to enable recollection, reminiscence and reflection with the collection owners are established and its benefit in sharing past personal experience experiences is outlined. Finally, in a novel investigation with motivated third parties we demonstrate the opportunities such narrative accounts may have beyond the scope of the collection owner in: personal, societal and cultural explorations, artistic endeavours and as a generational heirloom

A lifelogging system supporting multimodal access

Today, technology has progressed to allow us to capture our lives digitally such as taking pictures, recording videos and gaining access to WiFi to share experiences using smartphones. People’s lifestyles are changing. One example is from the traditional memo writing to the digital lifelog. Lifelogging is the process of using digital tools to collect personal data in order to illustrate the user’s daily life (Smith et al., 2011). The availability of smartphones embedded with different sensors such as camera and GPS has encouraged the development of lifelogging. It also has brought new challenges in multi-sensor data collection, large volume data storage, data analysis and appropriate representation of lifelog data across different devices. This study is designed to address the above challenges. A lifelogging system was developed to collect, store, analyse, and display multiple sensors’ data, i.e. supporting multimodal access. In this system, the multi-sensor data (also called data streams) is firstly transmitted from smartphone to server only when the phone is being charged. On the server side, six contexts are detected namely personal, time, location, social, activity and environment. Events are then segmented and a related narrative is generated. Finally, lifelog data is presented differently on three widely used devices which are the computer, smartphone and E-book reader. Lifelogging is likely to become a well-accepted technology in the coming years. Manual logging is not possible for most people and is not feasible in the long-term. Automatic lifelogging is needed. This study presents a lifelogging system which can automatically collect multi-sensor data, detect contexts, segment events, generate meaningful narratives and display the appropriate data on different devices based on their unique characteristics. The work in this thesis therefore contributes to automatic lifelogging development and in doing so makes a valuable contribution to the development of the field

A privacy-aware and secure system for human memory augmentation

The ubiquity of digital sensors embedded in today's mobile and wearable devices (e.g., smartphones, wearable cameras, wristbands) has made technology more intertwined with our life. Among many other things, this allows us to seamlessly log our daily experiences in increasing numbers and quality, a process known as ``lifelogging''. This practice produces a great amount of pictures and videos that can potentially improve human memory. Consider how a single photograph can bring back distant childhood memories, or how a song can help us reminisce about our last vacation. Such a vision of a ``memory augmentation system'' can offer considerable benefits, but it also raises new security and privacy challenges. Maybe obviously, a system that captures everywhere we go, and everything we say, see, and do, is greatly increasing the danger to our privacy. Any data breach of such a memory repository, whether accidental or malicious, could negatively impact both our professional and private reputation. In addition, the threat of memory manipulation might be the most worrisome aspect of a memory augmentation system: if an attacker is able to remove, add, or change our captured information, the resulting data may implant memories in our heads that never took place, or, in turn, accelerate the loss of other memories. Starting from such key challenges, this thesis investigates how to design secure memory augmentation systems. In the course of this research, we develop tools and prototypes that can be applied by researchers and system engineers to develop pervasive applications that help users capture and later recall episodic memories in a secure fashion. We build trusted sensors and protocols to securely capture and store experience data, and secure software for the secure and privacy-aware exchange of experience data with others. We explore the suitability of various access control models to put users in control of the plethora of data that the system captures on their behalf. We also explore the possibility of using in situ physical gestures to control different aspects regarding the capturing and sharing of experience data. Ultimately, this thesis contributes to the design and development of secure systems for memory augmentation