Knowledge Representation for Robots through Human-Robot Interaction

The representation of the knowledge needed by a robot to perform complex tasks is restricted by the limitations of perception. One possible way of overcoming this situation and designing "knowledgeable" robots is to rely on the interaction with the user. We propose a multi-modal interaction framework that allows to effectively acquire knowledge about the environment where the robot operates. In particular, in this paper we present a rich representation framework that can be automatically built from the metric map annotated with the indications provided by the user. Such a representation, allows then the robot to ground complex referential expressions for motion commands and to devise topological navigation plans to achieve the target locations.Comment: Knowledge Representation and Reasoning in Robotics Workshop at ICLP 201

FastDeepIoT: Towards Understanding and Optimizing Neural Network Execution Time on Mobile and Embedded Devices

Deep neural networks show great potential as solutions to many sensing application problems, but their excessive resource demand slows down execution time, pausing a serious impediment to deployment on low-end devices. To address this challenge, recent literature focused on compressing neural network size to improve performance. We show that changing neural network size does not proportionally affect performance attributes of interest, such as execution time. Rather, extreme run-time nonlinearities exist over the network configuration space. Hence, we propose a novel framework, called FastDeepIoT, that uncovers the non-linear relation between neural network structure and execution time, then exploits that understanding to find network configurations that significantly improve the trade-off between execution time and accuracy on mobile and embedded devices. FastDeepIoT makes two key contributions. First, FastDeepIoT automatically learns an accurate and highly interpretable execution time model for deep neural networks on the target device. This is done without prior knowledge of either the hardware specifications or the detailed implementation of the used deep learning library. Second, FastDeepIoT informs a compression algorithm how to minimize execution time on the profiled device without impacting accuracy. We evaluate FastDeepIoT using three different sensing-related tasks on two mobile devices: Nexus 5 and Galaxy Nexus. FastDeepIoT further reduces the neural network execution time by $48\%$ to $78\%$ and energy consumption by $37\%$ to $69\%$ compared with the state-of-the-art compression algorithms.Comment: Accepted by SenSys '1

Out there and in here: design for blended scientific inquiry learning

One of the benefits of mobile technologies is to combine ‘the digital’ (e.g., data, information, photos) with ‘field’ experiences in novel ways that are contextualized by people’s current located activities. However, often cost, mobility disabilities and time exclude students from engaging in such peripatetic experiences. The Out There and In Here project, is exploring a combination of mobile and tabletop technologies in support for collaborative learning. A system is being developed for synchronous collaboration between geology students in the field and peers at an indoor location. The overarching goal of this research is to develop technologies that support people working together in a suitable manner for their locations. There are two OTIH project research threads. The first deals with disabled learner access issues: these complex issues are being reviewed in subsequent evaluations and publications. This paper will deal with issues of technology supported learning design for remote and co-located science learners. Several stakeholder evaluations and two field trials have reviewed two research questions: 1. What will enhance the learning experience for those in the field and laboratory? 2. How can learning trajectories and appropriate technologies be designed to support equitable co-located and remote learning collaboration? This paper focuses on describing the iterative linked development of technologies and scientific inquiry pedagogy. Two stages within the research project are presented. The 1st stage details several pilot studies over 3 years with 21 student participants in synchronous collaborations with traditional technology and pedagogical models. Findings revealed that this was an engaging and useful experience although issues of equity in collaboration needed further research. The 2nd stage, in this project, has been to evaluate data from over 25 stakeholders (academics, learning and technology designers) to develop pervasive ambient technological solutions supporting orchestration of mixed levels of pedagogy (i.e. abstract synthesis to specific investigation). Middleware between tabletop ‘surface’ technologies and mobile devices are being designed with Microsoft and OOKL (a mobile software company) to support these developments. Initial findings reveal issues around equity, ownership and professional identity

Modelling of building interiors with mobile phone sensor data

Creating as-built plans of building interiors is a challenging task. In this paper we present a semi-automatic modelling system for creating residential building interior plans and their integration with existing map data to produce building models. Taking a set of imprecise measurements made with an interactive mobile phone room mapping application, the system performs spatial adjustments in accordance with soft and hard constraints imposed on the building plan geometry. The approach uses an optimisation model that exploits a high accuracy building outline, such as can be found in topographic map data, and the building topology to improve the quality of interior measurements and generate a standardised output. We test our system on building plans of five residential homes. Our evaluation shows that the approach enables construction of accurate interior plans from imprecise measurements. The experiments report an average accuracy of 0.24 m, close to the 0.20 m recommended by the CityGML LoD4 specificatio

Shared visiting in Equator city

In this paper we describe an infrastructure and prototype system for sharing of visiting experiences across multiple media. The prototype supports synchronous co-visiting by physical and digital visitors, with digital access via either the World Wide Web or 3-dimensional graphics

Development and Impact of a Mobile Application that Allows Users to Track Their Location on an Educational Institution Campus

This research study aims to solve user location issues within the campus at an educational institution. As this campus comprises a large number of places and departments, users often get confused about how to reach a specific location. To address this problem, the “Ubícate” (“locate by yourself” in Spanish) application was developed following the CDIO methodology, which encompasses four creative process steps: conceive, design, implement, and operate. The “Ubícate” app provides users with information on places of interest such as schools, departments, halls, auditoriums, and sports venues, offering a visual reference of available locations through 360-degree images. The application also uses Google Maps to track user location within the campus, thus marking a reference route between university gates and the different locations available, in addition to providing information on university-sponsored events. In this paper, Section 2 describes the methodology and each of the stages that were addressed in the following sections. Section 3 presents the development itself and the data used for the purposes thereof. Next, Section 4 reveals the results from this study. Later, Section 5 assesses these results and the findings from the study. In Section 6, our conclusions are discussed. Finally, Section 7 lists topics for future research. The application did indeed contribute to improving the attendance of the academic community at events. Where the application was used, the first-hand perception of visitors and their own was very positive and enhanced the institutional image and sense of belonging. The contribution of this study consists of presenting a mobile application as a solution from three approaches: the technical aspects for application development, the business vision to satisfy the user’s needs, and the end user’s perception. All three approaches provide a technical reader, an entrepreneur, or an end user an overview of a scalable solution to different types of implementations in different types of businesses that require indoor location through the use of technologies in mobile applications. The mobile application performs the location indoors using the Google Maps platform, allowing a more agile development in implementing the APP

EXTENDING A MOBILE DEVICE WITH LOW-COST 3D MODELING AND BUILDING-SCALE MAPPING CAPABILITIES, FOR APPLICATION IN ARCHITECTURE AND ARCHAEOLOGY

One of the most challenging problem in architecture is the automated construction of 3D (and 4D) digital models of cultural objects with the aim of implementing open data repositories, scientifically authenticated and responding to well accepted standards of validation, evaluation, preservation, publication, updating and dissemination. The realization of such an ambitious objective requires the adoption of special technological instruments. In this paper we plan to use portable devices (i.e. smartphones, tablets or PDAs eventually extended to wearable ones), extended with a small plug-in, for automatically extracting 3D models of single objects and building-scale mapping of the surrounding environment. At the same time, the device will provide the capability of inserting notes and observations. Where the instrument cannot be directly applied, for example for exploring the top of a complex building, we consider mounting our device, or using equivalent existing equipment, on a drone, in a modular approach for obtaining data de-facto interchangeable. The approach based on the expansion packs has the advantage of anticipating (or even promoting) future extensions of new mobile devices, when the spectrum of possible applications justify the corresponding increased costs. In order to experiment and verify this approach we plan to test it in two specific scenarios of the cultural heritage domain in which such devices seem particularly promising: Strada Nuova in Genoa and Palazzo Ducale in Urbino, both located in Italy

Extending a mobile device with low-cost 3D modeling and building-scale mapping capabilities, for application in architecture and archaeology

One of the most challenging problem in architecture is the automated construction of 3D (and 4D) digital models of cultural objects with the aim of implementing open data repositories, scientifically authenticated and responding to well accepted standards of validation, evaluation, preservation, publication, updating and dissemination. The realization of such an ambitious objective requires the adoption of special technological instruments. In this paper we plan to use portable devices (i.e. smartphones, tablets or PDAs eventually extended to wearable ones), extended with a small plug-in, for automatically extracting 3D models of single objects and building-scale mapping of the surrounding environment. At the same time, the device will provide the capability of inserting notes and observations. Where the instrument cannot be directly applied, for example for exploring the top of a complex building, we consider mounting our device, or using equivalent existing equipment, on a drone, in a modular approach for obtaining data de-facto interchangeable. The approach based on the expansion packs has the advantage of anticipating (or even promoting) future extensions of new mobile devices, when the spectrum of possible applications justify the corresponding increased costs. In order to experiment and verify this approach we plan to test it in two specific scenarios of the cultural heritage domain in which such devices seem particularly promising: Strada Nuova in Genoa and Palazzo Ducale in Urbino, both located in Italy