ScanComplete: Large-Scale Scene Completion and Semantic Segmentation for 3D Scans

We introduce ScanComplete, a novel data-driven approach for taking an incomplete 3D scan of a scene as input and predicting a complete 3D model along with per-voxel semantic labels. The key contribution of our method is its ability to handle large scenes with varying spatial extent, managing the cubic growth in data size as scene size increases. To this end, we devise a fully-convolutional generative 3D CNN model whose filter kernels are invariant to the overall scene size. The model can be trained on scene subvolumes but deployed on arbitrarily large scenes at test time. In addition, we propose a coarse-to-fine inference strategy in order to produce high-resolution output while also leveraging large input context sizes. In an extensive series of experiments, we carefully evaluate different model design choices, considering both deterministic and probabilistic models for completion and semantic inference. Our results show that we outperform other methods not only in the size of the environments handled and processing efficiency, but also with regard to completion quality and semantic segmentation performance by a significant margin.Comment: Video: https://youtu.be/5s5s8iH0NF

Growing the use of Virtual Worlds in education : an OpenSim perspective

The growth in the range of disciplines that Virtual Worlds support for educational purposes is evidenced by recent applications in the fields of cultural heritage, humanitarian aid, space exploration, virtual laboratories in the physical sciences, archaeology, computer science and coastal geography. This growth is due in part to the flexibility of OpenSim, the open source virtual world platform which by adopting Second Life protocols and norms has created a de facto standard for open virtual worlds that is supported by a growing number of third party open source viewers. Yet while this diversity of use-cases is impressive and Virtual Worlds for open learning are highly popular with lecturers and learners alike immersive education remains an essentially niche activity. This paper identifies functional challenges in terms of Management, Network Infrastructure, the Immersive 3D Web and Programmability that must be addressed to enable the wider adoption of Open Virtual Worlds as a routine learning technology platform. We refer to specific use-cases based on OpenSim and abstract generic requirements which should be met to enable the growth in use of Open Virtual Worlds as a mainstream educational facility. A case study of a deployment to support a formal education curriculum and associated informal learning is used to illustrate key points.Postprin

Developing serious games for cultural heritage: a state-of-the-art review

Although the widespread use of gaming for leisure purposes has been well documented, the use of games to support cultural heritage purposes, such as historical teaching and learning, or for enhancing museum visits, has been less well considered. The state-of-the-art in serious game technology is identical to that of the state-of-the-art in entertainment games technology. As a result, the field of serious heritage games concerns itself with recent advances in computer games, real-time computer graphics, virtual and augmented reality and artificial intelligence. On the other hand, the main strengths of serious gaming applications may be generalised as being in the areas of communication, visual expression of information, collaboration mechanisms, interactivity and entertainment. In this report, we will focus on the state-of-the-art with respect to the theories, methods and technologies used in serious heritage games. We provide an overview of existing literature of relevance to the domain, discuss the strengths and weaknesses of the described methods and point out unsolved problems and challenges. In addition, several case studies illustrating the application of methods and technologies used in cultural heritage are presented

Interactive Camera Network Design using a Virtual Reality Interface

Traditional literature on camera network design focuses on constructing automated algorithms. These require problem specific input from experts in order to produce their output. The nature of the required input is highly unintuitive leading to an unpractical workflow for human operators. In this work we focus on developing a virtual reality user interface allowing human operators to manually design camera networks in an intuitive manner. From real world practical examples we conclude that the camera networks designed using this interface are highly competitive with, or superior to those generated by automated algorithms, but the associated workflow is much more intuitive and simple. The competitiveness of the human-generated camera networks is remarkable because the structure of the optimization problem is a well known combinatorial NP-hard problem. These results indicate that human operators can be used in challenging geometrical combinatorial optimization problems given an intuitive visualization of the problem.Comment: 11 pages, 8 figure

Serious Games in Cultural Heritage

Although the widespread use of gaming for leisure purposes has been well documented, the use of games to support cultural heritage purposes, such as historical teaching and learning, or for enhancing museum visits, has been less well considered. The state-of-the-art in serious game technology is identical to that of the state-of-the-art in entertainment games technology. As a result the field of serious heritage games concerns itself with recent advances in computer games, real-time computer graphics, virtual and augmented reality and artificial intelligence. On the other hand, the main strengths of serious gaming applications may be generalised as being in the areas of communication, visual expression of information, collaboration mechanisms, interactivity and entertainment. In this report, we will focus on the state-of-the-art with respect to the theories, methods and technologies used in serious heritage games. We provide an overview of existing literature of relevance to the domain, discuss the strengths and weaknesses of the described methods and point out unsolved problems and challenges. In addition, several case studies illustrating the application of methods and technologies used in cultural heritage are presented

Tracking a table tennis ball for umpiring purposes using a multi-agent system

Tracking a table tennis ball for umpiring purposes is a challenging task as, in real-match scenarios, the ball travels fast and can become occluded or merged with other background objects. This paper presents the design of a multi-view based tracking system that can overcome the challenges of tracking a ball in real match sequences. The system has been tested on a complete table tennis rally and the results are very promising. The system is able to continuously track the ball with only marginal variations in detection. Furthermore, the initialization of the multi-camera system means it is both a portable and cost-effective solution for umpiring purposes

Spectral analysis for long-term robotic mapping

This paper presents a new approach to mobile robot mapping in long-term scenarios. So far, the environment models used in mobile robotics have been tailored to capture static scenes and dealt with the environment changes by means of ‘memory decay’. While these models keep up with slowly changing environments, their utilization in dynamic, real world environments is difficult. The representation proposed in this paper models the environment’s spatio-temporal dynamics by its frequency spectrum. The spectral representation of the time domain allows to identify, analyse and remember regularly occurring environment processes in a computationally efficient way. Knowledge of the periodicity of the different environment processes constitutes the model predictive capabilities, which are especially useful for long-term mobile robotics scenarios. In the experiments presented, the proposed approach is applied to data collected by a mobile robot patrolling an indoor environment over a period of one week. Three scenarios are investigated, including intruder detection and 4D mapping. The results indicate that the proposed method allows to represent arbitrary timescales with constant (and low) memory requirements, achieving compression rates up to 106 . Moreover, the representation allows for prediction of future environment’s state with ∼ 90% precision