Simultaneous monocular 2D segmentation, 3D pose recovery and 3D reconstruction

We propose a novel framework for joint 2D segmentation and 3D pose and 3D shape recovery, for images coming from a single monocular source. In the past, integration of all three has proven difficult, largely because of the high degree of ambiguity in the 2D - 3D mapping. Our solution is to learn nonlinear and probabilistic low dimensional latent spaces, using the Gaussian Process Latent Variable Models dimensionality reduction technique. These act as class or activity constraints to a simultaneous and variational segmentation – recovery – reconstruction process. We define an image and level set based energy function, which we minimise with respect to 3D pose and shape, 2D segmentation resulting automatically as the projection of the recovered shape under the recovered pose. We represent 3D shapes as zero levels of 3D level set embedding functions, which we project down directly to probabilistic 2D occupancy maps, without the requirement of an intermediary explicit contour stage. Finally, we detail a fast, open-source, GPU-based implementation of our algorithm, which we use to produce results on both real and artificial video sequences.Victor Adrian Prisacariu, Aleksandr V. Segal, and Ian Rei

Dense reconstruction using 3D object shape priors

We propose a formulation of monocular SLAM which combines live dense reconstruction with shape priors-based 3D tracking and reconstruction. Current live dense SLAM approaches are limited to the reconstruction of visible surfaces. Moreover, most of them are based on the minimisation of a photo-consistency error, which usually makes them sensitive to specularities. In the 3D pose recovery literature, problems caused by imperfect and ambiguous image information have been dealt with by using prior shape knowledge. At the same time, the success of depth sensors has shown that combining joint image and depth information drastically increases the robustness of the classical monocular 3D tracking and 3D reconstruction approaches. In this work we link dense SLAM to 3D object pose and shape recovery. More specifically, we automatically augment our SLAM system with object specific identity, together with 6D pose and additional shape degrees of freedom for the object(s) of known class in the scene, combining image data and depth information for the pose and shape recovery. This leads to a system that allows for full scaled 3D reconstruction with the known object(s) segmented from the scene. The segmentation enhances the clarity, accuracy and completeness of the maps built by the dense SLAM system, while the dense 3D data aids the segmentation process, yielding faster and more reliable convergence than when using 2D image data alone.Amaury Dame, Victor A. Prisacariu, Carl Y. Ren, Ian Reidhttp://www.pamitc.org/cvpr13

Real-Time Tracking of Single and Multiple Objects from Depth-Colour Imagery Using 3D Signed Distance Functions

We describe a novel probabilistic framework for real-time tracking of multiple objects from combined depthcolour imagery. Object shape is represented implicitly using 3D signed distance functions. Probabilistic generative models based on these functions are developed to account for the observed RGB-D imagery, and tracking is posed as a maximum a posteriori problem. We present first a method suited to tracking a single rigid 3D object, and then generalise this to multiple objects by combining distance functions into a shape union in the frame of the camera. This second model accounts for similarity and proximity between objects, and leads to robust real-time tracking without recourse to bolt-on or ad-hoc collision detection

Integrating Social Assistive Robots, IoT, Virtual Communities and Smart Objects to Assist at-Home Independently Living Elders: the MoveCare Project

The integration of Ambient Assisted Living (AAL) frameworks with Socially Assistive Robots (SARs) has proven useful for monitoring and assisting older adults in their own home. However, the difficulties associated with long-term deployments in real-world complex environments are still highly under-explored. In this work, we first present the MoveCare system, an unobtrusive platform that, through the integration of a SAR into an AAL framework, aimed to monitor, assist and provide social, cognitive, and physical stimulation in the own houses of elders living alone and at risk of falling into frailty. We then focus on the evaluation and analysis of a long-term pilot campaign of more than 300 weeks of usages. We evaluated the system’s acceptability and feasibility through various questionnaires and empirically assessed the impact of the presence of an assistive robot by deploying the system with and without it. Our results provide strong empirical evidence that Socially Assistive Robots integrated with monitoring and stimulation platforms can be successfully used for long-term support to older adults. We describe how the robot’s presence significantly incentivised the use of the system, but slightly lowered the system’s overall acceptability. Finally, we emphasise that real-world long-term deployment of SARs introduces a significant technical, organisational, and logistical overhead that should not be neglected nor underestimated in the pursuit of long-term robust systems. We hope that the findings and lessons learned from our work can bring value towards future long-term real-world and widespread use of SARs

Runtime monitoring of electronic contracts

Abstract. Electronic inter-organizational relationships are governed by contracts regulating their interaction. It is necessary to run-time monitor the contracts, as to guarantee their fulfillment. The present work shows how to obtain a run-time monitor for contracts written in CL, a formal specification language which allows to write conditional obligations, permissions and prohibitions over actions. The trace semantics of CL formalizes the notion of a trace fulfills a contract. We show how to obtain, for a given contract, an alternating Büchi automaton which accepts exactly the traces that fulfill the contract. This automaton is the basis for obtaining a finite state machine which acts as a run-time monitor for CL contracts.

Instrument Pose Estimation Using Registration for Otobasis Surgery

Clinical outcome of several Minimally Invasive Surgeries (MIS) heavily depend on the accuracy of intraoperative pose estimation of the surgical instrument from intraoperative x-rays. The estimation consists of finding the tool in a given set of x-rays and extracting the necessary data to recreate the tool’s pose for further navigation - resulting in severe consequences of incorrect estimation. Though state-of-the-art MIS literature has exploited image registration as a tool for instrument pose estimation, lack of practical considerations in previous study design render their conclusion ineffective from a clinical standpoint. One major issue of such a study is the lack of Ground Truth in clinical data -as there are no direct ways of measuring the ground truth pose and indirect estimation accumulates error. A systematic way to overcome this problem is to generate Digitally Reconstructed Radiographs (DRR), however, such procedure generates data which are free from measuring errors (e.g. noise, number of projections), resulting claims of registration performance inconclusive. Generalization of registration performance across different instruments with different Degrees of Freedom (DoF) has not been studied as well. By marrying a rigorous study design involving several clinical scenarios with, for example, several optimizers, metrics and others parameters for image registration, this paper bridges this gap effectively. Although the pose estimation error scales inversely with instrument size, we show image registration generalizes well for different instruments and DoF. In particular, it is shown that increasing the number of x-ray projections can reduce the pose estimation error significantly across instruments - which might lead to the acquisition of several x-rays for pose estimation in a clinical workflow

A Simplified Toolbox for the Operability Assessment of the Built Environment in Middle School Buildings

The paper aims to propose a simplified set of tools to support the “ex ante” operability review of the built environment within the framework of new educational projects for themiddle schools. The starting hypothesis is that the educational projects should be developed in close connection with a forecast of the characteristics and performances that will be required for the built environment in order to optimally achieve the educational objectives. In order to identify a set of possible tools to be adopted for the above-mentioned process, the paper develops an analysis of the available models that are used in different areas such as, for instance, quality planning and value analysis. Two methods are investigated in detail in the paper and described as possible tools for the operability review: the QFD model (quality function deployment) and the FBS technique (function breakdown structures). A proposal is put forward in the paper in order to define key concepts and parameters to be used for adopting the QFD and FBS techniques within the scope of middle school educational projects. The proposed tools are also tested in a case study developed within amiddle school based in Milano with the aim of assessing the usability of the proposed tools by the teachers engaged in the set up of the new educational projects and educational proposal of the school