Distributed multi-object tracking over sensor networks: a random finite set approach

The aim of the present dissertation is to address distributed tracking over a network of heterogeneous and geographically dispersed nodes (or agents) with sensing, communication and processing capabilities. Tracking is carried out in the Bayesian framework and its extension to a distributed context is made possible via an information-theoretic approach to data fusion which exploits consensus algorithms and the notion of Kullback-Leibler Average (KLA) of the Probability Density Functions (PDFs) to be fused. The first step toward distributed tracking considers a single moving object. Consensus takes place in each agent for spreading information over the network so that each node can track the object. To achieve such a goal, consensus is carried out on the local single-object posterior distribution, which is the result of local data processing, in the Bayesian setting, exploiting the last available measurement about the object. The next step is in the direction of distributed estimation of multiple moving objects. In order to model, in a rigorous and elegant way, a possibly time-varying number of objects present in a given area of interest, the Random Finite Set (RFS) formulation is adopted since it provides the notion of probability density for multi-object states that allows to directly extend existing tools in distributed estimation to multi-object tracking. The last theoretical step of the present dissertation is toward distributed filtering with the further requirement of unique object identities. To this end the labeled RFS framework is adopted as it provides a tractable approach to the multi-object Bayesian recursion. A generalization of the KLA to the labeled RFS framework, enables the development of novel consensus multi-object tracking filters which are fully distributed, scalable and computationally efficient.Comment: Ph.D. thesis of Claudio Fantacci, Universit\`a di Firenze, Dipartimento di Ingegneria dell'Informazione (DINFO), Florence, Italy Successfully defended on the 5th of March 201

Markerless visual servoing on unknown objects for humanoid robot platforms

To precisely reach for an object with a humanoid robot, it is of central importance to have good knowledge of both end-effector, object pose and shape. In this work we propose a framework for markerless visual servoing on unknown objects, which is divided in four main parts: I) a least-squares minimization problem is formulated to find the volume of the object graspable by the robot's hand using its stereo vision; II) a recursive Bayesian filtering technique, based on Sequential Monte Carlo (SMC) filtering, estimates the 6D pose (position and orientation) of the robot's end-effector without the use of markers; III) a nonlinear constrained optimization problem is formulated to compute the desired graspable pose about the object; IV) an image-based visual servo control commands the robot's end-effector toward the desired pose. We demonstrate effectiveness and robustness of our approach with extensive experiments on the iCub humanoid robot platform, achieving real-time computation, smooth trajectories and sub-pixel precisions

π2vec\pi2\text{vec}: Policy Representations with Successor Features

This paper describes $\pi2\text{vec}$, a method for representing behaviors of black box policies as feature vectors. The policy representations capture how the statistics of foundation model features change in response to the policy behavior in a task agnostic way, and can be trained from offline data, allowing them to be used in offline policy selection. This work provides a key piece of a recipe for fusing together three modern lines of research: Offline policy evaluation as a counterpart to offline RL, foundation models as generic and powerful state representations, and efficient policy selection in resource constrained environments.Comment: Accepted paper at ICLR202

Lossless Adaptation of Pretrained Vision Models For Robotic Manipulation

Recent works have shown that large models pretrained on common visual learning tasks can provide useful representations for a wide range of specialized perception problems, as well as a variety of robotic manipulation tasks. While prior work on robotic manipulation has predominantly used frozen pretrained features, we demonstrate that in robotics this approach can fail to reach optimal performance, and that fine-tuning of the full model can lead to significantly better results. Unfortunately, fine-tuning disrupts the pretrained visual representation, and causes representational drift towards the fine-tuned task thus leading to a loss of the versatility of the original model. We introduce "lossless adaptation" to address this shortcoming of classical fine-tuning. We demonstrate that appropriate placement of our parameter efficient adapters can significantly reduce the performance gap between frozen pretrained representations and full end-to-end fine-tuning without changes to the original representation and thus preserving original capabilities of the pretrained model. We perform a comprehensive investigation across three major model architectures (ViTs, NFNets, and ResNets), supervised (ImageNet-1K classification) and self-supervised pretrained weights (CLIP, BYOL, Visual MAE) in 3 task domains and 35 individual tasks, and demonstrate that our claims are strongly validated in various settings.Comment: ICLR'23, Project page see https://sites.google.com/view/robo-adapters

An unscented Kalman filter based navigation algorithm for autonomous underwater vehicles

Robust and performing navigation systems for Autonomous Underwater Vehicles (AUVs) play a discriminant role towards the success of complex underwater missions involving one or more AUVs. The quality of the filtering algorithm for the estimation of the AUV navigation state strongly affects the performance of the overall system. In this paper, the authors present a comparison between the Extended Kalman Filter (EKF) approach, classically used in the field of underwater robotics and an Unscented Kalman Filter (UKF). The comparison results to be significant as the two strategies of filtering are based on the same process and sensors models. The UKF-based approach, here adapted to the AUV case, demonstrates to be a good trade-off between estimation accuracy and computational load. UKF has not yet been extensively used in practical underwater applications, even if it turns out to be quite promising. The proposed results rely on the data acquired during a sea mission performed by one of the two Typhoon class vehicles involved in the NATO CommsNet13 experiment (held in September 2013). As ground truth for performance evaluation and comparison, performed offline, position measurements obtained through Ultra-Short BaseLine (USBL) fixes are used. The result analysis leads to identify both the strategies as effective for the purpose of being included in the control loop of an AUV. The UKF approach demonstrates higher performance encouraging its implementation as a more suitable navigation algorithm even if, up to now, it is still not used much in this field