Localization and Optimization Problems for Camera Networks

In the framework of networked control systems, we focus on networks of autonomous PTZ cameras. A large set of cameras communicating each other through a network is a widely used architecture in application areas like video surveillance, tracking and motion. First, we consider relative localization in sensor networks, and we tackle the issue of investigating the error propagation, in terms of the mean error on each component of the optimal estimator of the position vector. The relative error is computed as a function of the eigenvalues of the network: using this formula and focusing on an exemplary class of networks (the Abelian Cayley networks), we study the role of the network topology and the dimension of the networks in the error characterization. Second, in a network of cameras one of the most crucial problems is calibration. For each camera this consists in understanding what is its position and orientation with respect to a global common reference frame. Well-known methods in computer vision permit to obtain relative positions and orientations of pairs of cameras whose sensing regions overlap. The aim is to propose an algorithm that, from these noisy input data makes the cameras complete the calibration task autonomously, in a distributed fashion. We focus on the planar case, formulating an optimization problem over the manifold SO(2). We propose synchronous deterministic and distributed algorithms that calibrate planar networks exploiting the cycle structure of the underlying communication graph. Performance analysis and numerical experiments are shown. Third, we propose a gossip-like randomized calibration algorithm, whose probabilistic convergence and numerical studies are provided. Forth and finally, we design surveillance trajectories for a network of calibrated autonomous cameras to detect intruders in an environment, through a continuous graph partitioning problem

A socio-economic approach to online vision graph generation and handover in distributed smart camera networks

Abstract—In this paper we propose an approach based on selfinterested autonomous cameras, which exchange responsibility for tracking objects in a market mechanism, in order to maximise their own utility. A novel ant-colony inspired mechanism is used to grow the vision graph during runtime, which may then be used to optimise communication between cameras. The key benefits of our completely decentralised approach are on the one hand generating the vision graph online which permits the addition and removal cameras to the network during runtime and on the other hand relying only on local information, increasing the robustness of the system. Since our market-based approach does not rely on a priori topology information, the need for any multicamera calibration can be avoided. Index Terms—Smart camera networks; multi-camera tracking; market-based control; topology identification; ant algorithms. I

Fast and Reliable Autonomous Surgical Debridement with Cable-Driven Robots Using a Two-Phase Calibration Procedure

Automating precision subtasks such as debridement (removing dead or diseased tissue fragments) with Robotic Surgical Assistants (RSAs) such as the da Vinci Research Kit (dVRK) is challenging due to inherent non-linearities in cable-driven systems. We propose and evaluate a novel two-phase coarse-to-fine calibration method. In Phase I (coarse), we place a red calibration marker on the end effector and let it randomly move through a set of open-loop trajectories to obtain a large sample set of camera pixels and internal robot end-effector configurations. This coarse data is then used to train a Deep Neural Network (DNN) to learn the coarse transformation bias. In Phase II (fine), the bias from Phase I is applied to move the end-effector toward a small set of specific target points on a printed sheet. For each target, a human operator manually adjusts the end-effector position by direct contact (not through teleoperation) and the residual compensation bias is recorded. This fine data is then used to train a Random Forest (RF) to learn the fine transformation bias. Subsequent experiments suggest that without calibration, position errors average 4.55mm. Phase I can reduce average error to 2.14mm and the combination of Phase I and Phase II can reduces average error to 1.08mm. We apply these results to debridement of raisins and pumpkin seeds as fragment phantoms. Using an endoscopic stereo camera with standard edge detection, experiments with 120 trials achieved average success rates of 94.5%, exceeding prior results with much larger fragments (89.4%) and achieving a speedup of 2.1x, decreasing time per fragment from 15.8 seconds to 7.3 seconds. Source code, data, and videos are available at https://sites.google.com/view/calib-icra/.Comment: Code, data, and videos are available at https://sites.google.com/view/calib-icra/. Final version for ICRA 201