Encoderless Gimbal Calibration of Dynamic Multi-Camera Clusters

Dynamic Camera Clusters (DCCs) are multi-camera systems where one or more cameras are mounted on actuated mechanisms such as a gimbal. Existing methods for DCC calibration rely on joint angle measurements to resolve the time-varying transformation between the dynamic and static camera. This information is usually provided by motor encoders, however, joint angle measurements are not always readily available on off-the-shelf mechanisms. In this paper, we present an encoderless approach for DCC calibration which simultaneously estimates the kinematic parameters of the transformation chain as well as the unknown joint angles. We also demonstrate the integration of an encoderless gimbal mechanism with a state-of-the art VIO algorithm, and show the extensions required in order to perform simultaneous online estimation of the joint angles and vehicle localization state. The proposed calibration approach is validated both in simulation and on a physical DCC composed of a 2-DOF gimbal mounted on a UAV. Finally, we show the experimental results of the calibrated mechanism integrated into the OKVIS VIO package, and demonstrate successful online joint angle estimation while maintaining localization accuracy that is comparable to a standard static multi-camera configuration.Comment: ICRA 201

GyroFlow: Gyroscope-Guided Unsupervised Optical Flow Learning

Existing optical flow methods are erroneous in challenging scenes, such as fog, rain, and night because the basic optical flow assumptions such as brightness and gradient constancy are broken. To address this problem, we present an unsupervised learning approach that fuses gyroscope into optical flow learning. Specifically, we first convert gyroscope readings into motion fields named gyro field. Then, we design a self-guided fusion module to fuse the background motion extracted from the gyro field with the optical flow and guide the network to focus on motion details. To the best of our knowledge, this is the first deep learning-based framework that fuses gyroscope data and image content for optical flow learning. To validate our method, we propose a new dataset that covers regular and challenging scenes. Experiments show that our method outperforms the state-of-art methods in both regular and challenging scenes

GyroFlow+: Gyroscope-Guided Unsupervised Deep Homography and Optical Flow Learning

Existing homography and optical flow methods are erroneous in challenging scenes, such as fog, rain, night, and snow because the basic assumptions such as brightness and gradient constancy are broken. To address this issue, we present an unsupervised learning approach that fuses gyroscope into homography and optical flow learning. Specifically, we first convert gyroscope readings into motion fields named gyro field. Second, we design a self-guided fusion module (SGF) to fuse the background motion extracted from the gyro field with the optical flow and guide the network to focus on motion details. Meanwhile, we propose a homography decoder module (HD) to combine gyro field and intermediate results of SGF to produce the homography. To the best of our knowledge, this is the first deep learning framework that fuses gyroscope data and image content for both deep homography and optical flow learning. To validate our method, we propose a new dataset that covers regular and challenging scenes. Experiments show that our method outperforms the state-of-the-art methods in both regular and challenging scenes.Comment: 12 pages. arXiv admin note: substantial text overlap with arXiv:2103.1372

Keyframe-based visual–inertial odometry using nonlinear optimization

Combining visual and inertial measurements has become popular in mobile robotics, since the two sensing modalities offer complementary characteristics that make them the ideal choice for accurate visual–inertial odometry or simultaneous localization and mapping (SLAM). While historically the problem has been addressed with filtering, advancements in visual estimation suggest that nonlinear optimization offers superior accuracy, while still tractable in complexity thanks to the sparsity of the underlying problem. Taking inspiration from these findings, we formulate a rigorously probabilistic cost function that combines reprojection errors of landmarks and inertial terms. The problem is kept tractable and thus ensuring real-time operation by limiting the optimization to a bounded window of keyframes through marginalization. Keyframes may be spaced in time by arbitrary intervals, while still related by linearized inertial terms. We present evaluation results on complementary datasets recorded with our custom-built stereo visual–inertial hardware that accurately synchronizes accelerometer and gyroscope measurements with imagery. A comparison of both a stereo and monocular version of our algorithm with and without online extrinsics estimation is shown with respect to ground truth. Furthermore, we compare the performance to an implementation of a state-of-the-art stochastic cloning sliding-window filter. This competitive reference implementation performs tightly coupled filtering-based visual–inertial odometry. While our approach declaredly demands more computation, we show its superior performance in terms of accuracy

3D INDOOR STATE ESTIMATION FOR RFID-BASED MOTION-CAPTURE SYSTEMS

The objective of this research is to realize 3D indoor state estimation for RFID-based motion-capture systems. The state estimation is based on sensor fusion by combining RF signal with IMU data together. 3D state-space model of sensor fusion and 3D nonlinear state estimation in NLE with both asynchronous and synchronous models to handle different sensor sampling rates were proposed. For 3D motion with indoor multipath, RMS error before estimation is 71.99 cm, in which 34.99 cm in xy- plane and 62.92 cm along z- axis. After NLE estimation using RF signal combined with IMU data, RMS error of 3D coordinates decreases to 31.90 cm, with 22.50 cm in xy- plane and 22.61 cm along z- axis, achieving a factor of 2 enhancement which is similar to the 2D estimation. In addition, using RF signal only obtains similar estimation results to using both RF and IMU, i.e., 3D RMS error of 31.90 cm, where 22.48 cm in xy- plane and 22.62 cm along z- axis. Hence, RF signal only is able to achieve fine-scale RFID-based motion capture in 3D motion, in consistency with the conclusion arrived at in 2D estimation. In this way, RFID-based motion capture systems can be simplified from embedding inertial sensors. EKF derives close results with 2 cm larger RMS error. In addition, ToF based position sensor in tracking achieves comparable and higher accuracy compared to RSS based position sensor based on the multipath simulation model, enabling ToF to be applied in fine-scale motion capture and tracking.Ph.D

Lifelong localization of robots

This work presents a novel technique for lifelong localization of robots. It performs a tight fusion of GPS and Multi-State Constraint Kalman Filter, a visual-inertial odometry method for robot localization. It is shown in exper- iments that the proposed algorithm achieves better position accuracy than either GPS and Multi-State Constraint Kalman Filter alone. Additionally, the experiments demonstrate that the algorithm is able to reliably operate when the GPS signal is highly corrupted by noise or even in presence of substantial GPS outages. 1Tato práce představuje novou techniku pro celoživotní lokalizaci robotů. Provádí pevné spojení GPS a Multi-State Constraint Kalman Filter, což je metoda vizuální-inerciální odometrie pro lokalizaci robotů. V experimentech je ukázáno, že navrhovaná technika dosahuje lepší přesnosti polohy než GPS nebo Multi-State Constraint Kalman Filter samostatně. Navíc experimenty ukazují, že algoritmus je schopen spolehlivě fungovat, když je signál GPS silně zašuměný nebo dokonce i v případě značných výpadků GPS. 1Katedra teoretické informatiky a matematické logikyDepartment of Theoretical Computer Science and Mathematical LogicFaculty of Mathematics and PhysicsMatematicko-fyzikální fakult

Enabling the Virtual Phones to remotely sense the Real Phones in real-time: A Sensor Emulation initiative for virtualized Android-x86

Smartphones nowadays have the ground-breaking features that were only a figment of one’s imagination. For the ever-demanding cellphone users, the exhaustive list of features that a smartphone supports just keeps getting more exhaustive with time. These features aid one’s personal and professional uses as well. Extrapolating into the future the features of a present-day smartphone, the lives of us humans using smartphones are going to be unimaginably agile. With the above said emphasis on the current and future potential of a smartphone, the ability to virtualize smartphones with all their real-world features into a virtual platform, is a boon for those who want to rigorously experiment and customize the virtualized smartphone hardware without spending an extra penny. Once virtualizable independently on a larger scale, the idea of virtualized smartphones with all the virtualized pieces of hardware takes an interesting turn with the sensors being virtualized in a way that’s closer to the real-world behavior. When accessible remotely with the real-time responsiveness, the above mentioned real-world behavior will be a real dealmaker in many real-world systems, namely, the life-saving systems like the ones that instantaneously get alerts about harmful magnetic radiations in the deep mining areas, etc. And these life-saving systems would be installed on a large scale on the desktops or large servers as virtualized smartphones having the added support of virtualized sensors which remotely fetch the real hardware sensor readings from a real smartphone in real-time. Based on these readings the lives working in the affected areas can be alerted and thus saved by the people who are operating the at the desktops or large servers hosting the virtualized smartphones

Développement d'une méthode de géolocalisation à l'intérieur de bâtiments par classification des fingerprints GSM et fusion de données de capteurs embarqués

GPS has long been used for accurate and reliable outdoor localization, but it cannot operate in indoor environments, which suggests developing indoor localization methods that can provide seamless and ubiquitous services for mobile users.In this thesis, indoor localization is realized making use of received signal strength fingerprinting technique based on the existing GSM networks. A room is defined as the minimum location unit, and support vector machine are used as a mean to discriminate the rooms by classifying received signal strengths from very large number of GSM carriers. At the same time, multiple sensors, such as accelerometer and gyroscope, are widely available for modern mobile devices, which provide additional information that helps location determination. The hybrid approach that combines the GSM fingerprinting results with mobile sensor and building layout information using a particle filter provides a more accurate and fine-grained localization result.The results of experiments under realistic conditions demonstrate that correct room number can be obtained 94% of the time provided the derived model is used before significant received signal strength drift sets in. Furthermore, if the training data is sampled over a few days, the performance can remain stable exceeding 80% over a period of months, and can be further improved with various post-processing techniques. Moreover, including the mobile sensors allows the system to localize the mobile trajectory coordinates with high accuracy and reliability.L’objet de cette thèse est l’étude de la localisation et de la navigation à l’intérieur de bâtiments à l’aide des signaux disponibles dans les systèmes mobiles cellulaires et, en particulier, les signaux GSM.Le système GPS est aujourd’hui couramment utilisé en extérieur pour déterminer la position d’un objet, mais les signaux GPS ne sont pas adaptés à la localisation en intérieurIci, la localisation en intérieur est obtenue à partir de la technique des «empreintes» de puissance des signaux reçus sur les canaux utilisés par les réseaux GSM. Elle est réalisée à l’échelle de la pièce. Une classification est effectuée à partir de machines à vecteurs supports et les descripteurs utilisés sont les puissances de toutes les porteuses GSM. D’autres capteurs physiques disponibles dans les téléphones portables fournissent des informations utiles pour déterminer la position ou le déplacement de l’utilisateur. Celles-ci, ainsi que la cartographie de l’environnement, sont associées aux résultats obtenus à partir des «empreintes» GSM au sein de filtres particulaires afin d’obtenir une localisation plus précise, et sous forme de coordonnées continues.Les résultats obtenus montrent que l’utilisation des seules empreintes GSM permet de déterminer la pièce correcte dans 94% des cas sur une durée courte et que les performances restent stables pendant plusieurs mois, de l’ordre de 80%, si les données d’apprentissage sont enregistrées sur quelques jours. L’association de la cartographie du lieu et des informations issues des autres capteurs aux données de classification permettent d’obtenir les coordonnées de la trajectoire du système mobile avec une bonne précision et une bonne fiabilité