16,316 research outputs found

    Satellite Image Based Cross-view Localization for Autonomous Vehicle

    Full text link
    Existing spatial localization techniques for autonomous vehicles mostly use a pre-built 3D-HD map, often constructed using a survey-grade 3D mapping vehicle, which is not only expensive but also laborious. This paper shows that by using an off-the-shelf high-definition satellite image as a ready-to-use map, we are able to achieve cross-view vehicle localization up to a satisfactory accuracy, providing a cheaper and more practical way for localization. While the utilization of satellite imagery for cross-view localization is an established concept, the conventional methodology focuses primarily on image retrieval. This paper introduces a novel approach to cross-view localization that departs from the conventional image retrieval method. Specifically, our method develops (1) a Geometric-align Feature Extractor (GaFE) that leverages measured 3D points to bridge the geometric gap between ground and overhead views, (2) a Pose Aware Branch (PAB) adopting a triplet loss to encourage pose-aware feature extraction, and (3) a Recursive Pose Refine Branch (RPRB) using the Levenberg-Marquardt (LM) algorithm to align the initial pose towards the true vehicle pose iteratively. Our method is validated on KITTI and Ford Multi-AV Seasonal datasets as ground view and Google Maps as the satellite view. The results demonstrate the superiority of our method in cross-view localization with median spatial and angular errors within 11 meter and 11^\circ, respectively.Comment: Accepted by ICRA202

    Neural Architecture Search: Insights from 1000 Papers

    Full text link
    In the past decade, advances in deep learning have resulted in breakthroughs in a variety of areas, including computer vision, natural language understanding, speech recognition, and reinforcement learning. Specialized, high-performing neural architectures are crucial to the success of deep learning in these areas. Neural architecture search (NAS), the process of automating the design of neural architectures for a given task, is an inevitable next step in automating machine learning and has already outpaced the best human-designed architectures on many tasks. In the past few years, research in NAS has been progressing rapidly, with over 1000 papers released since 2020 (Deng and Lindauer, 2021). In this survey, we provide an organized and comprehensive guide to neural architecture search. We give a taxonomy of search spaces, algorithms, and speedup techniques, and we discuss resources such as benchmarks, best practices, other surveys, and open-source libraries

    Skin Capacitive Image Stitching and Occlusion Measurements

    Get PDF
    The aim of this study is to develop new analysis techniques for skin capacitive image stitching and occlusion measurements. Through image stitching, small skin capacitive images can be stitched into large skin capacitive images and, therefore, provide more skin image information. Through occlusion, e.g., keeping the measurement device on skin for a period of time, the skin health status can be studied through time-dependent response curves. Results show that time-dependent skin capacitive imaging curves can tell us the information about transdermal water loss (TEWL) as well as skin surface profiles. By using the structural similarity index measure (SSIM), the TEWL map can be constructed, which shows the water loss map on the skin surface. We first present the theoretical background and then the experimental results

    Chiral active fluids: Odd viscosity, active turbulence, and directed flows of hydrodynamic microrotors

    Get PDF
    While the number of publications on rotating active matter has rapidly increased in recent years, studies on purely hydrodynamically interacting rotors on the microscale are still rare, especially from the perspective of particle based hydrodynamic simulations. The work presented here targets to fill this gap. By means of high-performance computer simulations, performed in a highly parallelised fashion on graphics processing units, the dynamics of ensembles of up to 70,000 rotating colloids immersed in an explicit mesoscopic solvent consisting out of up to 30 million fluid particles, are investigated. Some of the results presented in this thesis have been worked out in collaboration with experimentalists, such that the theoretical considerations developed in this thesis are supported by experiments, and vice versa. The studied system, modelled in order to resemble the essential physics of the experimentally realisable system, consists out of rotating magnetic colloidal particles, i.e., (micro-)rotors, rotating in sync to an externally applied magnetic field, where the rotors solely interact via hydrodynamic and steric interactions. Overall, the agreement between simulations and experiments is very good, proving that hydrodynamic interactions play a key role in this and related systems. While already an isolated rotating colloid is driven out of equilibrium, only collections of two or more rotors have experimentally shown to be able to convert the rotational energy input into translational dynamics in an orbital rotating fashion. The rotating colloids inject circular flows into the fluid, such that detailed balance is broken, and it is not a priori known whether equilibrium properties of colloids can be extended to isolated rotating colloids. A joint theoretical and experimental analysis of isolated, pairs, and small groups of hydrodynamically interacting rotors is given in chapter 2. While the translational dynamics of isolated rotors effectively resemble the dynamics of non-rotating colloids, the orbital rotation of pairs of rotors can be described with leading order hydrodynamics and a two-dimensional analogy of Faxén’s law is derived. In chapter 3, a homogeneously distributed ensemble of rotors (bulk) as a realisation of a chiral active fluid is studied and it is explicitly shown computationally and experimentally that it carries odd viscosity. The mutual orbital translation of rotors and an increase of the effective solvent viscosity with rotor density lead to a non-monotonous behaviour of the average translational velocity. Meanwhile, the rotor suspension bears a finite osmotic compressibility resulting from the long-ranged nature of hydrody- namic interactions such that rotational and odd stresses are transmitted through the solvent also at small and intermediate rotor densities. Consequently, density inhomogeneities predicted for chiral active fluids with odd viscosity can be found and allow for an explicit measurement of odd viscosity in simulations and experiments. At intermediate densities, the collective dynamics shows the emergence of multi-scale vortices and chaotic motion which is identified as active turbulence with a self-similar power-law decay in the energy spectrum, showing that the injected energy on the rotor scale is transported to larger scales, similar to the inverse energy cascade of clas- sical two-dimensional turbulence. While either odd viscosity or active turbulence have been reported in chiral active matter previously, the system studied here shows that the emergence of both simultaneously is possible resulting from the osmotic compressibility and hydrodynamic mediation of odd and active stresses. The collective dynamics of colloids rotating out of phase, i.e., where a constant torque instead of a constant angular velocity is applied, is shown to be qualitatively very similar. However, at smaller densities, local density inhomogeneities imply position dependent angular velocities of the rotors resulting from inter-rotor friction. While the friction of a quasi-2D layer of active colloids with the substrate is often not easily modifiable in experiments, the incorporation of substrate friction into the simulation models typically implies a considerable increase in computational effort. In chapter 4, a very efficient way of incorporating the friction with a substrate into a two-dimensional multiparticle collision dynamics solvent is introduced, allowing for an explicit investigation of the influences of substrate on active dynamics. For the rotor fluid, it is explicitly shown that the influence of the substrate friction results in a cutoff of the hydrodynamic interaction length, such that the maximum size of the formed vortices is controlled by the substrate friction, also resulting in a cutoff in the energy spectrum, because energy is taken out of the system at the respective length. These findings are in agreement with the experiments. Since active particles in confinement are known to organise in states of collective dynamics, ensembles of rotationally actuated colloids are studied in circular confinement and in the presence of periodic obstacle lattices in chapters 5 and 6, respectively. The results show that the chaotic active turbulent transport of rotors in suspension can be enhanced and guided resulting from edge flows generated at the boundaries, as has recently been reported for a related chiral active system. The consequent collective rotor dynamics can be regarded as a superposition of active turbulent and imposed flows, leading to on average stationary flows. In contrast to the bulk dynamics, the imposed flows inject additional energy into the system on the long length scales, and the same scaling behaviour of the energy spectrum as in bulk is only obtained if the energy injection scales, due to the mutual generation of rotor translational dynamics throughout the system and the edge flows, are well separated. The combination of edge flow and entropic layering at the boundaries leads to oscillating hydrodynamic stresses and consequently to an oscillating vorticity profile. In the presence of odd viscosity, this consequently leads to non-trivial steady-state density modulations at the boundary, resulting from a balance of osmotic pressure and odd stresses. Relevant for the efficient dispersion and mixing of inert particles on the mesoscale by means of active turbulent mixing powered by rotors, a study of the dynamics of a binary mixture consisting out of rotors and passive particles is presented in chapter 7. Because the rotors are not self-propelled, but the translational dynamics is induced by the surrounding rotors, the passive particles, which do not inject further energy into the system, are transported according to the same mechanism as the rotors. The collective dynamics thus resembles the pure rotor bulk dynamics at the respective density of only rotors. However, since no odd stresses act between the passive particles, only mutual rotor interactions lead to odd stresses leading to the accumulation of rotors in the regions of positive vorticity. This density increase is associated with a pressure increase, which balances the odd stresses acting on the rotors. However, the passive particles are only subject to the accumulation induced pressure increase such that these particles are transported into the areas of low rotor concentration, i.e., the regions of negative vorticity. Under conditions of sustained vortex flow, this results in segregation of both particle types. Since local symmetry breaking can convert injected rotational into translational energy, microswimmers can be constructed out of rotor materials when a suitable breaking of symmetry is kept in the vicinity of a rotor. One hypothetical realisation, i.e., a coupled rotor pair consisting out of two rotors of opposite angular velocity and of fixed distance, termed a birotor, are studied in chapter 8. The birotor pumps the fluid into one direction and consequently translates into the opposite direction, and creates a flow field reminiscent of a source doublet, or sliplet flow field. Fixed in space the birotor might be an interesting realisation of a microfluidic pump. The trans- lational dynamics of a birotor can be mapped onto the active Brownian particle model for single swimmers. However, due to the hydrodynamic interactions among the rotors, the birotor ensemble dynamics do not show the emergence of stable motility induced clustering. The reason for this is the flow created by birotor in small aggregates which effectively pushes further arriving birotors away from small aggregates, which eventually are all dispersed by thermal fluctuations

    Loop Closure Detection Based on Object-level Spatial Layout and Semantic Consistency

    Full text link
    Visual simultaneous localization and mapping (SLAM) systems face challenges in detecting loop closure under the circumstance of large viewpoint changes. In this paper, we present an object-based loop closure detection method based on the spatial layout and semanic consistency of the 3D scene graph. Firstly, we propose an object-level data association approach based on the semantic information from semantic labels, intersection over union (IoU), object color, and object embedding. Subsequently, multi-view bundle adjustment with the associated objects is utilized to jointly optimize the poses of objects and cameras. We represent the refined objects as a 3D spatial graph with semantics and topology. Then, we propose a graph matching approach to select correspondence objects based on the structure layout and semantic property similarity of vertices' neighbors. Finally, we jointly optimize camera trajectories and object poses in an object-level pose graph optimization, which results in a globally consistent map. Experimental results demonstrate that our proposed data association approach can construct more accurate 3D semantic maps, and our loop closure method is more robust than point-based and object-based methods in circumstances with large viewpoint changes

    Leveraging Hidden Positives for Unsupervised Semantic Segmentation

    Full text link
    Dramatic demand for manpower to label pixel-level annotations triggered the advent of unsupervised semantic segmentation. Although the recent work employing the vision transformer (ViT) backbone shows exceptional performance, there is still a lack of consideration for task-specific training guidance and local semantic consistency. To tackle these issues, we leverage contrastive learning by excavating hidden positives to learn rich semantic relationships and ensure semantic consistency in local regions. Specifically, we first discover two types of global hidden positives, task-agnostic and task-specific ones for each anchor based on the feature similarities defined by a fixed pre-trained backbone and a segmentation head-in-training, respectively. A gradual increase in the contribution of the latter induces the model to capture task-specific semantic features. In addition, we introduce a gradient propagation strategy to learn semantic consistency between adjacent patches, under the inherent premise that nearby patches are highly likely to possess the same semantics. Specifically, we add the loss propagating to local hidden positives, semantically similar nearby patches, in proportion to the predefined similarity scores. With these training schemes, our proposed method achieves new state-of-the-art (SOTA) results in COCO-stuff, Cityscapes, and Potsdam-3 datasets. Our code is available at: https://github.com/hynnsk/HP.Comment: Accepted to CVPR 202

    Eigen-Factors an Alternating Optimization for Back-end Plane SLAM of 3D Point Clouds

    Full text link
    Modern depth sensors can generate a huge number of 3D points in few seconds to be latter processed by Localization and Mapping algorithms. Ideally, these algorithms should handle efficiently large sizes of Point Clouds under the assumption that using more points implies more information available. The Eigen Factors (EF) is a new algorithm that solves SLAM by using planes as the main geometric primitive. To do so, EF exhaustively calculates the error of all points at complexity O(1)O(1), thanks to the {\em Summation matrix} SS of homogeneous points. The solution of EF is highly efficient: i) the state variables are only the sensor poses -- trajectory, while the plane parameters are estimated previously in closed from and ii) EF alternating optimization uses a Newton-Raphson method by a direct analytical calculation of the gradient and the Hessian, which turns out to be a block diagonal matrix. Since we require to differentiate over eigenvalues and matrix elements, we have developed an intuitive methodology to calculate partial derivatives in the manifold of rigid body transformations SE(3)SE(3), which could be applied to unrelated problems that require analytical derivatives of certain complexity. We evaluate EF and other state-of-the-art plane SLAM back-end algorithms in a synthetic environment. The evaluation is extended to ICL dataset (RGBD) and LiDAR KITTI dataset. Code is publicly available at https://github.com/prime-slam/EF-plane-SLAM

    CrossLoc3D: Aerial-Ground Cross-Source 3D Place Recognition

    Full text link
    We present CrossLoc3D, a novel 3D place recognition method that solves a large-scale point matching problem in a cross-source setting. Cross-source point cloud data corresponds to point sets captured by depth sensors with different accuracies or from different distances and perspectives. We address the challenges in terms of developing 3D place recognition methods that account for the representation gap between points captured by different sources. Our method handles cross-source data by utilizing multi-grained features and selecting convolution kernel sizes that correspond to most prominent features. Inspired by the diffusion models, our method uses a novel iterative refinement process that gradually shifts the embedding spaces from different sources to a single canonical space for better metric learning. In addition, we present CS-Campus3D, the first 3D aerial-ground cross-source dataset consisting of point cloud data from both aerial and ground LiDAR scans. The point clouds in CS-Campus3D have representation gaps and other features like different views, point densities, and noise patterns. We show that our CrossLoc3D algorithm can achieve an improvement of 4.74% - 15.37% in terms of the top 1 average recall on our CS-Campus3D benchmark and achieves performance comparable to state-of-the-art 3D place recognition method on the Oxford RobotCar. We will release the code and CS-Campus3D benchmark

    Deep Learning for Scene Flow Estimation on Point Clouds: A Survey and Prospective Trends

    Get PDF
    Aiming at obtaining structural information and 3D motion of dynamic scenes, scene flow estimation has been an interest of research in computer vision and computer graphics for a long time. It is also a fundamental task for various applications such as autonomous driving. Compared to previous methods that utilize image representations, many recent researches build upon the power of deep analysis and focus on point clouds representation to conduct 3D flow estimation. This paper comprehensively reviews the pioneering literature in scene flow estimation based on point clouds. Meanwhile, it delves into detail in learning paradigms and presents insightful comparisons between the state-of-the-art methods using deep learning for scene flow estimation. Furthermore, this paper investigates various higher-level scene understanding tasks, including object tracking, motion segmentation, etc. and concludes with an overview of foreseeable research trends for scene flow estimation
    corecore