Real-time indoor assistive localization with mobile omnidirectional vision and cloud GPU acceleration

In this paper we propose a real-time assistive localization approach to help blind and visually impaired people in navigating an indoor environment. The system consists of a mobile vision front end with a portable panoramic lens mounted on a smart phone, and a remote image feature-based database of the scene on a GPU-enabled server. Compact and elective omnidirectional image features are extracted and represented in the smart phone front end, and then transmitted to the server in the cloud. These features of a short video clip are used to search the database of the indoor environment via image-based indexing to find the location of the current view within the database, which is associated with floor plans of the environment. A median-filter-based multi-frame aggregation strategy is used for single path modeling, and a 2D multi-frame aggregation strategy based on the candidates’ distribution densities is used for multi-path environmental modeling to provide a final location estimation. To deal with the high computational cost in searching a large database for a realistic navigation application, data parallelism and task parallelism properties are identified in the database indexing process, and computation is accelerated by using multi-core CPUs and GPUs. User-friendly HCI particularly for the visually impaired is designed and implemented on an iPhone, which also supports system configurations and scene modeling for new environments. Experiments on a database of an eight-floor building are carried out to demonstrate the capacity of the proposed system, with real-time response (14 fps) and robust localization results

Device-free Localization using Received Signal Strength Measurements in Radio Frequency Network

Device-free localization (DFL) based on the received signal strength (RSS) measurements of radio frequency (RF)links is the method using RSS variation due to the presence of the target to localize the target without attaching any device. The majority of DFL methods utilize the fact the link will experience great attenuation when obstructed. Thus that localization accuracy depends on the model which describes the relationship between RSS loss caused by obstruction and the position of the target. The existing models is too rough to explain some phenomenon observed in the experiment measurements. In this paper, we propose a new model based on diffraction theory in which the target is modeled as a cylinder instead of a point mass. The proposed model can will greatly fits the experiment measurements and well explain the cases like link crossing and walking along the link line. Because the measurement model is nonlinear, particle filtering tracing is used to recursively give the approximate Bayesian estimation of the position. The posterior Cramer-Rao lower bound (PCRLB) of proposed tracking method is also derived. The results of field experiments with 8 radio sensors and a monitored area of 3.5m 3.5m show that the tracking error of proposed model is improved by at least 36 percent in the single target case and 25 percent in the two targets case compared to other models.Comment: This paper has been withdrawn by the author due to some mistake

Real-Time Multi-Fisheye Camera Self-Localization and Egomotion Estimation in Complex Indoor Environments

In this work a real-time capable multi-fisheye camera self-localization and egomotion estimation framework is developed. The thesis covers all aspects ranging from omnidirectional camera calibration to the development of a complete multi-fisheye camera SLAM system based on a generic multi-camera bundle adjustment method

Improving Omnidirectional Camera-Based Robot Localization Through Self-Supervised Learning

Autonomous agents in any environment require accurate and reliable position and motion estimation to complete their required tasks. Many different sensor modalities have been utilized for this task such as GPS, ultra-wide band, visual simultaneous localization and mapping (SLAM), and light detection and ranging (LiDAR) SLAM. Many of the traditional positioning systems do not take advantage of the recent advances in the machine learning field. In this work, an omnidirectional camera position estimation system relying primarily on a learned model is presented. The positioning system benefits from the wide field of view provided by an omnidirectional camera. Recent developments in the self-supervised learning field for generating useful features from unlabeled data are also assessed. A novel radial patch pretext task for omnidirectional images is presented in this work. The resulting implementation will be a robot localization and tracking algorithm that can be adapted to a variety of environments such as warehouses and college campuses. Further experiments with additional types of sensors including 3D LiDAR, 60 GHz wireless, and Ultra-Wideband localization systems utilizing machine learning are also explored. A fused learned localization model utilizing multiple sensor modalities is evaluated in comparison to individual sensor models

Towards a generalized theory of low-frequency sound source localization

Low-frequency sound source localization generates considerable amount of disagreement between audio/acoustics researchers, with some arguing that below a certain frequency humans cannot localize a source with others insisting that in certain cases localization is possible, even down to the lowest audible of frequencies. Nearly all previous work in this area depends on subjective evaluations to formulate theorems for low-frequency localization. This, of course, opens the argument of data reliability, a critical factor that may go some way to explain the reported ambiguities with regard to low-frequency localization. The resulting proposal stipulates that low-frequency source localization is highly dependent on room dimensions, source/listener location and absorptive properties. In some cases, a source can be accurately localized down to the lowest audible of frequencies, while in other situations it cannot. This is relevant as the standard procedure in live sound reinforcement, cinema sound and home-theater surround sound is to have a single mono channel for the low-frequency content, based on the assumption that human’s cannot determine direction in this band. This work takes the first steps towards showing that this may not be a universally valid simplification and that certain sound reproduction systems may actually benefit from directional low-frequency content

A comparative evaluation of interest point detectors and local descriptors for visual SLAM

Abstract In this paper we compare the behavior of different interest points detectors and descriptors under the conditions needed to be used as landmarks in vision-based simultaneous localization and mapping (SLAM). We evaluate the repeatability of the detectors, as well as the invariance and distinctiveness of the descriptors, under different perceptual conditions using sequences of images representing planar objects as well as 3D scenes. We believe that this information will be useful when selecting an appropriat