Feature recognition and obstacle detection for drive assistance in indoor environments

The goal of this research project was to develop a robust feature recognition and obstacle detection method for smart wheelchair navigation in indoor environments. As two types of depth sensors were employed, two different methods were proposed and implemented in this thesis. The two methods combined information of colour, edge, depth and motion to detect obstacles, compute movements and recognize indoor room features. The first method was based on a stereo vision sensor and started with optimizing the noisy disparity images, then, RANSAC was used to estimate the ground plane, followed by a watershed based image segmentation algorithm for ground pixel classification. Meanwhile, a novel algorithm named a standard deviation ridge straight line detector was performed to extract straight lines from the RGB images. The algorithm is able to provide more useful information than using the Canny edge detector and the Hough Transform. Then, the novel drop-off detection and stairs-up detection algorithms based on the proposed straight line detector were carried out. Moreover, the camera movements were calculated by optical flow. The second method was based on a structured light sensor. After RANSAC ground plane estimation, morphology operations were applied to smooth the ground surface area. Then, an obstacle detection algorithm was carried out to create a top-down map of the ground plane using inverse perspective mapping and segment obstacles using a region growing-based algorithm. Both the drop-off and open door detection algorithms employ the straight lines extracted from depth discontinuities maps. The performance and accuracy of the two proposed methods were evaluated. Results show that the ground plane classification using the first method achieved 98.58% true positives, and the figure improved with the second method to 99%. The drop-off detection algorithms using the first method also achieved good results, with no false negatives found in the test video sequences. The system provided the top-down maps of the surroundings to detect and segment obstacles correctly. Overall, the results showing accurate distances to various detected indoor features and obstacles, suggests that this proposed colour/edge/motion/depth approach would be useful as a navigation aid through doorways and hallways

Computational Modeling of Human Dorsal Pathway for Motion Processing

Reliable motion estimation in videos is of crucial importance for background iden- tification, object tracking, action recognition, event analysis, self-navigation, etc. Re- constructing the motion field in the 2D image plane is very challenging, due to variations in image quality, scene geometry, lighting condition, and most importantly, camera jit- tering. Traditional optical flow models assume consistent image brightness and smooth motion field, which are violated by unstable illumination and motion discontinuities that are common in real world videos. To recognize observer (or camera) motion robustly in complex, realistic scenarios, we propose a biologically-inspired motion estimation system to overcome issues posed by real world videos. The bottom-up model is inspired from the infrastructure as well as functionalities of human dorsal pathway, and the hierarchical processing stream can be divided into three stages: 1) spatio-temporal processing for local motion, 2) recogni- tion for global motion patterns (camera motion), and 3) preemptive estimation of object motion. To extract effective and meaningful motion features, we apply a series of steer- able, spatio-temporal filters to detect local motion at different speeds and directions, in a way that\u27s selective of motion velocity. The intermediate response maps are cal- ibrated and combined to estimate dense motion fields in local regions, and then, local motions along two orthogonal axes are aggregated for recognizing planar, radial and circular patterns of global motion. We evaluate the model with an extensive, realistic video database that collected by hand with a mobile device (iPad) and the video content varies in scene geometry, lighting condition, view perspective and depth. We achieved high quality result and demonstrated that this bottom-up model is capable of extracting high-level semantic knowledge regarding self motion in realistic scenes. Once the global motion is known, we segment objects from moving backgrounds by compensating for camera motion. For videos captured with non-stationary cam- eras, we consider global motion as a combination of camera motion (background) and object motion (foreground). To estimate foreground motion, we exploit corollary dis- charge mechanism of biological systems and estimate motion preemptively. Since back- ground motions for each pixel are collectively introduced by camera movements, we apply spatial-temporal averaging to estimate the background motion at pixel level, and the initial estimation of foreground motion is derived by comparing global motion and background motion at multiple spatial levels. The real frame signals are compared with those derived by forward predictions, refining estimations for object motion. This mo- tion detection system is applied to detect objects with cluttered, moving backgrounds and is proved to be efficient in locating independently moving, non-rigid regions. The core contribution of this thesis is the invention of a robust motion estimation system for complicated real world videos, with challenges by real sensor noise, complex natural scenes, variations in illumination and depth, and motion discontinuities. The overall system demonstrates biological plausibility and holds great potential for other applications, such as camera motion removal, heading estimation, obstacle avoidance, route planning, and vision-based navigational assistance, etc

Spatial Displays and Spatial Instruments

The conference proceedings topics are divided into two main areas: (1) issues of spatial and picture perception raised by graphical electronic displays of spatial information; and (2) design questions raised by the practical experience of designers actually defining new spatial instruments for use in new aircraft and spacecraft. Each topic is considered from both a theoretical and an applied direction. Emphasis is placed on discussion of phenomena and determination of design principles

Pre-Trained Driving in Localized Surroundings with Semantic Radar Information and Machine Learning

Entlang der Signalverarbeitungskette von Radar Detektionen bis zur Fahrzeugansteuerung, diskutiert diese Arbeit eine semantischen Radar Segmentierung, einen darauf aufbauenden Radar SLAM, sowie eine im Verbund realisierte autonome Parkfunktion. Die Radarsegmentierung der (statischen) Umgebung wird durch ein Radar-spezifisches neuronales Netzwerk RadarNet erreicht. Diese Segmentierung ermöglicht die Entwicklung des semantischen Radar Graph-SLAM SERALOC. Auf der Grundlage der semantischen Radar SLAM Karte wird eine beispielhafte autonome Parkfunktionalität in einem realen Versuchsträger umgesetzt. Entlang eines aufgezeichneten Referenzfades parkt die Funktion ausschließlich auf Basis der Radar Wahrnehmung mit bisher unerreichter Positioniergenauigkeit. Im ersten Schritt wird ein Datensatz von 8.2 · 10^6 punktweise semantisch gelabelten Radarpunktwolken über eine Strecke von 2507.35m generiert. Es sind keine vergleichbaren Datensätze dieser Annotationsebene und Radarspezifikation öffentlich verfügbar. Das überwachte Training der semantischen Segmentierung RadarNet erreicht 28.97% mIoU auf sechs Klassen. Außerdem wird ein automatisiertes Radar-Labeling-Framework SeRaLF vorgestellt, welches das Radarlabeling multimodal mittels Referenzkameras und LiDAR unterstützt. Für die kohärente Kartierung wird ein Radarsignal-Vorfilter auf der Grundlage einer Aktivierungskarte entworfen, welcher Rauschen und andere dynamische Mehrwegreflektionen unterdrückt. Ein speziell für Radar angepasstes Graph-SLAM-Frontend mit Radar-Odometrie Kanten zwischen Teil-Karten und semantisch separater NDT Registrierung setzt die vorgefilterten semantischen Radarscans zu einer konsistenten metrischen Karte zusammen. Die Kartierungsgenauigkeit und die Datenassoziation werden somit erhöht und der erste semantische Radar Graph-SLAM für beliebige statische Umgebungen realisiert. Integriert in ein reales Testfahrzeug, wird das Zusammenspiel der live RadarNet Segmentierung und des semantischen Radar Graph-SLAM anhand einer rein Radar-basierten autonomen Parkfunktionalität evaluiert. Im Durchschnitt über 42 autonome Parkmanöver (∅3.73 km/h) bei durchschnittlicher Manöverlänge von ∅172.75m wird ein Median absoluter Posenfehler von 0.235m und End-Posenfehler von 0.2443m erreicht, der vergleichbare Radar-Lokalisierungsergebnisse um ≈ 50% übertrifft. Die Kartengenauigkeit von veränderlichen, neukartierten Orten über eine Kartierungsdistanz von ∅165m ergibt eine ≈ 56%-ige Kartenkonsistenz bei einer Abweichung von ∅0.163m. Für das autonome Parken wurde ein gegebener Trajektorienplaner und Regleransatz verwendet

Underwater Vehicles

For the latest twenty to thirty years, a significant number of AUVs has been created for the solving of wide spectrum of scientific and applied tasks of ocean development and research. For the short time period the AUVs have shown the efficiency at performance of complex search and inspection works and opened a number of new important applications. Initially the information about AUVs had mainly review-advertising character but now more attention is paid to practical achievements, problems and systems technologies. AUVs are losing their prototype status and have become a fully operational, reliable and effective tool and modern multi-purpose AUVs represent the new class of underwater robotic objects with inherent tasks and practical applications, particular features of technology, systems structure and functional properties

Activity in area V3A predicts positions of moving objects

No description supplie

Feature-Based Probabilistic Data Association for Video-Based Multi-Object Tracking

This work proposes a feature-based probabilistic data association and tracking approach (FBPDATA) for multi-object tracking. FBPDATA is based on re-identification and tracking of individual video image points (feature points) and aims at solving the problems of partial, split (fragmented), bloated or missed detections, which are due to sensory or algorithmic restrictions, limited field of view of the sensors, as well as occlusion situations

The role of direction-selective visual interneurons T4 and T5 in Drosophila orientation behavior

In order to safely move through the environment, visually-guided animals use several types of visual cues for orientation. Optic flow provides faithful information about ego-motion and can thus be used to maintain a straight course. Additionally, local motion cues or landmarks indicate potentially interesting targets or signal danger, triggering approach or avoidance, respectively. The visual system must reliably and quickly evaluate these cues and integrate this information in order to orchestrate behavior. The underlying neuronal computations for this remain largely inaccessible in higher organisms, such as in humans, but can be studied experimentally in more simple model species. The fly Drosophila, for example, heavily relies on such visual cues during its impressive flight maneuvers. Additionally, it is genetically and physiologically accessible. Hence, it can be regarded as an ideal model organism for exploring neuronal computations during visual processing. In my PhD studies, I have designed and built several autonomous virtual reality setups to precisely measure visual behavior of walking flies. The setups run in open-loop and in closed-loop configuration. In an open-loop experiment, the visual stimulus is clearly defined and does not depend on the behavioral response. Hence, it allows mapping of how specific features of simple visual stimuli are translated into behavioral output, which can guide the creation of computational models of visual processing. In closedloop experiments, the behavioral response is fed back onto the visual stimulus, which permits characterization of the behavior under more realistic conditions and, thus, allows for testing of the predictive power of the computational models. In addition, Drosophila’s genetic toolbox provides various strategies for targeting and silencing specific neuron types, which helps identify which cells are needed for a specific behavior. We have focused on visual interneuron types T4 and T5 and assessed their role in visual orientation behavior. These neurons build up a retinotopic array and cover the whole visual field of the fly. They constitute major output elements from the medulla and have long been speculated to be involved in motion processing. This cumulative thesis consists of three published studies: In the first study, we silenced both T4 and T5 neurons together and found that such flies were completely blind to any kind of motion. In particular, these flies could not perform an optomotor response anymore, which means that they lost their normally innate following responses to motion of large-field moving patterns. This was an important finding as it ruled out the contribution of another system for motion vision-based behaviors. However, these flies were still able to fixate a black bar. We could show that this behavior is mediated by a T4/T5-independent flicker detection circuitry which exists in parallel to the motion system. In the second study, T4 and T5 neurons were characterized via twophoton imaging, revealing that these cells are directionally selective and have very similar temporal and orientation tuning properties to directionselective neurons in the lobula plate. T4 and T5 cells responded in a contrast polarity-specific manner: T4 neurons responded selectively to ON edge motion while T5 neurons responded only to OFF edge motion. When we blocked T4 neurons, behavioral responses to moving ON edges were more impaired than those to moving OFF edges and the opposite was true for the T5 block. Hence, these findings confirmed that the contrast polarityspecific visual motion pathways, which start at the level of L1 (ON) and L2 (OFF), are maintained within the medulla and that motion information is computed twice independently within each of these pathways. Finally, in the third study, we used the virtual reality setups to probe the performance of an artificial microcircuit. The system was equipped with a camera and spherical fisheye lens. Images were processed by an array of Reichardt detectors whose outputs were integrated in a similar way to what is found in the lobula plate of flies. We provided the system with several rotating natural environments and found that the fly-inspired artificial system could accurately predict the axes of rotation

NASA Thesaurus. Volume 1: Hierarchical listing

There are 16,713 postable terms and 3,716 nonpostable terms approved for use in the NASA scientific and technical information system in the Hierarchical Listing of the NASA Thesaurus. The generic structure is presented for many terms. The broader term and narrower term relationships are shown in an indented fashion that illustrates the generic structure better than the more widely used BT and NT listings. Related terms are generously applied, thus enhancing the usefulness of the Hierarchical Listing. Greater access to the Hierarchical Listing may be achieved with the collateral use of Volume 2 - Access Vocabulary