Razvoj hijerarhijske strukture upravljanja mobilnim robotom za praćenje ljudi na bazi robusne stereo robotske vizije

The main topic of this doctoral thesis refers to the development and implementation of the hierarchical control structure in which the algorithms are being executed on the high-level control. By applying the stochastic methods in robotic vision to these algorithms, we can detect people, estimate their position, follow them and recognise their actions in order to carry out tasks where the robot behaves like a human's collaborator. In this thesis, some solutions are offered and present a step forward towards solving the problems that robotic vision system that provides reliable inputs to the control module of the mobile human-collaboration robot, is facing. The robust vision module for human tracking which can be applied in various applications where is necessary for robots to work together with humans and which can be applied on different types of mobile robots, was developed. In this thesis, it is devoted a special attention to the integration, testing and experimental verification of the stochastic algorithms for human tracking such as Kaman and Particle filters, as well as a comparative analysis of algorithms for solving problems of robotic people tracking. A part of the research presented in this thesis is based on a scientific collaboration between the researchers from the Faculty of Mechanical Engineering of Nis and the researchers from the Institute of Automation (IAT) of the University of Bremen. The stereo vision module for a human detection that was developed at the Institute of Automatic Control, University of Bremen (IAT), was used for testing the tracking module which was developed in this thesis. Beside the detection system, the systems for human detection that use 3D sensors, such as the Asus Xtion PRO LIVE 3D sensor were used. The main focus of this thesis was the development of a simulation environment and its control system, as well as the development of modules for human tracking, estimation of a human position and recognition of a human behavior. The simulation environment represents the support to the development and implementation of the real world control system. By adding the appropriate modifications, other mobile robots can easily use this simulation environment. The developed algorithms are evaluated on Faculty of Mechanical Engineering, University of Niš as a part of the doctoral dissertation. In this dissertation, advanced hierarchical control was implemented for the purpose of controlling the mobile robot DaNI, developed by the company National Instruments. This advanced hierarchical control was implemented by using 3D sensor Asus Xtion Pro Live which in the laboratory experimental scenario represents the robotic vision sensor for the detection modules and human tracking. In addition, at the IAT, the vision module which consists of two sub-modules was implemented. These two sub-modules are the stereo vision for a human detection and the tracking module based on the Kalman filter developed in this doctoral thesis

Speech rhythms and multiplexed oscillatory sensory coding in the human brain

Cortical oscillations are likely candidates for segmentation and coding of continuous speech. Here, we monitored continuous speech processing with magnetoencephalography (MEG) to unravel the principles of speech segmentation and coding. We demonstrate that speech entrains the phase of low-frequency (delta, theta) and the amplitude of high-frequency (gamma) oscillations in the auditory cortex. Phase entrainment is stronger in the right and amplitude entrainment is stronger in the left auditory cortex. Furthermore, edges in the speech envelope phase reset auditory cortex oscillations thereby enhancing their entrainment to speech. This mechanism adapts to the changing physical features of the speech envelope and enables efficient, stimulus-specific speech sampling. Finally, we show that within the auditory cortex, coupling between delta, theta, and gamma oscillations increases following speech edges. Importantly, all couplings (i.e., brain-speech and also within the cortex) attenuate for backward-presented speech, suggesting top-down control. We conclude that segmentation and coding of speech relies on a nested hierarchy of entrained cortical oscillations

A Reverse Hierarchy Model for Predicting Eye Fixations

A number of psychological and physiological evidences suggest that early visual attention works in a coarse-to-fine way, which lays a basis for the reverse hierarchy theory (RHT). This theory states that attention propagates from the top level of the visual hierarchy that processes gist and abstract information of input, to the bottom level that processes local details. Inspired by the theory, we develop a computational model for saliency detection in images. First, the original image is downsampled to different scales to constitute a pyramid. Then, saliency on each layer is obtained by image super-resolution reconstruction from the layer above, which is defined as unpredictability from this coarse-to-fine reconstruction. Finally, saliency on each layer of the pyramid is fused into stochastic fixations through a probabilistic model, where attention initiates from the top layer and propagates downward through the pyramid. Extensive experiments on two standard eye-tracking datasets show that the proposed method can achieve competitive results with state-of-the-art models.Comment: CVPR 2014, 27th IEEE Conference on Computer Vision and Pattern Recognition (CVPR). CVPR 201

Closed-loop Bayesian Semantic Data Fusion for Collaborative Human-Autonomy Target Search

In search applications, autonomous unmanned vehicles must be able to efficiently reacquire and localize mobile targets that can remain out of view for long periods of time in large spaces. As such, all available information sources must be actively leveraged -- including imprecise but readily available semantic observations provided by humans. To achieve this, this work develops and validates a novel collaborative human-machine sensing solution for dynamic target search. Our approach uses continuous partially observable Markov decision process (CPOMDP) planning to generate vehicle trajectories that optimally exploit imperfect detection data from onboard sensors, as well as semantic natural language observations that can be specifically requested from human sensors. The key innovation is a scalable hierarchical Gaussian mixture model formulation for efficiently solving CPOMDPs with semantic observations in continuous dynamic state spaces. The approach is demonstrated and validated with a real human-robot team engaged in dynamic indoor target search and capture scenarios on a custom testbed.Comment: Final version accepted and submitted to 2018 FUSION Conference (Cambridge, UK, July 2018