1,098 research outputs found

    Closed-loop Bayesian Semantic Data Fusion for Collaborative Human-Autonomy Target Search

    Full text link
    In search applications, autonomous unmanned vehicles must be able to efficiently reacquire and localize mobile targets that can remain out of view for long periods of time in large spaces. As such, all available information sources must be actively leveraged -- including imprecise but readily available semantic observations provided by humans. To achieve this, this work develops and validates a novel collaborative human-machine sensing solution for dynamic target search. Our approach uses continuous partially observable Markov decision process (CPOMDP) planning to generate vehicle trajectories that optimally exploit imperfect detection data from onboard sensors, as well as semantic natural language observations that can be specifically requested from human sensors. The key innovation is a scalable hierarchical Gaussian mixture model formulation for efficiently solving CPOMDPs with semantic observations in continuous dynamic state spaces. The approach is demonstrated and validated with a real human-robot team engaged in dynamic indoor target search and capture scenarios on a custom testbed.Comment: Final version accepted and submitted to 2018 FUSION Conference (Cambridge, UK, July 2018

    Probabilistic Human-Robot Information Fusion

    Get PDF
    This thesis is concerned with combining the perceptual abilities of mobile robots and human operators to execute tasks cooperatively. It is generally agreed that a synergy of human and robotic skills offers an opportunity to enhance the capabilities of today’s robotic systems, while also increasing their robustness and reliability. Systems which incorporate both human and robotic information sources have the potential to build complex world models, essential for both automated and human decision making. In this work, humans and robots are regarded as equal team members who interact and communicate on a peer-to-peer basis. Human-robot communication is addressed using probabilistic representations common in robotics. While communication can in general be bidirectional, this work focuses primarily on human-to-robot information flow. More specifically, the approach advocated in this thesis is to let robots fuse their sensor observations with observations obtained from human operators. While robotic perception is well-suited for lower level world descriptions such as geometric properties, humans are able to contribute perceptual information on higher abstraction levels. Human input is translated into the machine representation via Human Sensor Models. A common mathematical framework for humans and robots reinforces the notion of true peer-to-peer interaction. Human-robot information fusion is demonstrated in two application domains: (1) scalable information gathering, and (2) cooperative decision making. Scalable information gathering is experimentally demonstrated on a system comprised of a ground vehicle, an unmanned air vehicle, and two human operators in a natural environment. Information from humans and robots was fused in a fully decentralised manner to build a shared environment representation on multiple abstraction levels. Results are presented in the form of information exchange patterns, qualitatively demonstrating the benefits of human-robot information fusion. The second application domain adds decision making to the human-robot task. Rational decisions are made based on the robots’ current beliefs which are generated by fusing human and robotic observations. Since humans are considered a valuable resource in this context, operators are only queried for input when the expected benefit of an observation exceeds the cost of obtaining it. The system can be seen as adjusting its autonomy at run-time based on the uncertainty in the robots’ beliefs. A navigation task is used to demonstrate the adjustable autonomy system experimentally. Results from two experiments are reported: a quantitative evaluation of human-robot team effectiveness, and a user study to compare the system to classical teleoperation. Results show the superiority of the system with respect to performance, operator workload, and usability

    Active Perception by Interaction with Other Agents in a Predictive Coding Framework: Application to Internet of Things Environment

    Get PDF
    Predicting the state of an agent\u27s partially-observable environment is a problem of interest in many domains. Typically in the real world, the environment consists of multiple agents, not necessarily working towards a common goal. Though the goal and sensory observation for each agent is unique, one agent might have acquired some knowledge that may benefit the other. In essence, the knowledge base regarding the environment is distributed among the agents. An agent can sample this distributed knowledge base by communicating with other agents. Since an agent is not storing the entire knowledge base, its model can be small and its inference can be efficient and fault-tolerant. However, the agent needs to learn -- when, with whom and what -- to communicate (in general interact) under different situations.This dissertation presents an agent model that actively and selectively communicates with other agents to predict the state of its environment efficiently. Communication is a challenge when the internal models of other agents is unknown and unobservable. The proposed agent learns communication policies as mappings from its belief state to when, with whom and what to communicate. The policies are learned using predictive coding in an online manner, without any reinforcement. The proposed agent model is evaluated on widely-studied applications, such as human activity recognition from multimodal, multisource and heterogeneous sensor data, and transferring knowledge across sensor networks. In the applications, either each sensor or each sensor network is assumed to be monitored by an agent. The recognition accuracy on benchmark datasets is comparable to the state-of-the-art, even though our model has significantly fewer parameters and infers the state in a localized manner. The learned policy reduces number of communications. The agent is tolerant to communication failures and can recognize the reliability of each agent from its communication messages. To the best of our knowledge, this is the first work on learning communication policies by an agent for predicting the state of its environment

    Distributed detection, localization, and estimation in time-critical wireless sensor networks

    Get PDF
    In this thesis the problem of distributed detection, localization, and estimation (DDLE) of a stationary target in a fusion center (FC) based wireless sensor network (WSN) is considered. The communication process is subject to time-critical operation, restricted power and bandwidth (BW) resources operating over a shared communication channel Buffering from Rayleigh fading and phase noise. A novel algorithm is proposed to solve the DDLE problem consisting of two dependent stages: distributed detection and distributed estimation. The WSN performs distributed detection first and based on the global detection decision the distributed estimation stage is performed. The communication between the SNs and the FC occurs over a shared channel via a slotted Aloha MAC protocol to conserve BW. In distributed detection, hard decision fusion is adopted, using the counting rule (CR), and sensor censoring in order to save power and BW. The effect of Rayleigh fading on distributed detection is also considered and accounted for by using distributed diversity combining techniques where the diversity combining is among the sensor nodes (SNs) in lieu of having the processing done at the FC. Two distributed techniques are proposed: the distributed maximum ratio combining (dMRC) and the distributed Equal Gain Combining (dEGC). Both techniques show superior detection performance when compared to conventional diversity combining procedures that take place at the FC. In distributed estimation, the segmented distributed localization and estimation (SDLE) framework is proposed. The SDLE enables efficient power and BW processing. The SOLE hinges on the idea of introducing intermediate parameters that are estimated locally by the SNs and transmitted to the FC instead of the actual measurements. This concept decouples the main problem into a simpler set of local estimation problems solved at the SNs and a global estimation problem solved at the FC. Two algorithms are proposed for solving the local problem: a nonlinear least squares (NLS) algorithm using the variable projection (VP) method and a simpler gird search (GS) method. Also, Four algorithms are proposed to solve the global problem: NLS, GS, hyperspherical intersection method (HSI), and robust hyperspherical intersection (RHSI) method. Thus, the SDLE can be solved through local and global algorithm combinations. Five combinations are tied: NLS2 (NLS-NLS), NLS-HSI, NLS-RHSI, GS2, and GS-N LS. It turns out that the last algorithm combination delivers the best localization and estimation performance. In fact , the target can be localized with less than one meter error. The SNs send their local estimates to the FC over a shared channel using the slotted-Aloha MAC protocol, which suits WSNs since it requires only one channel. However, Aloha is known for its relatively high medium access or contention delay given the medium access probability is poorly chosen. This fact significantly hinders the time-critical operation of the system. Hence, multi-packet reception (MPR) is used with slotted Aloha protocol, in which several channels are used for contention. The contention delay is analyzed for slotted Aloha with and without MPR. More specifically, the mean and variance have been analytically computed and the contention delay distribution is approximated. Having theoretical expressions for the contention delay statistics enables optimizing both the medium access probability and the number of MPR channels in order to strike a trade-off between delay performance and complexity

    HARPS: An Online POMDP Framework for Human-Assisted Robotic Planning and Sensing

    Full text link
    Autonomous robots can benefit greatly from human-provided semantic characterizations of uncertain task environments and states. However, the development of integrated strategies which let robots model, communicate, and act on such 'soft data' remains challenging. Here, the Human Assisted Robotic Planning and Sensing (HARPS) framework is presented for active semantic sensing and planning in human-robot teams to address these gaps by formally combining the benefits of online sampling-based POMDP policies, multimodal semantic interaction, and Bayesian data fusion. This approach lets humans opportunistically impose model structure and extend the range of semantic soft data in uncertain environments by sketching and labeling arbitrary landmarks across the environment. Dynamic updating of the environment model while during search allows robotic agents to actively query humans for novel and relevant semantic data, thereby improving beliefs of unknown environments and states for improved online planning. Simulations of a UAV-enabled target search application in a large-scale partially structured environment show significant improvements in time and belief state estimates required for interception versus conventional planning based solely on robotic sensing. Human subject studies in the same environment (n = 36) demonstrate an average doubling in dynamic target capture rate compared to the lone robot case, and highlight the robustness of active probabilistic reasoning and semantic sensing over a range of user characteristics and interaction modalities

    A Comparative Study of Modern Inference Techniques for Structured Discrete Energy Minimization Problems

    Get PDF
    International audienceSzeliski et al. published an influential study in 2006 on energy minimization methods for Markov Random Fields (MRF). This study provided valuable insights in choosing the best optimization technique for certain classes of problems. While these insights remain generally useful today, the phenomenal success of random field models means that the kinds of inference problems that have to be solved changed significantly. Specifically , the models today often include higher order interactions, flexible connectivity structures, large label-spaces of different car-dinalities, or learned energy tables. To reflect these changes, we provide a modernized and enlarged study. We present an empirical comparison of more than 27 state-of-the-art optimization techniques on a corpus of 2,453 energy minimization instances from diverse applications in computer vision. To ensure reproducibility, we evaluate all methods in the OpenGM 2 framework and report extensive results regarding runtime and solution quality. Key insights from our study agree with the results of Szeliski et al. for the types of models they studied. However, on new and challenging types of models our findings disagree and suggest that polyhedral methods and integer programming solvers are competitive in terms of runtime and solution quality over a large range of model types
    • 

    corecore