Improved data association and occlusion handling for vision-based people tracking by mobile robots

This paper presents an approach for tracking multiple persons using a combination of colour and thermal vision sensors on a mobile robot. First, an adaptive colour model is incorporated into the measurement model of the tracker. Second, a new approach for detecting occlusions is introduced, using a machine learning classifier for pairwise comparison of persons (classifying which one is in front of the other). Third, explicit occlusion handling is then incorporated into the tracker

Data association and occlusion handling for vision-based people tracking by mobile robots

This paper presents an approach for tracking multiple persons on a mobile robot with a combination of colour and thermal vision sensors, using several new techniques. First, an adaptive colour model is incorporated into the measurement model of the tracker. Second, a new approach for detecting occlusions is introduced, using a machine learning classifier for pairwise comparison of persons (classifying which one is in front of the other). Third, explicit occlusion handling is incorporated into the tracker. The paper presents a comprehensive, quantitative evaluation of the whole system and its different components using several real world data sets

Pictures in Your Mind: Using Interactive Gesture-Controlled Reliefs to Explore Art

Tactile reliefs offer many benefits over the more classic raised line drawings or tactile diagrams, as depth, 3D shape, and surface textures are directly perceivable. Although often created for blind and visually impaired (BVI) people, a wider range of people may benefit from such multimodal material. However, some reliefs are still difficult to understand without proper guidance or accompanying verbal descriptions, hindering autonomous exploration. In this work, we present a gesture-controlled interactive audio guide (IAG) based on recent low-cost depth cameras that can be operated directly with the hands on relief surfaces during tactile exploration. The interactively explorable, location-dependent verbal and captioned descriptions promise rapid tactile accessibility to 2.5D spatial information in a home or education setting, to online resources, or as a kiosk installation at public places. We present a working prototype, discuss design decisions, and present the results of two evaluation studies: the first with 13 BVI test users and the second follow-up study with 14 test users across a wide range of people with differences and difficulties associated with perception, memory, cognition, and communication. The participant-led research method of this latter study prompted new, significant and innovative developments

Sparse Inertial Poser: Automatic 3D Human Pose Estimation from Sparse IMUs

We address the problem of making human motion capture in the wild more practical by using a small set of inertial sensors attached to the body. Since the problem is heavily under-constrained, previous methods either use a large number of sensors, which is intrusive, or they require additional video input. We take a different approach and constrain the problem by: (i) making use of a realistic statistical body model that includes anthropometric constraints and (ii) using a joint optimization framework to fit the model to orientation and acceleration measurements over multiple frames. The resulting tracker Sparse Inertial Poser (SIP) enables 3D human pose estimation using only 6 sensors (attached to the wrists, lower legs, back and head) and works for arbitrary human motions. Experiments on the recently released TNT15 dataset show that, using the same number of sensors, SIP achieves higher accuracy than the dataset baseline without using any video data. We further demonstrate the effectiveness of SIP on newly recorded challenging motions in outdoor scenarios such as climbing or jumping over a wall.Comment: 12 pages, Accepted at Eurographics 201

The Impact of Extreme Virtual Elevation above Grade on Construction Workers\u27 Physiological Responses, Physical Responses, and Task Performance

On average, in every two work hours, one person dies from work-related injuries at construction sites. Most incidents are due to falling from elevated surfaces. Slips, trips, and loss of balance are the main causes. Studies suggest that instigating visual mismatch and physiological changes are among the most important reasons behind falling from narrow elevated surfaces. By using advanced virtual reality models, this dissertation aims to highlight some of the possible effects of a destabilizing environment (i.e., elevation above grade) on workers’ physiological responses and task performance. More specifically, this dissertation strives to find potential effects of elevation above grade and a moving structural beam as destabilizing environments on construction workers’ postural sway, gait pattern, and task performance accuracy. To that end, a series of virtual reality experiments was conducted on thirty volunteers, all students from the University of Nebraska - Lincoln. There were three required VR tasks asked from the subjects once on the ground and again on the 20th floor of an unfinished building: walking on virtual structural beams, standing still on the virtual platform (force plate in reality), and performing hand-steadiness and pursuit tests (physiological battery tests). In addition, to study the plausible relationship between self-perceived fear (and acrophobia) and physiological responses, all subjects were instructed to complete the electronic James Geer’s fear and Cohen’s acrophobia questionnaires. The result of this study showed that elevation above grade has a substantial effect on the gait pattern. More specifically, exposure to elevation increases gait stride height variability and decreases gait stride length. As a result, subjects spend more time on gait tasks executed on narrow elevated surfaces. Also, the findings indicated that the presence of the virtual avatar significantly affects gait parameters. The presence of synchronized virtual legs caused subjects to increase their stride height and spend more time on similar virtual tasks on the ground. However, the subjects did not exhibit similar differences once exposed to virtual elevation. Furthermore, the moving structural beam significantly increased the heart rate of the subjects. As part of the steel erection simulation, the experimental results implied that construction workers could show noticeable physiological responses in the vicinity of large moving objects. In terms of task performance, working at height affects the result of the posturography and battery tests. This finding suggests that dual-tasks performed in a static position, and in the presence of elevation-related visual stimuli, can cause a reduction in the postural sway. In contrast, in the absence of visual depth, fear of height can positively influence the outcome of the construction tasks performed on elevated platforms. Advisors: Jay Puckett and Terry Stent

A multi-modal person perception framework for socially interactive mobile service robots

In order to meet the increasing demands of mobile service robot applications, a dedicated perception module is an essential requirement for the interaction with users in real-world scenarios. In particular, multi sensor fusion and human re-identification are recognized as active research fronts. Through this paper we contribute to the topic and present a modular detection and tracking system that models position and additional properties of persons in the surroundings of a mobile robot. The proposed system introduces a probability-based data association method that besides the position can incorporate face and color-based appearance features in order to realize a re-identification of persons when tracking gets interrupted. The system combines the results of various state-of-the-art image-based detection systems for person recognition, person identification and attribute estimation. This allows a stable estimate of a mobile robot’s user, even in complex, cluttered environments with long-lasting occlusions. In our benchmark, we introduce a new measure for tracking consistency and show the improvements when face and appearance-based re-identification are combined. The tracking system was applied in a real world application with a mobile rehabilitation assistant robot in a public hospital. The estimated states of persons are used for the user-centered navigation behaviors, e.g., guiding or approaching a person, but also for realizing a socially acceptable navigation in public environments

A study to identify and compare airborne systems for in-situ measurements of launch vehicle effluents

An in-situ system for monitoring the concentration of HCl, CO, CO2, and Al2O3 in the cloud of reaction products that form as a result of a launch of solid propellant launch vehicle is studied. A wide array of instrumentation and platforms are reviewed to yield the recommended system. An airborne system suited to monitoring pollution concentrations over urban areas for the purpose of calibrating remote sensors is then selected using a similar methodology to yield the optimal configuration

The Design and Evaluation of a Kinect-Based Postural Symmetry Assessment and Training System

abstract: The increased risk of falling and the worse ability to perform other daily physical activities in the elderly cause concern about monitoring and correcting basic everyday movement. In this thesis, a Kinect-based system was designed to assess one of the most important factors in balance control of human body when doing Sit-to-Stand (STS) movement: the postural symmetry in mediolateral direction. A symmetry score, calculated by the data obtained from a Kinect RGB-D camera, was proposed to reflect the mediolateral postural symmetry degree and was used to drive a real-time audio feedback designed in MAX/MSP to help users adjust themselves to perform their movement in a more symmetrical way during STS. The symmetry score was verified by calculating the Spearman correlation coefficient with the data obtained from Inertial Measurement Unit (IMU) sensor and got an average value at 0.732. Five healthy adults, four males and one female, with normal balance abilities and with no musculoskeletal disorders, were selected to participate in the experiment and the results showed that the low-cost Kinect-based system has the potential to train users to perform a more symmetrical movement in mediolateral direction during STS movement.Dissertation/ThesisMasters Thesis Electrical Engineering 201

Pedestrian Detection using Triple Laser Range Finders

Pedestrian detection is one of the important features in autonomous ground vehicle (AGV). It ensures the capability for safety navigation in urban environment. Therefore, the detection accuracy became a crucial part which leads to implementation using Laser Range Finder (LRF) for better data representation. In this study, an improved laser configuration and fusion technique is introduced by implementation of triple LRFs in two layers with Pedestrian Data Analysis (PDA) to recognize multiple pedestrians. The PDA integrates various features from feature extraction process for all clusters and fusion of multiple layers for better recognition. The experiments were conducted in various occlusion scenarios such as intersection, closed-pedestrian and combine scenarios. The analysis of the laser fusion and PDA for all scenarios showed an improvement of detection where the pedestrians were represented by various detection categories which solve occlusion issues when low numberof laser data were obtained