Terrain Classification from Body-mounted Cameras during Human Locomotion

Abstract—This paper presents a novel algorithm for terrain type classification based on monocular video captured from the viewpoint of human locomotion. A texture-based algorithm is developed to classify the path ahead into multiple groups that can be used to support terrain classification. Gait is taken into account in two ways. Firstly, for key frame selection, when regions with homogeneous texture characteristics are updated, the fre-quency variations of the textured surface are analysed and used to adaptively define filter coefficients. Secondly, it is incorporated in the parameter estimation process where probabilities of path consistency are employed to improve terrain-type estimation. When tested with multiple classes that directly affect mobility a hard surface, a soft surface and an unwalkable area- our proposed method outperforms existing methods by up to 16%, and also provides improved robustness. Index Terms—texture, classification, recursive filter, terrain classification I

Comparative Study of Different Methods in Vibration-Based Terrain Classification for Wheeled Robots with Shock Absorbers

open access articleAutonomous robots that operate in the field can enhance their security and efficiency by accurate terrain classification, which can be realized by means of robot-terrain interaction-generated vibration signals. In this paper, we explore the vibration-based terrain classification (VTC), in particular for a wheeled robot with shock absorbers. Because the vibration sensors are usually mounted on the main body of the robot, the vibration signals are dampened significantly, which results in the vibration signals collected on different terrains being more difficult to discriminate. Hence, the existing VTC methods applied to a robot with shock absorbers may degrade. The contributions are two-fold: (1) Several experiments are conducted to exhibit the performance of the existing feature-engineering and feature-learning classification methods; and (2) According to the long short-term memory (LSTM) network, we propose a one-dimensional convolutional LSTM (1DCL)-based VTC method to learn both spatial and temporal characteristics of the dampened vibration signals. The experiment results demonstrate that: (1) The feature-engineering methods, which are efficient in VTC of the robot without shock absorbers, are not so accurate in our project; meanwhile, the feature-learning methods are better choices; and (2) The 1DCL-based VTC method outperforms the conventional methods with an accuracy of 80.18%, which exceeds the second method (LSTM) by 8.23%

An Exploratory Approach to Manipulating Dynamic Stability: Investigating the Role of Visual Control during a Precision Foot Placement Task

ABSTRACT Background: The visual system provides the body with an accurate sensory system; designed to gather information at a distance and acts as a feedforward control mechanism during human locomotion. By doing so, visual information contributes coordination of the head-arm-trunk (HAT) segment and modulating foot placement. The purpose of this study was to examine the effects of a constrained pathway during a complex navigational stone-stepping task on HAT segment control and how the visual system guides locomotion during a complex foot placement task. Methods: Nine university-aged females (Mean age: 22.5 years old +/-1.75) participated in this study. Participants were instrumented with four rigid bodies (4x3 IRED markers) on the head, trunk and feet and two IRED markers on the wrists in order to measure kinematic data, collected by Optotrak system (NDI, Waterloo, Canada). Additionally, each participant was outfitted with an ASL H7-HS High Speed Head Mounted Optics (ASL, Bedford, USA) eye tracking unit to assess gaze behaviours. The experimental protocol required participants to perform 40 walking trials across four conditions (i.e., constrained and self-selected pathways; starting with either the left or the right foot), on a 7.2m x 1.2m raised-target platform. The platform consisted of 60 sloper-style rock climbing holds, whose location was designed to satisfy one of three criterion: 1) in line with natural footfall locations (e.g. normal step length and/or width dimensions of 60cm by 10cm); 2) greater or less than one of the dimensions of a natural step length or width; or 3) to act as a possible option/distractor on the pathway. The two constrained pathways were indicated with a high-contrasting moldable material placed inside each hold’s screw hole. Measurements were compared across conditions (i.e., constrained versus unconstrained), time points (e.g. first, middle, and last trial performed of each condition), and segment (Segment 1: first 3m of path or Segment 2: last 3m of path). The measurements included: horizontal and vertical pupil velocity RMS; average walking speed; trunk rotations about the hip (i.e., pitch and roll), and whole-body movement (i.e., ML COM variability). Results: Findings revealed that there was a significant difference between conditions such that: 1) the constrained vertical pupil RMS velocity was higher than the unconstrained (F(3,24)=4.71; p= .04; d=.46); 2) the unconstrained horizontal pupil RMS velocity was higher than the unconstrained (F(3,24)=4.40; p= .03; d=.36); 3) the constrained average walking speed was greater than the unconstrained (F(3,24)=23.27; p=0.04; d=.30); 4) the constrained trunk roll was greater than the unconstrained (F(3,21)=4.84; p=0.01; d=.45); and 5) the unconstrained dynamic stability margin minimum (DSMmin) was greater than the constrained (F(3,21)=4.89; p= .01; d=.41). Conclusions: The complex nature of the raised-target foot placement task challenged individuals from the start of each condition, forcing participants to learn how to control body movements—especially in the AP direction. During constrained condition, there was evidence to suggest that there was a greater regulation of trunk control than during unconstrained trials. This was attributed to the conditional demands of predetermined pathway to follow. However, during unconstrained trials, individuals were able to choose footholds, which were most likely based on their current state of stability. And thus, conditional demands of the pathway influenced gaze behaviours, such that during the constrained condition participants used a scanning behaviour (i.e., greater vertical pupil velocity RMS) whereas participants used more of a sampling behaviour (i.e., greater horizontal and slower vertical pupil velocities) during the free choice pathway condition. Therefore, the finding from this study suggest that gaze behaviours are influenced by stepping characteristics and these different gaze behaviours have different effects on trunk control

An Exploratory Approach to Manipulating Dynamic Stability: Investigating the Role of Visual Control during a Precision Foot Placement Task

ABSTRACT Background: The visual system provides the body with an accurate sensory system; designed to gather information at a distance and acts as a feedforward control mechanism during human locomotion. By doing so, visual information contributes coordination of the head-arm-trunk (HAT) segment and modulating foot placement. The purpose of this study was to examine the effects of a constrained pathway during a complex navigational stone-stepping task on HAT segment control and how the visual system guides locomotion during a complex foot placement task. Methods: Nine university-aged females (Mean age: 22.5 years old +/-1.75) participated in this study. Participants were instrumented with four rigid bodies (4x3 IRED markers) on the head, trunk and feet and two IRED markers on the wrists in order to measure kinematic data, collected by Optotrak system (NDI, Waterloo, Canada). Additionally, each participant was outfitted with an ASL H7-HS High Speed Head Mounted Optics (ASL, Bedford, USA) eye tracking unit to assess gaze behaviours. The experimental protocol required participants to perform 40 walking trials across four conditions (i.e., constrained and self-selected pathways; starting with either the left or the right foot), on a 7.2m x 1.2m raised-target platform. The platform consisted of 60 sloper-style rock climbing holds, whose location was designed to satisfy one of three criterion: 1) in line with natural footfall locations (e.g. normal step length and/or width dimensions of 60cm by 10cm); 2) greater or less than one of the dimensions of a natural step length or width; or 3) to act as a possible option/distractor on the pathway. The two constrained pathways were indicated with a high-contrasting moldable material placed inside each hold’s screw hole. Measurements were compared across conditions (i.e., constrained versus unconstrained), time points (e.g. first, middle, and last trial performed of each condition), and segment (Segment 1: first 3m of path or Segment 2: last 3m of path). The measurements included: horizontal and vertical pupil velocity RMS; average walking speed; trunk rotations about the hip (i.e., pitch and roll), and whole-body movement (i.e., ML COM variability). Results: Findings revealed that there was a significant difference between conditions such that: 1) the constrained vertical pupil RMS velocity was higher than the unconstrained (F(3,24)=4.71; p= .04; d=.46); 2) the unconstrained horizontal pupil RMS velocity was higher than the unconstrained (F(3,24)=4.40; p= .03; d=.36); 3) the constrained average walking speed was greater than the unconstrained (F(3,24)=23.27; p=0.04; d=.30); 4) the constrained trunk roll was greater than the unconstrained (F(3,21)=4.84; p=0.01; d=.45); and 5) the unconstrained dynamic stability margin minimum (DSMmin) was greater than the constrained (F(3,21)=4.89; p= .01; d=.41). Conclusions: The complex nature of the raised-target foot placement task challenged individuals from the start of each condition, forcing participants to learn how to control body movements—especially in the AP direction. During constrained condition, there was evidence to suggest that there was a greater regulation of trunk control than during unconstrained trials. This was attributed to the conditional demands of predetermined pathway to follow. However, during unconstrained trials, individuals were able to choose footholds, which were most likely based on their current state of stability. And thus, conditional demands of the pathway influenced gaze behaviours, such that during the constrained condition participants used a scanning behaviour (i.e., greater vertical pupil velocity RMS) whereas participants used more of a sampling behaviour (i.e., greater horizontal and slower vertical pupil velocities) during the free choice pathway condition. Therefore, the finding from this study suggest that gaze behaviours are influenced by stepping characteristics and these different gaze behaviours have different effects on trunk control

Kinematic primitives for walking and trotting gaits of a quadruped robot with compliant legs

In this work we research the role of body dynamics in the complexity of kinematic patterns in a quadruped robot with compliant legs. Two gait patterns, lateral sequence walk and trot, along with leg length control patterns of different complexity were implemented in a modular, feed-forward locomotion controller. The controller was tested on a small, quadruped robot with compliant, segmented leg design, and led to self-stable and self-stabilizing robot locomotion. In-air stepping and on-ground locomotion leg kinematics were recorded, and the number and shapes of motion primitives accounting for 95% of the variance of kinematic leg data were extracted. This revealed that kinematic patterns resulting from feed-forward control had a lower complexity (in-air stepping, 2 to 3 primitives) than kinematic patterns from on-ground locomotion (4 primitives), although both experiments applied identical motor patterns. The complexity of on-ground kinematic patterns had increased, through ground contact and mechanical entrainment. The complexity of observed kinematic on-ground data matches those reported from level-ground locomotion data of legged animals. Results indicate that a very low complexity of modular, rhythmic, feed-forward motor control is sufficient for level-ground locomotion in combination with passive compliant legged hardware

Learning to See Physical Properties with Active Sensing Motor Policies

Knowledge of terrain's physical properties inferred from color images can aid in making efficient robotic locomotion plans. However, unlike image classification, it is unintuitive for humans to label image patches with physical properties. Without labeled data, building a vision system that takes as input the observed terrain and predicts physical properties remains challenging. We present a method that overcomes this challenge by self-supervised labeling of images captured by robots during real-world traversal with physical property estimators trained in simulation. To ensure accurate labeling, we introduce Active Sensing Motor Policies (ASMP), which are trained to explore locomotion behaviors that increase the accuracy of estimating physical parameters. For instance, the quadruped robot learns to swipe its foot against the ground to estimate the friction coefficient accurately. We show that the visual system trained with a small amount of real-world traversal data accurately predicts physical parameters. The trained system is robust and works even with overhead images captured by a drone despite being trained on data collected by cameras attached to a quadruped robot walking on the ground.Comment: In CoRL 2023. Website: https://gmargo11.github.io/active-sensing-loco