Modeling Humans at Rest with Applications to Robot Assistance

Humans spend a large part of their lives resting. Machine perception of this class of body poses would be beneficial to numerous applications, but it is complicated by line-of-sight occlusion from bedding. Pressure sensing mats are a promising alternative, but data is challenging to collect at scale. To overcome this, we use modern physics engines to simulate bodies resting on a soft bed with a pressure sensing mat. This method can efficiently generate data at scale for training deep neural networks. We present a deep model trained on this data that infers 3D human pose and body shape from a pressure image, and show that it transfers well to real world data. We also present a model that infers pose, shape and contact pressure from a depth image facing the person in bed, and it does so in the presence of blankets. This model similarly benefits from synthetic data, which is created by simulating blankets on the bodies in bed. We evaluate this model on real world data and compare it to an existing method that requires RGB, depth, thermal and pressure imagery in the input. Our model only requires an input depth image, yet it is 12% more accurate. Our methods are relevant to applications in healthcare, including patient acuity monitoring and pressure injury prevention. We demonstrate this work in the context of robotic caregiving assistance, by using it to control a robot to move to locations on a person’s body in bed.Ph.D

ULTRASONIC IMAGING AND TACTILE SENSING FOR ROBOTIC SYSTEMS

This research develops several novel algorithms that enhance the operation of ultrasonic and tactile sensors for robotic applications. The emphasis is on reducing the overall cost, system complexity, and enabling operation on resource-constrained embedded devices with the main focus on ultrasonics. The research improves key performance characteristics of pulse-echo sensor systems -- the minimum range, range resolution, and multi-object localization. The former two aspects are improved through the application of model-based and model-free techniques. Time optimal principles precisely control the oscillations of transmitting and receiving ultrasonic transducers, influencing the shape of the pressure waves. The model-free approach develops simple learning procedures to manipulate transducer oscillations, resulting in algorithms that are insensitive to parameter variations. Multi-object localization is achieved through phased array techniques that determine the positions of reflectors in 3-D space using a receiver array consisting of a small number of elements. The array design and the processing algorithm allow simultaneous determination of the reflector positions, achieving high sensor throughputs. Tactile sensing is a minor focus of this research that leverages machine learning in combination with an exploratory procedure to estimate the unknown stiffness of a grasped object. Gripper mechanisms with full-actuation and under-actuation are studied, and the object stiffness is estimated using regression. Sensor measurements use actuator position and current as the inputs. Regressor design, dataset generation, and the estimation performance under nonlinear effects, such as dry friction, parameter variations, and under-actuated transmission mechanisms are addressed.Ph.D