Force/Torque Sensing for Soft Grippers using an External Camera

Robotic manipulation can benefit from wrist-mounted force/torque (F/T) sensors, but conventional F/T sensors can be expensive, difficult to install, and damaged by high loads. We present Visual Force/Torque Sensing (VFTS), a method that visually estimates the 6-axis F/T measurement that would be reported by a conventional F/T sensor. In contrast to approaches that sense loads using internal cameras placed behind soft exterior surfaces, our approach uses an external camera with a fisheye lens that observes a soft gripper. VFTS includes a deep learning model that takes a single RGB image as input and outputs a 6-axis F/T estimate. We trained the model with sensor data collected while teleoperating a robot (Stretch RE1 from Hello Robot Inc.) to perform manipulation tasks. VFTS outperformed F/T estimates based on motor currents, generalized to a novel home environment, and supported three autonomous tasks relevant to healthcare: grasping a blanket, pulling a blanket over a manikin, and cleaning a manikin's limbs. VFTS also performed well with a manually operated pneumatic gripper. Overall, our results suggest that an external camera observing a soft gripper can perform useful visual force/torque sensing for a variety of manipulation tasks.Comment: Accepted for presentation at 2023 IEEE International Conference on Robotics and Automation (ICRA

Visual Contact Pressure Estimation for Grippers in the Wild

Sensing contact pressure applied by a gripper can benefit autonomous and teleoperated robotic manipulation, but adding tactile sensors to a gripper's surface can be difficult or impractical. If a gripper visibly deforms, contact pressure can be visually estimated using images from an external camera that observes the gripper. While researchers have demonstrated this capability in controlled laboratory settings, prior work has not addressed challenges associated with visual pressure estimation in the wild, where lighting, surfaces, and other factors vary widely. We present a model and associated methods that enable visual pressure estimation under widely varying conditions. Our model, Visual Pressure Estimation for Robots (ViPER), takes an image from an eye-in-hand camera as input and outputs an image representing the pressure applied by a soft gripper. Our key insight is that force/torque sensing can be used as a weak label to efficiently collect training data in settings where pressure measurements would be difficult to obtain. When trained on this weakly labeled data combined with fully labeled data that includes pressure measurements, ViPER outperforms prior methods, enables precision manipulation in cluttered settings, and provides accurate estimates for unseen conditions relevant to in-home use.Comment: Accepted for presentation at the 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2023

ForceSight: Text-Guided Mobile Manipulation with Visual-Force Goals

We present ForceSight, a system for text-guided mobile manipulation that predicts visual-force goals using a deep neural network. Given a single RGBD image combined with a text prompt, ForceSight determines a target end-effector pose in the camera frame (kinematic goal) and the associated forces (force goal). Together, these two components form a visual-force goal. Prior work has demonstrated that deep models outputting human-interpretable kinematic goals can enable dexterous manipulation by real robots. Forces are critical to manipulation, yet have typically been relegated to lower-level execution in these systems. When deployed on a mobile manipulator equipped with an eye-in-hand RGBD camera, ForceSight performed tasks such as precision grasps, drawer opening, and object handovers with an 81% success rate in unseen environments with object instances that differed significantly from the training data. In a separate experiment, relying exclusively on visual servoing and ignoring force goals dropped the success rate from 90% to 45%, demonstrating that force goals can significantly enhance performance. The appendix, videos, code, and trained models are available at https://force-sight.github.io/

Ponderomotive effects in multiphoton pair production

The Dirac-Heisenberg-Wigner formalism is employed to investigate electron-positron pair production in cylindrically symmetric but otherwise spatially inhomogeneous, oscillating electric fields. The oscillation frequencies are hereby tuned to obtain multiphoton pair production in the nonperturbative threshold regime. An effective mass as well as a trajectory-based semi-classical analysis are introduced in order to interpret the numerical results for the distribution functions as well as for the particle yields and spectra. The results, including the asymptotic particle spectra, display clear signatures of ponderomotive forces.Comment: 9 pages, 3 Tables, 3 Figure

Adaptive model reference control of highly maneuverable high performance aircraft

This thesis presents an adaptive model reference controller for a highly maneuverable high performance aircraft, in particular, a modified F18. An adaptive controller is developed to maneuver an aircraft at a high angle of attack. Thus, the aircraft is required to fly over a highly nonlinear flight regime. The adaptive controller presented in this thesis can be viewed as a combination of a linear and a nonlinear controller. Around a fixed flight condition the adaptive controller converges to a linear controller; however, the controller remains a nonlinear controller during maneuvers. The contributions of this thesis lie in two areas. The first area is in control. A successful application of linear adaptive control is presented for a highly nonlinear system. A new method is used to generate the reference trajectory. The reference model uses output feedback to improve the reference trajectory. It is shown that this improvement is necessary because of the control limitations. This work is also important to the control of highly maneuverable high performance aircraft. A successful adaptive controller has been developed to rapidly maneuver an aircraft to a high angle of attack. The main focus of this thesis is adaptive control

Trend analysis of tuberculosis case notifications with scale-up of antiretroviral therapy and roll-out of isoniazid preventive therapy in Zimbabwe, 2000-2018.

OBJECTIVES: Antiretroviral therapy (ART) and isoniazid preventive therapy (IPT) are known to have a tuberculosis (TB) protective effect at the individual level among people living with HIV (PLHIV). In Zimbabwe where TB is driven by HIV infection, we have assessed whether there is a population-level association between IPT and ART scale-up and annual TB case notification rates (CNRs) from 2000 to 2018. DESIGN: Ecological study using aggregate national data. SETTING: Annual aggregate national data on TB case notification rates (stratified by TB category and type of disease), numbers (and proportions) of PLHIV in ART care and of these, numbers (and proportions) ever commenced on IPT. RESULTS: ART coverage in the public sector increased from 1.1 million PLHIV patients) by December 2018, while IPT coverage among PLHIV in ART care increased from <1% (98 PLHIV) in 2012 to ~33% (373 917 PLHIV) by December 2018. These HIV-related interventions were associated with significant declines in TB CNRs: between the highest CNR prior to national roll-out of ART (in 2004) to the lowest recorded CNR after national IPT roll-out from 2012, these were (1) for all TB case (510 to 173 cases/100 000 population; 66% decline, p<0.001); (2) for those with new TB (501 to 159 cases/100 000 population; 68% decline, p<0.001) and (3) for those with new clinically diagnosed PTB (284 to 63 cases/100 000 population; 77.8% decline, p<0.001). CONCLUSIONS: This study shows the population-level impact of the continued scale-up of ART among PLHIV and the national roll-out of IPT among those in ART care in reducing TB, particularly clinically diagnosed TB which is largely associated with HIV. There are further opportunities for continued mitigation of TB with increasing coverage of ART and in particular IPT which still has a low coverage

Visual Estimation of Fingertip Pressure on Diverse Surfaces using Easily Captured Data

People often use their hands to make contact with the world and apply pressure. Machine perception of this important human activity could be widely applied. Prior research has shown that deep models can estimate hand pressure based on a single RGB image. Yet, evaluations have been limited to controlled settings, since performance relies on training data with high-resolution pressure measurements that are difficult to obtain. We present a novel approach that enables diverse data to be captured with only an RGB camera and a cooperative participant. Our key insight is that people can be prompted to perform actions that correspond with categorical labels describing contact pressure (contact labels), and that the resulting weakly labeled data can be used to train models that perform well under varied conditions. We demonstrate the effectiveness of our approach by training on a novel dataset with 51 participants making fingertip contact with instrumented and uninstrumented objects. Our network, ContactLabelNet, dramatically outperforms prior work, performs well under diverse conditions, and matched or exceeded the performance of human annotators

Dispersion Relations and Rescattering Effects in B Nonleptonic Decays

Recently, the final state strong interactions in nonleptonic B decays were investigated in a formalism based on hadronic unitarity and dispersion relations in terms of the off-shell mass squared of the $B$ meson. We consider an heuristic derivation of the dispersion relations in the mass variables using the reduction LSZ formalism and find a discrepancy between the spectral function and the dispersive variable used in the recent works. The part of the unitarity sum which describes final state interactions is shown to appear as spectral function in a dispersion relation based on the analytic continuation in the mass squared of one final particles. As an application, by combining this formalism with Regge theory and SU(3) flavour symmetry we obtain constraints on the tree and the penguin amplitudes of the decay $B^0\to \pi^+\pi^-$.Comment: 17 pages, Latex, 2 figure