Gionfrida, Letizia

Guzman, Edgar

Howe, Robert D.

Nuckols, Richard W.

Walsh , Conor J.

English

King's Research Portal

                       King’s Research Portal  Document VersionOther versionLink to publication record in King's Research PortalCitation for published version (APA):Nuckols, R. W., Guzman, E., Gionfrida, L., Walsh , C. J., & Howe, R. D. (2023). Environmental Sensing andUltrasound for Adaptive Assistance, Computer Vision. In Wearable Robotics Workshop at IEEE RASInternational Conference in Robotics and Automation (ICRA) 2023 IEEE.Citing this paperPlease note that where the full-text provided on King's Research Portal is the Author Accepted Manuscript or Post-Print version this maydiffer from the final Published version. If citing, it is advised that you check and use the publisher's definitive version for pagination,volume/issue, and date of publication details. And where the final published version is provided on the Research Portal, if citing you areagain advised to check the publisher's website for any subsequent corrections.General rightsCopyright and moral rights for the publications made accessible in the Research Portal are retained by the authors and/or other copyrightowners and it is a condition of accessing publications that users recognize and abide by the legal requirements associated with these rights.•Users may download and print one copy of any publication from the Research Portal for the purpose of private study or research.•You may not further distribute the material or use it for any profit-making activity or commercial gain•You may freely distribute the URL identifying the publication in the Research PortalTake down policyIf you believe that this document breaches copyright please contact librarypure@kcl.ac.uk providing details, and we will remove access tothe work immediately and investigate your claim.Download date: 07. Sep. 2024    Abstract— This work presents a pipeline to generate assistance during different walking tasks. The pipeline uses an RGB-D camera to detect walking velocity, classifies level ground and ramps with a semantic classifier (with a detection accuracy of 91% and 95%, respectively), and selects tailored assistance by referring to previously created profiles from B-mode ultrasound. While the previous B-mode needed training of the users’ walking to generate individualized assistance, this pipeline uses a camera to factor human intent and guide individuals as they approach new tasks. Overall, the presented pipeline shows promising potential for vision to enhance in-vivo muscle-driven assistance in wearables.    I. INTRODUCTION Appropriate robotic assistance  during walking is influenced by the environment, the wearer’s physiology, and the walking task [1]. Previous, we used B-mode ultrasound images of the calf muscle to generate muscle-based assistance that was user and tasks specific for level walking at multiple speeds (1.0, 1.25, 1.5, and 1.75 m/s) and incline walking (5.71°) [2]. However, the set-up required tuning specific to the walking task and missed the opportunity to assist the muscle during different operation modes. Our proposed pipeline (Fig. 1) identifies different walking tasks using an RGBD camera, to then update the appropriate muscle-based assistance profile from B-mode ultrasound. This paper examines the calibration and operation modes of the proposed approach, utilizing RGBD sensing to ensure compatibility with a wide range of tasks, with plans to include live recordings of B-mode ultrasound in future investigations. II. METHODS We collected 1,451 RGB and depth image frames using Intel RealSense D435i camera mounted on the individuals’ chests. From the RGBD frames, we generated point cloud data and downsampled to remove outliers. Walking velocity was approximated from dividing distance by time. An existing architecture (PointNet) pre-trained in simulation [3], was evaluated on indoor ramps. Once the walk task was identified (i.e., walking speed, presence of the ramp), the exosuit selected the appropriate tailored assistance from the B-mode ultrasound at different   speeds on level ground and at incline.  Fig. 1.  Pipeline where RGBD and B-mode ultrasound are combinedin the calibration mode, while not in operation mode.  III. RESULTS AND DISCUSSION PointNet classified ramps and level ground terrain with an overall accuracy of 91% and 95% classification accuracy. The precision values are lower than in simulated scenarios, previously presented [3]. This may be due to lighting, camera placement, and user variability, which will be evaluated in future investigations.  Previous studies found vision-based environmental awareness inadequate for determining proper assistance, while ultrasound-generated profiles can overcome this limitation [2]. This approach generates task-specific assistance profiles, minimizing the need for B-mode ultrasound during operation, except for initial calibration. In future studies, we plan to enhance our approach by evaluating different ramp angles (0°-90°) and walking distances, as well as declines, curbs, and stairs. Finally, with ramp identification, we will incorporate tailored assistance profiles from live B-mode ultrasound recordings. REFERENCES [1] Zhang et al., arXiv preprint arXiv:1903.07674, 2019. [2] Nuckols et al., Science Robotics, vol. 6, no. 60, p. eabj1362, 2021. [3] Turgut and Kaleci, IJISAE, vol. 7, no. 3, pp. 159–165, 2019.   Richard W. Nuckols, Edgar Guzman, Letizia Gionfrida, Conor J. Walsh and Robert D. Howe Environmental Sensing and Ultrasound for Adaptive Assistance  *Research supported by the National Institutes of Health (NIH Grant Number: 5R21AR076686-02).  R. W. Nuckols is with Department of Systems Design Engineering University of Waterloo, University Ave W Waterloo, ON N2L 3G1 (e-mail: rnuckols@uwaterloo.ca).  E. Guzman, L. Gionfrida C.J. Walsh and R.D. Howe are with the Harvard Paulson School of Engineering and Applied Sciences, 150 Western Ave, Boston, MA 02134, USA. (e-mail: {eguzman, gionfrida, walsh, howe}@seas.harvard.edu).  L. Gionfrida is with the Department of Informatics, King's College London, Strand, London, WC2R 2LS, UK (e-mail: letizia.gionfrida@kcl.ac.uk).  

Environmental Sensing and Ultrasound for Adaptive Assistance, Computer Vision

https://kclpure.kcl.ac.uk/portal/files/284190761/CVWR-2023_paper_10_1_.pdf

Environmental Sensing and Ultrasound for Adaptive Assistance, Computer Vision

Abstract

Similar works

Full text

Available Versions

King's Research Portal