Interfaz multimodal para modelado, estudio y asistencia a la marcha humana mediante andadores robóticos

Texto en español y resumen en español e inglésEn esta tesis doctoral se presenta la concepción, implementación y validación de una interfaz multimodal para el modelado, estudio y asistencia a la marcha humana. Esta interfaz se ha integrado en un andador para conseguir un soporte activo, adaptado, seguro y de fácil manejo para ciertas personas con movilidad muy reducida, que de otro modo quedarían relegados a la silla de ruedas con los problemas fisiológicos que su uso conlleva. Inicialmente, se realiza una revisión del estado del arte de los dispositivos de ayuda a la movilidad y, en especial, un estudio crítico de los andadores. Entre estos dispositivos, se encuentra una clase de elementos conocidos como andadores avanzados o Smart Walkers, tema de interés en la actualidad de numerosos grupos de investigación en Estados Unidos, Europa y Japón. En este estudio, se realiza, además, una clasificación funcional de los andadores avanzados y un análisis detallado de los sistemas más significativos de la literatura en este campo. A partir de este análisis crítico inicial y de las carencias encontradas se propone y se presenta el andador SIMBIOSIS, desarrollo central del trabajo realizado en esta tesis doctoral y dentro del proyecto CYCIT (DPI-2005-07417) con el mismo acrónimo. Con el fin de cubrir las carencias mencionadas, se aborda en profundidad el estudio y la caracterización de la marcha asistida por andadores y la interacción usuario-andador con el fin no solo de conocer mejor el proceso en sí, sino también de utilizar esta información para la obtención de técnicas de procesamiento de señales y de desarrollo de estrategias de control y conducción del andador de forma más natural y segura para el usuario. Con este enfoque, se trata de potenciar el uso de los andadores como herramienta eficiente de compensación funcional y de rehabilitación de la marcha humana. En este contexto se centran las principales aportaciones científicas de este trabajo: la caracterización de la marcha humana con el andador desarrollado, la descripción y el modelado de la interacción entre los dos agentes y su relación con el entorno, la obtención de una metodología de procesamiento de señales y de control centrada en el usuario y su interacción con el andador y, finalmente, la validación funcional del sistema desarrollado con usuarios. El sistema y los métodos desarrollados han sido efectivamente validados de manera experimental con pacientes lesionados medulares con distintos grados de afección y capacidades de locomoción. Los resultados, obtenidos tanto objetivos como subjetivos, se ha considerado satisfactorios en un alto grado y se presentan en la memoria, constatando el potencial del andador desarrollado no solo como elemento de compensación funcional para ciertos sujetos con afecciones de marcha, sino también como elemento rehabilitador para personas con diferentes disfunciones en sus miembros inferiores. Por todo ello, el resultado más significativo del trabajo de esta tesis doctoral ha sido la obtención de una herramienta segura y fiable de asistencia a personas con movilidad muy reducida

Review and classification of human gait training and rehabilitation devices

The number of people with reduced mobility capabilities increases every year. This reduction arises mainly due to spinal cord injuries; strokes which caused hemiparesis; or due to an advanced age. This decrease in mobility is a factor that influences both their quality of life and their dependence of others in daily life. Thus, it becomes necessary to find means and tools to prevent, compensate, improve or help to restore and increase the mobility of the affected people. The main expectation is that such means help to recover or ameliorate their independence in their daily life. Traditional training employs a treadmill with a support-weight system. This training is based on the principle of repetition of all the physical movements of a gait and has shown to produce good results in terms of rehabilitation of patients. However, this therapy requires two or more therapists in assisting patients during walking, to hold and adjust the patient’s lower limbs to correctly produce the desired gait. Thus, it requires a substantial commitment and effort of the therapists [1], and it is very expensive in terms of human resources. This leads to a boost on the population healthcare and assistive services demand and, thus an increase in the need for care givers. Assistive mobility robotic devices for gait training of disabled patients in treadmills and in the ground are one successful alternative. Other alternatives include devices that allow a broader training of patients, in different ground types, and the repetition of gait movements in uphill, downhill and trip. This paper reviews state of the art training gait devices focusing on passive and active devices. Passive devices rely on the principle of Gravity-Balancing in that they try to reduce or eliminate the effects of gravity during walking. Active devices are usually classified according to three different approaches: (i) treadmillexoskeleton based devices, (ii) robotic manipulators generating different types of gait patterns, and (iii) mobilite devices. In this review, several examples of current devices are presented

Multivariate analysis of walker-assisted ambulation

In an aging society it is extremely important to develop devices which can support and aid the elderly in their daily life. Walkers play an important role, due to the large number of potential users, its simplicity and their ambulatory potential. However, there are no clinical evidences that prove the efficacy of such devices, mainly rollators that present forearm supports. In this context, the authors aim to propose a protocol for an innovative gait analysis that addresses some benefits and limitations of these devices on the rehabilitation process, by addressing a multivariate analysis of spatiotemporal and kinematic gait parameters assessed during normal and assisted ambulation with a walker with forearm supports. For the 3Dreconstruction of the body segments it was used a movement analysis system. Results showed that the effects of assisted gait can be explained through support, energy consumption, posture and balance characteristics. These results are very satisfactory since aspects regarding these characteristics enhance the rehabilitation potential of the use of walkers with forearm supports. These results will be used to advance towards an active robotic walker that will provide for safety and natural manoeuvrability and offer a certain degree of intelligence in assistance and decision-making

Assistive mobility devices focusing on smart walkers : classification and review

In an aging society it is extremely important to develop devices, which can support and aid the elderly in their daily life. This demands means and tools that extend independent living and promote improved health. Thus, the goal of this article is to review the state of the art in the robotic technology for mobility assistive devices for people with mobility disabilities. The important role that robotics can play in mobility assistive devices is presented, as well as the identification and survey of mobility assistive devices subsystems with a particular focus on the walkers technology. The advances in the walkers’ field have been enormous and have shown a great potential on helping people with mobility disabilities. Thus it is presented a review of the available literature of walkers and are discussed major advances that have been made and limitations to be overcome

Revisão e classificação de dispositivos de treino e reabilitação da marcha humana

O treino de marcha convencional recorre a uma passadeira. Este tipo de treino comporta custos consideráveis, para além de exigir o constante envolvimento e esforço de dois ou três terapeutas. De modo a reduzir este esforço, inúmeros grupos de investigação têm desenvolvido uma série de alternativas a este método através da ajuda de dispositivos robóticos que realizam a assistência de pacientes na passadeira. Para além destes, têm surgido outros dispositivos que permitem um treino mais alargado dos pacientes no sentido em que permitem ao paciente executar a sua marcha no solo, bem como a repetição de movimentos de subida, descida e de tropeçar

Extraction of user's navigation commands from upper body force interaction in walker assisted gait

<p>Abstract</p> <p>Background</p> <p>The advances in technology make possible the incorporation of sensors and actuators in rollators, building safer robots and extending the use of walkers to a more diverse population. This paper presents a new method for the extraction of navigation related components from upper-body force interaction data in walker assisted gait. A filtering architecture is designed to cancel: (i) the high-frequency noise caused by vibrations on the walker's structure due to irregularities on the terrain or walker's wheels and (ii) the cadence related force components caused by user's trunk oscillations during gait. As a result, a third component related to user's navigation commands is distinguished.</p> <p>Results</p> <p>For the cancelation of high-frequency noise, a Benedict-Bordner g-h filter was designed presenting very low values for Kinematic Tracking Error ((2.035 ± 0.358)·10^-2<it>kgf</it>) and delay ((1.897 ± 0.3697)·10¹<it>ms</it>). A <it>Fourier Linear Combiner </it>filtering architecture was implemented for the adaptive attenuation of about 80% of the cadence related components' energy from force data. This was done without compromising the information contained in the frequencies close to such notch filters.</p> <p>Conclusions</p> <p>The presented methodology offers an effective cancelation of the undesired components from force data, allowing the system to extract in real-time voluntary user's navigation commands. Based on this real-time identification of voluntary user's commands, a classical approach to the control architecture of the robotic walker is being developed, in order to obtain stable and safe user assisted locomotion.</p

Extraction of user's navigation commands from upper body force interaction in walker assisted gait

<p>Abstract</p> <p>Background</p> <p>The advances in technology make possible the incorporation of sensors and actuators in rollators, building safer robots and extending the use of walkers to a more diverse population. This paper presents a new method for the extraction of navigation related components from upper-body force interaction data in walker assisted gait. A filtering architecture is designed to cancel: (i) the high-frequency noise caused by vibrations on the walker's structure due to irregularities on the terrain or walker's wheels and (ii) the cadence related force components caused by user's trunk oscillations during gait. As a result, a third component related to user's navigation commands is distinguished.</p> <p>Results</p> <p>For the cancelation of high-frequency noise, a Benedict-Bordner g-h filter was designed presenting very low values for Kinematic Tracking Error ((2.035 ± 0.358)·10^-2<it>kgf</it>) and delay ((1.897 ± 0.3697)·10¹<it>ms</it>). A <it>Fourier Linear Combiner </it>filtering architecture was implemented for the adaptive attenuation of about 80% of the cadence related components' energy from force data. This was done without compromising the information contained in the frequencies close to such notch filters.</p> <p>Conclusions</p> <p>The presented methodology offers an effective cancelation of the undesired components from force data, allowing the system to extract in real-time voluntary user's navigation commands. Based on this real-time identification of voluntary user's commands, a classical approach to the control architecture of the robotic walker is being developed, in order to obtain stable and safe user assisted locomotion.</p

Reconocimiento en-línea de acciones humanas basado en patrones de RWE aplicado en ventanas dinámicas de momentos invariantes

[EN] This paper presents a methodology for online human action recognition on video sequences. It addresses an efficient approach to use invariant moments as image descriptors, applied in processing silhouettes obtained from depth maps. A quick comparison between size-4 windows (equivalent to 4 frames) is performed by computing the Mahalanobis distance, on one of the invariant moment sequences identified as less sensitive to noise and more stable during movement absence. This approach is used for rapid detection of the idle/motion state, which allows the capture of dynamic growth intervals (windows) for further processing, rescuing from the signal contained their temporal and frequential properties. By applying the Haar wavelet transform, three decomposition levels are used for calculating Relative Wavelet Energy (RWE - Relative Wavelet Energy) and SSC (Slope Sign Change), obtaining 11-dimensional patterns. In experiments, 97 % of 4 movements online-captured were recognized correctly, and 10 movements taken from Muhavi-MAS database were recognized with 94.2 % efficiency[ES] En este trabajo se presenta una metodología para el reconocimiento en-línea de acciones humanas en secuencias de vídeo. Se aborda un enfoque eficiente para el uso de momentos invariantes como descriptores de imagen, aplicados en siluetas obtenidas del procesamiento de mapas de profundidad. Una comparación rápida entre ventanas de tamaño 4 (equivalente a 4 frames) es realizada mediante el cómputo de la distancia de Mahalanobis, sobre una de las secuencias de momentos invariantes identificada como la menos sensible al ruido de captura y la más estable durante ausencia de movimiento. Este enfoque es usado para la detección rápida del estado de parada/movimiento, el cual permite la captura de intervalos (ventanas) de crecimiento dinámico para su posterior procesamiento, rescatando de la señal contenida sus propiedades temporales y frecuenciales. Mediante la aplicación de la transformada Wavelet Haar, tres niveles de descomposición son utilizados para el cómputo de la Energía Relativa Wavelet (RWE - Relative Wavelet Energy) y SSC (Slope Sign Change), obteniendo patrones 11-dimensionales. En experimentos realizados, el 97% de 4 movimientos capturados en-línea fueron reconocidos correctamente, y 10 movimientos tomados de la base de datos Muhavi-MAS fueron reconocidos con 94,2% de efectividad.Este proyecto de investigacion es financiado por el Programa Primeros Proyectos, CNPq/FAPES No. 02/2011 y por el CNPq a traves de beca de doctorado para el primer autor.Romero López, D.; Frizera Neto, A.; Freire Bastos, T. (2014). Reconocimiento en-línea de acciones humanas basado en patrones de RWE aplicado en ventanas dinámicas de momentos invariantes. Revista Iberoamericana de Automática e Informática industrial. 11(2):202-211. https://doi.org/10.1016/j.riai.2013.09.009OJS202211112Antoniou, A., 2005. Digital Signal Processing. McGraw-Hill.Broggi, A., Bertozzi, M., Fascioli, A., Sechi, M., 2000. Shape-based pedestrian detection. In: Intelligent Vehicles Symposium, 2000. IV 2000. Proceedings of the IEEE. pp. 215-220.Chan, T., Vese, L., feb 2001. Active contours without edges. Image Processing, IEEE Transactions on 10 (2), 266-277.Chen, Q., Petriu, E., Yang, X., may 2004. A comparative study of fourier descriptors and hu's seven moment invariants for image recognition. In: Electrical and Computer Engineering, 2004. Canadian Conference on. Vol. 1. pp. 103-106 Vol.1.Chockalingam, P., Pradeep, N., Birchfield, S., oct. 2009. Adaptive fragments- based tracking of non-rigid objects using level sets. In: Computer Vision, 2009 IEEE 12th International Conference on. pp. 1530-1537.Cifuentes, C., Braidot, A., Rodriguez, L., Frisoli, M., Santiago, A., Frizera, A., june 2012. Development of a wearable zigbee sensor system for upper limb rehabilitation robotics. In: Biomedical Robotics and Biomechatronics (Bio- Rob), 2012 4th IEEE RAS EMBS International Conference on. pp. 1989-1994. DOI: 10.1109/BioRob.2012.6290926.Franke, U., Joos, A., 2000. Real-time stereo vision for urban traffic scene understanding. In: Intelligent Vehicles Symposium. Proceedings of the IEEE. pp. 273-278.Garcia-Costa, C., Egea-Lopez, E., Tomas-Gabarron, J., Garcia-Haro, J., Haas, Z., 2011. A stochastic model for chain collisions of vehicles equipped with vehicular communications. Intelligent Transportation Systems, IEEE Transactions on 13, 503-518.Geronimo, D., Lopez, A., Sappa, D., july 2010. Survey of pedestrian detection for advanced driver assistance systems. Pattern Analysis and Machine Intelligence, IEEE Transactions on 32 (7), 1239-1258.Gonzalez, R.C., 2010. Digital Image Processing, 2nd Edition. McGraw-Hill.Grubb, G., Zelinsky, A., Nilsson, L., Rilbe, M., june 2004. 3d vision sensing for improved pedestrian safety. In: Intelligent Vehicles Symposium, IEEE. pp. 19-24.Haibin, Z., Xu, W., Hong, W., may 2008. Feature selection using relative wavelet energy for brain-computer interface design. In: Bioinformatics and Biomedical Engineering, 2008. ICBBE 2008. The 2nd International Conference on. pp. 1434-1437.Hu, X., Kong, B., Zheng, F., Wang, S., july 2007. Image recognition based on wavelet invariant moments and wavelet neural networks. In: Information Acquisition, 2007. ICIA ‘07. International Conference on. pp. 275-279.Huang, Z., Leng, J., april 2010. Analysis of hu's moment invariants on image scaling and rotation. In: Computer Engineering and Technology (ICCET), 2010 2nd International Conference on. Vol. 7. pp. V7-476 –V7-480.Itti, L., Koch, C., Niebur, E., nov 1998. A model of saliency-based visual attention for rapid scene analysis. Pattern Analysis and Machine Intelligence, IEEE Transactions on 20 (11), 1254-1259.Jones, M., Snow, D., dec. 2008. Pedestrian detection using boosted features over many frames. In: Pattern Recognition, 2008. ICPR 2008. 19th International Conference on. pp. 1-4.Knoll, P., 2007. Hdr vision for driver assistance. In: Hoefflinger, B. (Ed.), High- Dynamic-Range (HDR) Vision. Vol. 26. Springer, pp. 123-136.Marsi, S., Impoco, G., Ukovich, A., Ramponi, G., 2007. Video enhancement and dynamic range control of hdr sequences for automotive applications. Advances in Signal Processing (EURASIP) 2007, 9.Mercimek, M., Gulez, K., Mumcu, T., 2005. Real object recognition using moment invariants. Sadhna - Acad. Proc. Eng. Sci. 30, 765-775.Miau, F., Papageorgiou, C.S., Itti, L., 2001. Neuromorphic algorithms for computer vision and attention. Proc.Intl Symp. Optical Science and Technology 01 (46), 12-23.Moeslund, T., Kruger, V., 2006. A survey of advances in vision-based human motion capture and analysis. Computer Vision and Image Understanding 103, 90-126.Nayar, S., Branzoi, V., 2003. Adaptive dynamic range imaging: optical control of pixel exposures over space and time. In: Computer Vision, 2003. Proceedings. Ninth IEEE International Conference on. pp. 1168-1175 vol.2.Park, S., Trivedi, M., 2007. Multi-person interaction and activity analysis: a synergistic track- and body-level analysis framework. Machine Vision and Applications 18, 151-166.Park, S., Trivedi, M., jul 2008. Understanding human interactions with track and body synergies (tbs) captured from multiple views. Computer Vision and Image Understanding 111 (1), 2-20.Phinyomark, A., Limsakul, C., Phukpattaranont, P., 2009. A novel feature extraction for robust emg pattern recognition. CoRR abs/0912.3973.Poppe, R., 2010. A survey on vision-based human action recognition. Image and Vision Computing 28 (6), 976-990.Qiao, Y., Wang, X., Xu, C., june 2011. Learning mahalanobis distance for dtw based online signature verification. In: Information and Automation (ICIA), 2011 IEEE International Conference on. pp. 333-338.Rabiner, L., feb 1989. A tutorial on hidden markov models and selected applications in speech recognition. Proceedings of the IEEE 77 (2), 257-286.Romero, D., Frizera, A., Bastos, T., jan. 2012a. Movement analysis in learning by repetitive recall. an approach for automatic assistance in physiotherapy. In: Biosignals and Biorobotics Conference (BRC), 2012 ISSNIP. pp. 1-8.Romero, D., Vintimilla, B., Frizera, A., Bastos, T.F., jun 2012b. Rwe patterns extraction for on-line human action recognition through window-based analysis of invariant moments. In: Robocontrol (2012). Bauru - SP, pp. 20-27.Rosso, O., Martin, M., Plastino, A., 2003. Brain electrical activity analysis using wavelet-based informational tools (ii): Tsallis non-extensivity and complexity measures. Physica A: Statistical Mechanics and its Applications 320 (0), 497-511.Rosso, O.A., Blanco, S., Yordanova, J., Kolev, V., Figliola, A., Schurmann, M., Basar, E., 2001. Wavelet entropy: a new tool for analysis of short duration brain electrical signals. Journal of Neuroscience Methods 105 (1), 65-75.Salas-Lopez, G., Sandoval-Gonzalez, O., Herrera-Aguilar, I., Martà nez-Sibaja, A., Portillo-Rodriguez, O., Vilchis-Gonzalez, A., 2012. Design and development of a planar robot for upper extremities rehabilitation with visuovibrotactile feedback. Procedia Technology 3, 147-156.Sarvaiya, J.N., 2011. Automatic image registration using mexican hat wavelet, invariant moment, and radon transform. IJACSA - International Journal of Advanced Computer Science and Applications 01 (Special Issue), 75-84.Singh, S., Velastin, S., Ragheb, H., september 2010. Muhavi: A multicamera human action video dataset for the evaluation of action recognition methods. In: Advanced Video and Signal Based Surveillance (AVSS), 2010 Seventh IEEE International Conference on. pp. 48-55.Soga, M., Kato, T., Ohta, M., Ninomiya, Y., april 2005. Pedestrian detection with stereo vision. In: Data Engineering Workshops. 21st International Conference on. Vol. 01. pp. 20-28.Viola, P., Jones, M., Snow, D., oct. 2003. Detecting pedestrians using patterns of motion and appearance. In: Computer Vision, 2003. Proceedings. Ninth IEEE International Conference on. Vol. 2. pp. 734-741.Wang, L., Hu, W., T. Tan, 2003. Recent developments in human motion analysis. Pattern Recognition 36, 585-601.Yan, L., Casperson, D., Chen, L., june 2011. Survey: Dimension reduction by pattern decomposition. In: Modelling, Identification and Control (ICMIC), Proceedings of 2011 International Conference on. pp. 69-74

A novel human-machine interface for guiding : the NeoASAS Smart Walker

In an aging society it is extremely important to develop devices, which can support and aid the elderly in their daily life. This demands tools that extend independent living and promote improved health. In this work it is proposed a new interface approach integrated into a walker. This interface is based on a joystick and it is intended to extract the user’s movement intentions. The interface is designed to be userfriendly, simple and intuitive, efficient and economic, meeting usability aspects and focused on a commercial implementation, but not being demanding at the user cognitive level. Preliminary sets of experiments were performed which showed the sensibility of the joystick to extract navigation commands from the user. These signals presented a higher frequency component that was attenuated by a Benedict-Bordner g-h filter. The presented methodology offers an effective cancelation of the undesired components from joystick data, allowing the system to extract in real-time voluntary user’s navigation commands. Based on this real-time identification of voluntary user’s commands, an approach to the control architecture of the robotic walker is being developed, in order to obtain stable and safe user assisted locomotion.(undefined

On human-in-the-loop CPS in healthcare: a cloud-enabled mobility assistance service

Despite recent advancements on cloud-enabled and human-in-the-loop cyber-physical systems, there is still a lack of understanding of how infrastructure-related quality of service (QoS) issues affect user-perceived quality of experience (QoE). This work presents a pilot experiment over a cloud-enabled mobility assistive device providing a guidance service and investigates the relationship between QoS and QoE in such a system. In our pilot experiment, we employed the CloudWalker, a system linking smart walkers and cloud platforms, to physically interact with users. Different QoS conditions were emulated to represent an architecture in which control algorithms are performed remotely. Results point out that users report satisfactory interaction with the system even under unfavorable QoS conditions. We also found statistically significant data linking QoE degradation to poor QoS conditions. We finalize discussing the interplay between QoS requirements, the human-in-the-loop effect, and the perceived QoE in healthcare applications