Sanders, David Adrian

English

Portsmouth University Research Portal (Pure)

Country Code 21. 
 
Journal of Intelligent Mobility, Volume 16, 2013 294  PCR   ISSN  1472-7633 
 
 
 
 
 
 
SIMPLE GESTURE RECOGNITION TO ASSIST IN CONTROLLING 
A POWERED WHEELCHAIR 
 
D.A. Sanders 
 
School of Engineering, University of Portsmouth, Portsmouth, Hampshire, PO1 3DJ 
 
david.sanders@port.ac.uk 
 
 
 
 
 
Abstract 
 
This short paper describes some initial work to create 
a simple hand gesture recognition system for a 
powered wheelchair.  It is employed to assist people 
with mobility issues to drive a powered wheelchair.  It 
replaces a standard joystick with hand gestures. 
Infrared sensors detect simple gestures and they are 
categorized in order to assist in driving a powered 
wheelchair. Some initial technical testing and results 
are described. 
 
Key words: Wheelchair, Assist, Driving, Sensor, 
Infrared, Gesture. 
 
Introduction 
 
This short paper describes some initial work to create a 
simple hand gesture recognition system for a powered 
wheelchair.  It replaces a standard joystick with hand 
gestures. Some initial technical testing and results are 
described. 
 
Almost all powered wheelchairs are provided with a 
standard joystick controller but users often need a 
bespoke interface.  Joysticks can only be used by 
someone able to apply enough force to move it in a 
controlled way in any direction..  An alternative is 
gesture based detection.  Gesture based detection and 
classification has been successfully used for human 
computer interaction in various applications such as 
robot control, computer gaming, editing text and 
controlling vehicles. 
 
There are a few papers about camera based hand gesture 
recognition [Zhang et al, 2011; Garg et al, 2009].   
Those systems needed an expensive and complex system 
for image processing. 
 
In this paper a new simple concept for a low cost 
approach is presented.  Discussion with wheelchair users, 
clinicians and carers in the NHS suggested that the main 
factors considered when deliberating on whether to 
provide a powered wheelchair or not were functional 
ability and potential for return to normal life and work. 
 
Wheelchairs controlled by a joystick are not suitable for 
some stroke patients and some spinal cord injury 
patients.  Patients with acute quadriplegia take time to 
recover motor control in their hands.  The simple system 
described here can be used by patients who are only able 
to move their hands a little. 
 
Because powered wheelchairs with joysticks are not 
always appropriate, clinicians tend to suggest the use of 
manual wheelchairs. It is unusual to suggest 
modifications to a powered wheelchair to create a 
bespoke solution, especially as that can make a powered 
wheelchair more expensive.  
 
Background 
 
Many gesture recognition systems have been reported in 
the literature [Ahmad et al, 2011; Kaustubh & Bhiksha 
2009; Perrin et al, 2004; Wang  et al, 2010].  Several 
algorithms and methods have been proposed to interpret 
gestures [Igorevich et al, 2010;  Hanning et al, 2004; 
Miners et al, Mu-Chun, 2000; Songmin et al, 2012;  
Ghosh & Ari, 2011; Tuan et al,  2011; Wang  et al, 
2010; Yamato & Ohya, 1992; Yamato & Ohya, 1992]. 
 
According to the stroke Association, roughly 15 million 
people worldwide are affected by stroke and 80% of 
them have mobility issues. In addition, 250,000 to 
500,000 people suffer a spinal cord injury every year. 
 
Country Code 21. 
 
Journal of Intelligent Mobility, Volume 16, 2013 295  PCR   ISSN  1472-7633 
 
A lack of basic mobility is a factor resulting in the 
exclusion of people with a spinal cord injury from 
participating fully in society. The World Health 
Organisation suggests the use of suitable assistive 
devices to help them with basic everyday activities.  
There are about 400,000 spinal cord injury patients in the 
USA, with about 15,000 new injuries every year [Hasan 
Al-Nashash, 2009]. 
 
Assistive devices tend to be either generic or bespoke.  
There are systems based on movement of the tongue 
[Lund et al, 2010; Xueliang & Maysam, 2009], head 
[Ahmad et al, 2011; Zhang-fang et al, 2010; [Montesano 
et al, 2010], brain [Ituratte et al, 2009; Kyuwan et al, 
2006], electrooculography and eyes [Chern-Sheng et al, 
2006; as well as joysticks.  Research presented here 
could create a low cost infrared sensor based assistive 
device that would fit between these two extremes. 
 
The prototype system described in this paper can detect 
the movement of any part of the body, for example head, 
fingers, hands, arms etc.  If movement can be detected 
then a powered wheelchair can be controlled by it. 
 
The infrared sensors needed to interpret four simple 
gestures to control a wheelchair: forward, reverse, right 
and left.   
 
Sensors 
 
Infrared sensor technology is used in wide range of 
applications [Garcia et al, 2004; Singh et al, 2010; 
Rakshit & Chatterjee, 2010].  In this work, the motor 
drives were controlled from an infrared sensor array 
based on the principle of reflection from an incident 
surface. 
 
Infrared LEDs emit continuous beams of infrared.  When 
a reflecting surface is in front of the infrared receiver (a 
photodiode) then the infrared rays reflect back and are 
captured by the photodiode, which generates a current.  
The amount of current generated was proportional to the 
amount of infrared rays reflected.  A part of a user's body 
serves as a reflecting surface. 
 
When the user places a part of the body in front of the 
array, then software attempts to identify a gesture. 
 
When a gesture is identified, the system responds with a 
movement of the wheelchair: forward , Right, Reverse or  
Left. 
  
Testing and results 
 
Response timings were compared in the laboratory with 
those of a joystick.  The new system was slower but 
successfully steered the wheelchair. 
 
A group of 13 volunteers were then asked to rate the use 
of the gesture system compared to using a joystick.  All 
the volunteers preferred the use of the joystick but all 
were able to steer the wheelchair with both. 
 
After using the prototype, volunteers rated the new 
system as very good, good, satisfactory, unsatisfactory 
and poor.  The average rating was satisfactory. 
 
The success rate of the prototype in identifying correct 
gestures was low.  Overall success rate in recognizing 
gestures was 29.5%, although in some cases the 
percentage was a high as 70%. 
 
For any gesture change, the response time was around 
one second. That was slower than all the joysticks 
considered. 
 
Discussion 
 
Although the new system was slower and all the 
volunteers preferred the use of the joystick, it did 
successfully steer the wheelchair and the average rating 
was satisfactory. 
 
A main problem was that the success rate was low so that 
gestures needed to be repeated.  That needs to be 
improved.  Response times were slow compared to a 
joystick and that also needs to be improved. 
 
Response time for gesture change was about one second 
and that was too slow so that the new systems will not be 
suitable for everybody with impaired mobility.  That 
said, it could be another alternative to existing control 
methods. 
 
Future work will concentrate on improving recognition 
and response times. 
 
References 
 
Ahmad Akl, Chen Feng, and Shahrokh Valaee, "A Novel 
Accelerometer-Based Gesture Recognition System", 
IEEE Transactions on signal processing 59 (12), 2011. 
Chern-Sheng Lin; Chien-Wa Ho; Wen-Chen Chen, 
“Chuang-Chien Chiu; Mau-Shiun Yeh, Powered 
wheelchair controlled by eye-tracking system”, Optica 
Applicata, 01/2006, Vol. 36, pp. 401-412. 
Garcia, J.J. ; Urena, J. ; Hernandez, A. ; Mazo, M. ; 
Garcia, , JC ; Alvarez, F. ; Jimenez, J.A. ; Donato, P. ; 
Perez, M.C., "IR sensor array configuration and signal 
processing for detecting obstacles in railways", Sensor 
Array and Multichannel Signal Processing Workshop 
Proceedings, 2004, pp: 216 -220. 
Garg, R. ; Shriram, N. ; Gupta, V. ; Agrawal, V., "A 
smart mobility solution for physically challenged", 
2009 IEEE International Conference on Vehicular 
Electronics and Safety (ICVES), pp: 168 – 173. 
Country Code 21. 
 
Journal of Intelligent Mobility, Volume 16, 2013 296  PCR   ISSN  1472-7633 
 
Ghosh, D.K. ; Ari, S., "A static hand gesture recognition 
algorithm using k-mean based radial basis function 
neural network", 8th International Conference on 
Information, Communications and Signal Processing 
(ICICS) 2011 pp: 1 - 5 
Hasan Al-Nashash, Noreen A. Fatoo, Nabil N. Mirza, 
Rabi I. Ahmed, Gracee Agrawal, Nitish V. Thakor, 
Angelo H. All, "Spinal Cord Injury Detection and 
Monitoring Using Spectral Coherence", IEEE 
Transactions On Biomedical Engineering, 56(8), 
pp:1971-9. 2009. 
Hong P, Turk M, and Huang TS, “Gesture modeling and 
recognition using finite state machines,” in Proc. 4th 
IEEE Int. Conf. Autom. Face Gesture Recogn., 
Grenoble, France, Mar. 2000, pp. 410–415. 
Igorevich, R.R., Pusik Park, Dugki Min, Yunjung Park, 
Jongchan Choi, Eunmi Choi, "Hand gesture 
recognition algorithm based on grayscale histogram of 
the image", 2010 4th International Conference on 
Application of Information & Communication 
Technologies (AICT), pp: 1 – 4. 
Inake Ituratte, Javier M Antellis, Andrea Kubler and 
Javier Minguez, "A Noninvasive brain actuated 
wheelchair based on a P300 Neuropsychological 
protocol and Automated Navigation", IEEE 
Transactions on Robotics, Vol.25, No.3, pp: 614-626.  
June 2009. 
Kaustubh Kalgaonkar, Bhiksha Raj, "One-Handed 
Gesture Recognition Using Ultrasonic Doppler Sonar", 
ICASSP 2009, 978-1-4244-2354-5/09. pp: 1889 – 
1892. 
Kyuwan Choi; Sato, M.; Koike, Y, "A new, human-
centered wheelchair system controlled by the EMG 
signal", International Joint Conference on Neural 
Networks, 2006. IJCNN '06,pp: 4664 – 4671. 
Lund, M.E., Christiensen, H.V., Caltenco, H.A., Lontis, 
E.R., Bentsen, B., Struijk, J.J. "Inductive tongue 
control of powered wheelchairs", 2010 Annual 
International Conference of the IEEE Engineering in 
Medicine and Biology Society (EMBC), pp: 3361 – 
3364. 
Megalingam, RK; Nair, Ramesh N; Prakhya, SM; 
Mohan M, “Gest-BOT' - A Highly Convenient 
Locomotive Solution for the Elderly and Physically 
Challenged”, Global Humanitarian Technology 
Conference (GHTC), 21-24, October in Seattle, 
Washington, USA, IEEE Xplorer. pp: 350 – 355. 
Miners BW, Basir OA, and Kamel MS, "Understanding 
Hand Gestures Using Approximate Graph Matching", 
IEEE Transactions On Systems, Man, And 
Cybernetics—Part A: Systems And Humans, Vol. 35, 
No. 2, March 2005, pp: 239 – 248. 
Mu-Chun Su, ‘A Fuzzy Rule-Based Approach to Spatio-
Temporal Hand Gesture Recognition', IEEE 
Transactions On Systems, Man, And Cybernetics- 
VOL. 30, NO. 2, MAY 2000, pp: 276 – 281. 
 
 
Perrin, S. ; Cassinelli, A. ; Ishikawa, M., " Gesture 
recognition using laser-based tracking system", Sixth 
IEEE International Conference on Automatic Face and 
Gesture Recognition, 2004. Proceedings. Pp: 541 – 
546. 
Ramamoorthy A, Vaswani N, Chaudhury S, and  
Banerjee S, “Recognition of dynamic hand gestures,” 
Pattern Recogn., vol. 36, pp: 2069–2081, 2003. 
Singh, S.K. ; Singh, D.K. ; Thapa, R. ; Khurana, R., 
"Design and implementation of navigation algorithm 
for intelligent racing car", 2010 International 
Conference on Emerging Trends in Robotics and 
Communication Technologies (INTERACT), pp: 303 
– 306. 
Songmin Jia ; Jun Yan ; Jinhui Fan ; Xiuzhi Li ; Liwen 
Gao, "Multimodal intelligent wheelchair control based 
on fuzzy algorithm", 2012 International Conference on 
Information and Automation (ICIA), pp: 582 – 587. 
Tuan Nghia Nguyen, Steven W. Su, and Hung T. 
Nguyen, “Robust Neuro-sliding Mode Multivariable 
Control Strategy for Powered Wheelchairs”, IEEE 
Transactions On Neural Systems And Rehabilitation 
Engineering, Vol. 19, No. 1, February 2011, pp: 105-
111. 
Wang Xinyu ; Sun Guan ; Han Dong ; Zhang Ting, 
"Data glove gesture recognition based on an improved 
neural network", 29th Chinese Control Conference 
(CCC), 2010 2010, pp: 2434 – 2437. 
Xueliang Huo, Maysam Ghovanloo, “Using 
Unconstrained Tongue Motion as an Alternative 
Control Mechanism for Wheeled Mobility”, IEEE 
Transactions On Biomedical Engineering, Vol. 56, No. 
6, June 2009. pp:1719-1726. 
Yamato, J. Ohya, J. Ishii, K “Recognizing human action 
in time sequential images using hidden Markov 
model,” in Proc. IEEE Int. Conf. Comput. Vis. Pattern 
Recogn., Champaign, IL, 1992, pp. 379–385. 
Yeasin M and Chaudhuri S, “Visual understanding of 
dynamic hand gestures,” Pattern Recogn., vol. 33, pp: 
1805–1817, 2000. 
Yoon HS, Soh J, Bae YJ and Yang HS, “Hand gesture 
recognition using combined features of location, angle 
and velocity,” Pattern Recogn, vol. 34, pp: 1491–1501, 
2001. 
Zhang-fang Hu ; Lin Li ; Yuan Luo ; Yi Zhang ; Xing 
Wei, "A novel intelligent wheelchair control approach 
based on head gesture recognition" Computer 
Application and System Modeling (ICCASM), 2010 
International Conference on Volume: 6. Pp: V6 159-
163. 
Zhang Y; Zhang J; Luo Y, "A novel intelligent 
wheelchair control system based on hand gesture 
recognition", 2011 IEEE/ICME International 
Conference on Complex Medical Engineering (CME), 
pp: 334 – 339. 
 


Simple gesture recognition to assist in controlling a powered wheelchair

https://researchportal.port.ac.uk/portal/files/2526637/IM_16_1_Simple_getsture_recognition_to_assist_in_controlling_a_powered_wheelchair.pdf

Simple gesture recognition to assist in controlling a powered wheelchair

Abstract

Similar works

Full text

Available Versions

Portsmouth University Research Portal (Pure)