Real-time Immersive human-computer interaction based on tracking and recognition of dynamic hand gestures

With fast developing and ever growing use of computer based technologies, human-computer interaction (HCI) plays an increasingly pivotal role. In virtual reality (VR), HCI technologies provide not only a better understanding of three-dimensional shapes and spaces, but also sensory immersion and physical interaction. With the hand based HCI being a key HCI modality for object manipulation and gesture based communication, challenges are presented to provide users a natural, intuitive, effortless, precise, and real-time method for HCI based on dynamic hand gestures, due to the complexity of hand postures formed by multiple joints with high degrees-of-freedom, the speed of hand movements with highly variable trajectories and rapid direction changes, and the precision required for interaction between hands and objects in the virtual world. Presented in this thesis is the design and development of a novel real-time HCI system based on a unique combination of a pair of data gloves based on fibre-optic curvature sensors to acquire finger joint angles, a hybrid tracking system based on inertia and ultrasound to capture hand position and orientation, and a stereoscopic display system to provide an immersive visual feedback. The potential and effectiveness of the proposed system is demonstrated through a number of applications, namely, hand gesture based virtual object manipulation and visualisation, hand gesture based direct sign writing, and hand gesture based finger spelling. For virtual object manipulation and visualisation, the system is shown to allow a user to select, translate, rotate, scale, release and visualise virtual objects (presented using graphics and volume data) in three-dimensional space using natural hand gestures in real-time. For direct sign writing, the system is shown to be able to display immediately the corresponding SignWriting symbols signed by a user using three different signing sequences and a range of complex hand gestures, which consist of various combinations of hand postures (with each finger open, half-bent, closed, adduction and abduction), eight hand orientations in horizontal/vertical plans, three palm facing directions, and various hand movements (which can have eight directions in horizontal/vertical plans, and can be repetitive, straight/curve, clockwise/anti-clockwise). The development includes a special visual interface to give not only a stereoscopic view of hand gestures and movements, but also a structured visual feedback for each stage of the signing sequence. An excellent basis is therefore formed to develop a full HCI based on all human gestures by integrating the proposed system with facial expression and body posture recognition methods. Furthermore, for finger spelling, the system is shown to be able to recognise five vowels signed by two hands using the British Sign Language in real-time

Mobile Motion Advisor — a feedback system for physical exercise in schools

AbstractThe Mobile Motion Advisor (MMA) is a mobile feedback system designed to support high school students by giving them instructions during their physical exercises based on the integration of up-to-date sensor, data transmission and processing technologies to provide optimal training assistance. The aim is to give individually customized exercise instructions. We expect students to develop a positive attitude towards physical exercising when they do not experience frustration due to overstrain. This article gives an overview of the system architecture of the MMA framework and highlights some of the implementations

Real-time immersive human-computer interaction based on tracking and recognition of dynamic hand gestures

With fast developing and ever growing use of computer based technologies, human-computer interaction (HCI) plays an increasingly pivotal role. In virtual reality (VR), HCI technologies provide not only a better understanding of three-dimensional shapes and spaces, but also sensory immersion and physical interaction. With the hand based HCI being a key HCI modality for object manipulation and gesture based communication, challenges are presented to provide users a natural, intuitive, effortless, precise, and real-time method for HCI based on dynamic hand gestures, due to the complexity of hand postures formed by multiple joints with high degrees-of-freedom, the speed of hand movements with highly variable trajectories and rapid direction changes, and the precision required for interaction between hands and objects in the virtual world. Presented in this thesis is the design and development of a novel real-time HCI system based on a unique combination of a pair of data gloves based on fibre-optic curvature sensors to acquire finger joint angles, a hybrid tracking system based on inertia and ultrasound to capture hand position and orientation, and a stereoscopic display system to provide an immersive visual feedback. The potential and effectiveness of the proposed system is demonstrated through a number of applications, namely, hand gesture based virtual object manipulation and visualisation, hand gesture based direct sign writing, and hand gesture based finger spelling. For virtual object manipulation and visualisation, the system is shown to allow a user to select, translate, rotate, scale, release and visualise virtual objects (presented using graphics and volume data) in three-dimensional space using natural hand gestures in real-time. For direct sign writing, the system is shown to be able to display immediately the corresponding SignWriting symbols signed by a user using three different signing sequences and a range of complex hand gestures, which consist of various combinations of hand postures (with each finger open, half-bent, closed, adduction and abduction), eight hand orientations in horizontal/vertical plans, three palm facing directions, and various hand movements (which can have eight directions in horizontal/vertical plans, and can be repetitive, straight/curve, clockwise/anti-clockwise). The development includes a special visual interface to give not only a stereoscopic view of hand gestures and movements, but also a structured visual feedback for each stage of the signing sequence. An excellent basis is therefore formed to develop a full HCI based on all human gestures by integrating the proposed system with facial expression and body posture recognition methods. Furthermore, for finger spelling, the system is shown to be able to recognise five vowels signed by two hands using the British Sign Language in real-time.EThOS - Electronic Theses Online ServiceGBUnited Kingdo

RTXP : A Localized Real-Time Mac-Routing Protocol for Wireless Sensor Networks

Protocols developed during the last years for Wireless Sensor Networks (WSNs) are mainly focused on energy efficiency and autonomous mechanisms (e.g. self-organization, self-configuration, etc). Nevertheless, with new WSN applications, appear new QoS requirements such as time constraints. Real-time applications require the packets to be delivered before a known time bound which depends on the application requirements. We particularly focus on applications which consist in alarms sent to the sink node. We propose Real-Time X-layer Protocol (RTXP), a real-time communication protocol. To the best of our knowledge, RTXP is the first MAC and routing real-time communication protocol that is not centralized, but instead relies only on local information. The solution is cross-layer (X-layer) because it allows to control the delays due to MAC and Routing layers interactions. RTXP uses a suited hop-count-based Virtual Coordinate System which allows deterministic medium access and forwarder selection. In this paper we describe the protocol mechanisms. We give theoretical bound on the end-to-end delay and the capacity of the protocol. Intensive simulation results confirm the theoretical predictions and allow to compare with a real-time centralized solution. RTXP is also simulated under harsh radio channel, in this case the radio link introduces probabilistic behavior. Nevertheless, we show that RTXP it performs better than a non-deterministic solution. It thus advocates for the usefulness of designing real-time (deterministic) protocols even for highly unreliable networks such as WSNs

A Survey of QoS Routing Protocols for Ad Hoc Networks

The aim of this paper is to give a big survey in enhancing the balance of the routing load and the consumption of resources using network layer metrics for the path discovery in the MAODV protocol. A ad hoc network (AD HOC NETWORKS) consists of a collection of wireless mobile nodes, which form a temporary network without relying on any existing infrastructure or centralized administration. The bandwidth of the ad hoc networks architecture is limited and shared between the participating nodes in the network, therefore an efficient utilization of the network bandwidth is very important. Multicasting technology can minimize the consumption of the link bandwidth and reduce the communication cost too. As multimedia and group-oriented computing gains more popularity for users of ad hoc networks, the effective Quality of Service (QoS) of the multicasting protocol plays a significant role in ad hoc networks. In this paper we propose a reconstruction of the MAODV protocol by extending some featuring QoS in MAODV. All simulations are prepared with the NS2 simulator and compare the performance of this algorithm with the MAODV algorithm. The achieved results illustrate faster path discovery and more performing routing balance in the use of MAODV-Extension.This paper would give relatively a modest support in Mobile Technology according to QoS communication

A Server-Based Mobile Coaching System

A prototype system for monitoring, transmitting and processing performance data in sports for the purpose of providing feedback has been developed. During training, athletes are equipped with a mobile device and wireless sensors using the ANT protocol in order to acquire biomechanical, physiological and other sports specific parameters. The measured data is buffered locally and forwarded via the Internet to a server. The server provides experts (coaches, biomechanists, sports medicine specialists etc.) with remote data access, analysis and (partly automated) feedback routines. In this way, experts are able to analyze the athlete’s performance and return individual feedback messages from remote locations

Resistance to change in scholarship: an experiment using an immersive and interactive 3D structure

The paper focuses on the process that generates resistance to change in the experiment of an immersive and interactive 3D structure by students during their course. Our case study enables us to complete the redefinition of resistance given by Piderit (2000). Four levels of resistance can be identified: emotional, physical, cognitive and intentional responses. We will show that these four dimensions are complex because they could take a positive or negative appearance for the change recipients (students: cyber-fear or cyber-utopia)

Review of multicast QoS routing protocols for mobile ad hoc networks

A Mobile Ad hoc NETwork (MANET) is consisting of a collection of wireless mobile nodes, which form a temporary network without relying on any existing infrastructure or centralized administration. Since the bandwidth of MANETs is limited and shared between the participating nodes in the network, it is important to efficiently utilize the network bandwidth. Multicasting can minimize the link bandwidth consumption and reduce the communication cost by sending the same data to multiple participants. Multicast service is critical for applications that need collaboration of team of users. Multicasting in MANETs becomes a hot research area due to the increasing popularity of group communication applications such as video conferencing and interactive television. Recently, multimedia and group-oriented computing gains more popularity for users of ad hoc networks. So, effective Quality of Service (QoS) multicasting protocol plays significant role in MANETs. In this paper, we are presenting an overview of set of the most recent QoS multicast routing protocols that have been proposed in order to provide the researchers with a clear view of what has been done in this field