Vision-Based Three Dimensional Hand Interaction In Markerless Augmented Reality Environment

Kemunculan realiti tambahan membolehkan objek maya untuk wujud bersama dengan dunia sebenar dan ini memberi kaedah baru untuk berinteraksi dengan objek maya. Sistem realiti tambahan memerlukan penunjuk tertentu, seperti penanda untuk menentukan bagaimana objek maya wujud dalam dunia sebenar. Penunjuk tertentu mesti diperolehi untuk menggunakan sistem realiti tambahan, tetapi susah untuk seseorang mempunyai penunjuk tersebut pada bila-bila masa. Tangan manusia, yang merupakan sebahagian dari badan manusia dapat menyelesaikan masalah ini. Selain itu, tangan boleh digunakan untuk berinteraksi dengan objek maya dalam dunia realiti tambahan. Tesis ini membentangkan sebuah sistem realiti tambahan yang menggunakan tangan terbuka untuk pendaftaran objek maya dalam persekitaran sebenar dan membolehkan pengguna untuk menggunakan tangan yang satu lagi untuk berinteraksi dengan objek maya yang ditambahkan dalam tiga-matra. Untuk menggunakan tangan untuk pendaftaran dan interaksi dalam realiti tambahan, postur dan isyarat tangan pengguna perlu dikesan. The advent of augmented reality (AR) enables virtual objects to be superimposed on the real world and provides a new way to interact with the virtual objects. AR system requires an indicator to determine for how the virtual objects aligned in the real world. The indicator must first be obtained to access to a particular AR system. It may be inconvenient to have the indicator in reach at all time. Human hand, which is part of the human body may be a solution for this. Besides, hand is also a promising tool for interaction with virtual objects in AR environment. This thesis presents a markerless Augmented Reality system which utilizes outstretched hand for registration of virtual objects in the real environment and enables the users to have three dimensional (3D) interaction with the augmented virtual objects. To employ the hand for registration and interaction in AR, hand postures and gestures that the user perform has to be recognized

Electromagnetic Position Sensing and Force Feedback for a Magnetic Stylus with an Interactive Display

This letter describes the design, implementation, validation, and demonstration of an electromagnetic system that can be incorporated into a graphical display to provide computer-controlled planar feedback forces on the tip of a stylus or fingertip-mounted magnet held near the display surface, according to the magnet position and virtual fixtures implemented in software. An array of magnetometer sensors is used to detect the position of the magnet, while a pair of box-shaped coils behind the display produces feedback forces on the stylus parallel to the plane of the display. Electromagnetic analysis for the system design is presented and system implementation is described. Validation results are given for force generation within a 100 mm × 100 mm area and force interaction with a virtual obstacle is demonstrated

Hand Pose Estimation Using Deep Stereovision and Markov-chain Monte Carlo

Hand pose is emerging as an important interface for human-computer interaction. The problem of hand pose estimation from passive stereo inputs has received less attention in the literature compared to active depth sensors. This paper seeks to address this gap by presenting a datadriven method to estimate a hand pose from a stereoscopic camera input, by introducing a stochastic approach to propose potential depth solutions to the observed stereo capture and evaluate these proposals using two convolutional neural networks (CNNs). The first CNN, configured in a Siamese network architecture, evaluates how consistent the proposed depth solution is to the observed stereo capture. The second CNN estimates a hand pose given the proposed depth. Unlike sequential approaches that reconstruct pose from a known depth, our method jointly optimizes the hand pose and depth estimation through Markov-chain Monte Carlo (MCMC) sampling. This way, pose estimation can correct for errors in depth estimation, and vice versa. Experimental results using an inexpensive stereo camera show that the proposed system more accurately measures pose better than competing methods

A vision-based approach for human hand tracking and gesture recognition.

Hand gesture interface has been becoming an active topic of human-computer interaction (HCI). The utilization of hand gestures in human-computer interface enables human operators to interact with computer environments in a natural and intuitive manner. In particular, bare hand interpretation technique frees users from cumbersome, but typically required devices in communication with computers, thus offering the ease and naturalness in HCI. Meanwhile, virtual assembly (VA) applies virtual reality (VR) techniques in mechanical assembly. It constructs computer tools to help product engineers planning, evaluating, optimizing, and verifying the assembly of mechanical systems without the need of physical objects. However, traditional devices such as keyboards and mice are no longer adequate due to their inefficiency in handling three-dimensional (3D) tasks. Special VR devices, such as data gloves, have been mandatory in VA. This thesis proposes a novel gesture-based interface for the application of VA. It develops a hybrid approach to incorporate an appearance-based hand localization technique with a skin tone filter in support of gesture recognition and hand tracking in the 3D space. With this interface, bare hands become a convenient substitution of special VR devices. Experiment results demonstrate the flexibility and robustness introduced by the proposed method to HCI.Dept. of Computer Science. Paper copy at Leddy Library: Theses & Major Papers - Basement, West Bldg. / Call Number: Thesis2004 .L8. Source: Masters Abstracts International, Volume: 43-03, page: 0883. Adviser: Xiaobu Yuan. Thesis (M.Sc.)--University of Windsor (Canada), 2004

Assessment of VR Technology and its Applications to Engineering Problems

Virtual reality applications are making valuable contributions to the field of product realization. This paper presents an assessment of the hardware and software capabilities of VR technology needed to support a meaningful integration of VR applications in the product life cycle analysis. Several examples of VR applications for the various stages of the product life cycle engineering are presented as case studies. These case studies describe research results, fielded systems, technical issues, and implementation issues in the areas of virtual design, virtual manufacturing, virtual assembly, engineering analysis, visualization of analysis results, and collaborative virtual environments. Current issues and problems related to the creation, use, and implementation of virtual environments for engineering design, analysis, and manufacturing are also discussed

Graphite immobilisation in glass composite materials

Irradiated graphite is a problematic nuclear waste stream and currently raises significant concern worldwide in identifying its long-term disposal route. This thesis describes the use of glass materials for the immobilisation of irradiated graphite prepared by microwave, conventional and sparks plasma sintering methods. Several potential glass compositions namely iron phosphate, aluminoborosilicate, calcium aluminosilicate, alkali borosilicate and obsidian were considered for the immobilisation of various loadings of graphite simulating irradiated graphite. The properties of the samples produced using different processing methods are compared selectively. An investigation of microwave processing using an iron phosphate glass composition revealed that full reaction of the raw materials and formation of a glass melt occurs with consequent removal of porosity at 8 minutes microwave processing. When graphite is present, iron phosphate crystalline phases are formed with much higher levels of residual porosity of up to 43 % than in the samples prepared using conventional sintering under argon. It is found that graphite reacts with the microwave field when in powder form but this reaction is minimised when the graphite is incorporated into a pellet, and that the graphite also impedes sintering of the glass. Mössbauer spectroscopy indicates that reduction of iron occurs with concomitant graphite oxidation. The production of graphite-glass samples using various powdered glass compositions by conventional sintering method still resulted in high porosity with an average of 6-17 % for graphite loadings of 20-25 wt%. Due to the use of pre-made glasses and controlled sintering parameters, the loss of graphite from the total mass is reduced compared to the microwaved samples; the average mass loss is < 0.8 %. The complication of iron oxidation and reduction is present in all the iron containing base glasses considered and this increases the total porosity of the graphite-glass samples. It is concluded that the presence of iron in the raw materials or base glasses as an encapsulation media for the immobilisation of the irradiated graphite waste is not advisable. The production of glass and graphite-glass samples based calcium aluminosilicate composition by spark plasma sintering method is found highly suitable for the immobilisation of irradiated graphite wastes. The advantages of the method includes short processing time i.e. < 40 minutes, improved sintering transport mechanisms, limited graphite oxidation, low porosity (1-4 %) and acceptable tensile strength (2-7 MPa). The most promising samples prepared using spark plasma sintering method were loaded with 30-50 wt% graphite

Electromagnetic Position Sensing and Force Feedback for a Magnetic Stylus with an Interactive Display

This letter describes the design, implementation, validation, and demonstration of an electromagnetic system that can be incorporated into a graphical display to provide computer-controlled planar feedback forces on the tip of a stylus or fingertip-mounted magnet held near the display surface, according to the magnet position and virtual fixtures implemented in software. An array of magnetometer sensors is used to detect the position of the magnet, while a pair of box-shaped coils behind the display produces feedback forces on the stylus parallel to the plane of the display. Electromagnetic analysis for the system design is presented and system implementation is described. Validation results are given for force generation within a 100 mm × 100 mm area and force interaction with a virtual obstacle is demonstrated