An Immersive Telepresence System using RGB-D Sensors and Head Mounted Display

We present a tele-immersive system that enables people to interact with each other in a virtual world using body gestures in addition to verbal communication. Beyond the obvious applications, including general online conversations and gaming, we hypothesize that our proposed system would be particularly beneficial to education by offering rich visual contents and interactivity. One distinct feature is the integration of egocentric pose recognition that allows participants to use their gestures to demonstrate and manipulate virtual objects simultaneously. This functionality enables the instructor to ef- fectively and efficiently explain and illustrate complex concepts or sophisticated problems in an intuitive manner. The highly interactive and flexible environment can capture and sustain more student attention than the traditional classroom setting and, thus, delivers a compelling experience to the students. Our main focus here is to investigate possible solutions for the system design and implementation and devise strategies for fast, efficient computation suitable for visual data processing and network transmission. We describe the technique and experiments in details and provide quantitative performance results, demonstrating our system can be run comfortably and reliably for different application scenarios. Our preliminary results are promising and demonstrate the potential for more compelling directions in cyberlearning.Comment: IEEE International Symposium on Multimedia 201

Towards Autonomous Unmanned Vehicle Systems

As an emerging technology, autonomous Unmanned Vehicle Systems (UVS) have found not only many military applications, but also various civil applications. For example, Google, Amazon and Facebook are developing their UVS plans to explore new markets. However, there are still a lot of challenging problems which deter the UVS’s development. We study two important and challenging problems in this dissertation, i.e. localization and 3D reconstruction. Specifically, most GPS based localization systems are not very accurate and can have problems in areas where no GPS signals are available. Based on the Received Signal Strength Indication (RSSI) and Inertial Navigation System (INS), we propose a new hybrid localization system, which is very efficient and can account for dynamic communication environments. Extensive simulation results demonstrate the efficiency of the proposed localization system. Besides, 3D reconstruction is a key problem in autonomous navigation and hence very important for UVS.With the help of high-speed Internet and powerful cloud servers, the light-weight computers on the UVS can now execute computationally expensive computer vision based algorithms. We develop a 3D reconstruction scheme which employs cloud computing to perform realtime 3D reconstruction. Simulations and experiments show the efficacy and efficiency of our scheme

Accelerated Object Tracking with Local Binary Features

Multi-object tracking is a problem with wide application in modern computing. Object tracking is leveraged in areas such as human computer interaction, autonomous vehicle navigation, panorama generation, as well as countless other robotic applications. Several trackers have demonstrated favorable results for tracking of single objects. However, modern object trackers must make significant tradeoffs in order to accommodate multiple objects while maintaining real-time performance. These tradeoffs include sacrifices in robustness and accuracy that adversely affect the results. This thesis details the design and multiple implementations of an object tracker that is focused on computational efficiency. The computational efficiency of the tracker is achieved through use of local binary descriptors in a template matching approach. Candidate templates are matched to a dictionary composed of both static and dynamic templates to allow for variation in the appearance of the object while minimizing the potential for drift in the tracker. Locality constraints have been used to reduce tracking jitter. Due to the significant promise for parallelization, the tracking algorithm was implemented on the Graphics Processing Unit (GPU) using the CUDA API. The tracker\u27s efficiency also led to its implantation on a mobile platform as one of the mobile trackers that can accurately track at faster than realtime speed. Benchmarks were performed to compare the proposed tracker to state of the art trackers on a wide range of standard test videos. The tracker implemented in this work has demonstrated a higher degree of accuracy while operating several orders of magnitude faster

A Survey of Applications and Human Motion Recognition with Microsoft Kinect

Microsoft Kinect, a low-cost motion sensing device, enables users to interact with computers or game consoles naturally through gestures and spoken commands without any other peripheral equipment. As such, it has commanded intense interests in research and development on the Kinect technology. In this paper, we present, a comprehensive survey on Kinect applications, and the latest research and development on motion recognition using data captured by the Kinect sensor. On the applications front, we review the applications of the Kinect technology in a variety of areas, including healthcare, education and performing arts, robotics, sign language recognition, retail services, workplace safety training, as well as 3D reconstructions. On the technology front, we provide an overview of the main features of both versions of the Kinect sensor together with the depth sensing technologies used, and review literatures on human motion recognition techniques used in Kinect applications. We provide a classification of motion recognition techniques to highlight the different approaches used in human motion recognition. Furthermore, we compile a list of publicly available Kinect datasets. These datasets are valuable resources for researchers to investigate better methods for human motion recognition and lower-level computer vision tasks such as segmentation, object detection and human pose estimation

Augmented reality applied to language translation

Being a tourist in a foreign country is an adventure full of memories and experiences, but it can be truly challenging when it comes to communication. Finding yourself in an unknown place, where all the road signs and guidelines have such different characters, may end up in a dead end or with some unexpected results. Then, what if we could use a smartphone to read that restaurant menu? Or even find the right department in a mall? The applications are so many and the market is ready to invest and give opportunities to creative and economic ideas. The dissertation intends to explore the field of Augmented Reality, while helping the user to enrich his view with information. Giving the ability to look around, detect the text in the surroundings and read its translation in our own dialect, is a great step to overcome language issues. Moreover, using smartphones at anyone’s reach, or wearing smartglasses that are even less intrusive, gives a chance to engage a complex matter in a daily routine. This technology requires flexible, accurate and fast Optical Character Recognition and Translation systems, in an Internet of Things scenery. Quality and precision is a must, yet to be further developed and improved. Entering in a realtime digital data environment, will support great causes and aid the progress and evolution of many intervention areas