Neural Network based Robot 3D Mapping and Navigation using Depth Image Camera

Robotics research has been developing rapidly in the past decade. However, in order to bring robots into household or office environments and cooperate well with humans, it is still required more research works. One of the main problems is robot localization and navigation. To be able to accomplish its missions, the mobile robot needs to solve problems of localizing itself in the environment, finding the best path and navigate to the goal. The navigation methods can be categorized into map-based navigation and map-less navigation. In this research we propose a method based on neural networks, using a depth image camera to solve the robot navigation problem. By using a depth image camera, the surrounding environment can be recognized regardless of the lighting conditions. A neural network-based approach is fast enough for robot navigation in real-time which is important to develop the full autonomous robots.In our method, mapping and annotating of the surrounding environment is done by the robot using a Feed-Forward Neural Network and a CNN network. The 3D map not only contains the geometric information of the environments but also their semantic contents. The semantic contents are important for robots to accomplish their tasks. For instance, consider the task “Go to cabinet to take a medicine”. The robot needs to know the position of the cabinet and medicine which is not supplied by solely the geometrical map. A Feed-Forward Neural Network is trained to convert the depth information from depth images into 3D points in real-world coordination. A CNN network is trained to segment the image into classes. By combining the two neural networks, the objects in the environment are segmented and their positions are determined.We implemented the proposed method using the mobile humanoid robot. Initially, the robot moves in the environment and build the 3D map with objects placed in their positions. Then, the robot utilizes the developed 3D map for goal-directed navigation.The experimental results show good performance in terms of the 3D map accuracy and robot navigation. Most of the objects in the working environments are classified by the trained CNN. Un-recognized objects are classified by Feed-Forward Neural Network. As a result, the generated maps reflected exactly working environments and can be applied for robots to safely navigate in them. The 3D geometric maps can be generated regardless of the lighting conditions. The proposed localization method is robust even in texture-less environments which are the toughest environments in the field of vision-based localization.博士(工学)法政大学 (Hosei University

Prioritizing Content of Interest in Multimedia Data Compression

Image and video compression techniques make data transmission and storage in digital multimedia systems more efficient and feasible for the system's limited storage and bandwidth. Many generic image and video compression techniques such as JPEG and H.264/AVC have been standardized and are now widely adopted. Despite their great success, we observe that these standard compression techniques are not the best solution for data compression in special types of multimedia systems such as microscopy videos and low-power wireless broadcast systems. In these application-specific systems where the content of interest in the multimedia data is known and well-defined, we should re-think the design of a data compression pipeline. We hypothesize that by identifying and prioritizing multimedia data's content of interest, new compression methods can be invented that are far more effective than standard techniques. In this dissertation, a set of new data compression methods based on the idea of prioritizing the content of interest has been proposed for three different kinds of multimedia systems. I will show that the key to designing efficient compression techniques in these three cases is to prioritize the content of interest in the data. The definition of the content of interest of multimedia data depends on the application. First, I show that for microscopy videos, the content of interest is defined as the spatial regions in the video frame with pixels that don't only contain noise. Keeping data in those regions with high quality and throwing out other information yields to a novel microscopy video compression technique. Second, I show that for a Bluetooth low energy beacon based system, practical multimedia data storage and transmission is possible by prioritizing content of interest. I designed custom image compression techniques that preserve edges in a binary image, or foreground regions of a color image of indoor or outdoor objects. Last, I present a new indoor Bluetooth low energy beacon based augmented reality system that integrates a 3D moving object compression method that prioritizes the content of interest.Doctor of Philosoph

A Framework for Site-Specific Spatial Audio Applications

As audio recording and reproduction technology has advanced over the past five decades, increasing attention has been paid to recreating the highly spatialised listening experience we understand from our physical environment. This is the logical next step in the quest for increasing audio clarity, particularly as virtual reality gaming and augmented reality experiences become more widespread. This study sought to develop and demonstrate a technical framework for the production of site-specific audio-based works that is user-friendly and cost effective. The system was intended to be used by existing content producers and audio programmers to work collaboratively with a range of site-based organisations such as museums and galleries to produce an audio augmentation of the physicality of the space. This research was guided by four key aims: 1. Demonstrate a compositional method for immersive spatial audio that references the novel physical environment and the listener’s movement within it. 2. Describe a framework for the development and deployment of a spatial audio visitor technology system. 3. Prototype a naturalistic method for the delivery and navigation of contextual information via audio. 4. Deploy, demonstrate, and evaluate a spatial audio experience within a representative environment. The resulting system makes use of a range of existing technologies to provide a development experience and output that meets a clearly defined set of criteria. Furthermore, a case study application has been developed that demonstrates the use of the system to augment a selection of six paintings in a gallery space. For each of these paintings, a creative spatial composition was produced that demonstrates the principles of spatial composition discussed in this thesis. A spoken informational layer sits on top of this acting as a museum audio guide, featuring navigation using head gestures for a hands-free experience. This thesis presents a detailed discussion of the artistic intentions and techniques employed in the production of the six soundscapes, as well as an evaluation of the resulting application in use in a public gallery space