要約のみTohoku University橋本浩一課

Chen Min

English

Tohoku University博士（情報科学）博士学位論文 (Thesis(doctor))要約のみthesi

Chen, Min

Tohoku University Repository (TOUR) / 東北大学機関リポジトリ

Vision Based System for Measuring andEstimating Navigation Activities of FreelyBehaving Animals著者 陳 ?学位授与機関 Tohoku UniversityURL http://hdl.handle.net/10097/00127336  1 Vision Based System for Measuring and Estimating Navigation Activities of Freely Behaving Animals（非拘束状態動物のナビゲーション行動を計測及び推定するためのビジョンシステム） Min Chen Abstract  Chapter 1 Introduction Animal navigation is the ability of animals to find their way accurately in their environment. It has been studied by biologists over more than a hundred years. Originally, studies on animal navigation were mainly conducted by observing the corresponding animals directly, at the macro level. Although some clues of animal navigation can be obtained by animal observation at the macro level, details on how various information is exactly processed in these functions are still remained mystery. Nowadays, studies are performed at a much lower level, tracing these phenomena using more advanced techniques such as manipulation on neural cells or adoption of genetic operations, trying to explain the most fundamental mechanisms of animal navigation. Many of these studies largely rely on tool sets used in experiments for measuring or quantifying information involved in the corresponding context. In navigation, three functions are generally believed to be significant for an animal: the acquisition of dynamically changing information from external and internal environment, the choice of route and destination based on the information, and the behavioral regulation to reach the destination. In order to understand the most fundamental mechanisms for animal navigation, it is essential to study how these functions interact with another. Therefore, it is desired that the animal under observation is behaving in a free and natural manner, with fewest possible restrictions placed on them. Observation under such conditions would be frustrating, if performed manually by a human observer, without automated aiding tools. Even obtaining very little data requires great human efforts. We use vision based measuring systems for this purpose. To obtain information from any interested target, it is a natural requirement that less disturbance is introduced during the process of data acquisition. In the case of observing animal behaviors, we generally expect that an animal under observation is less affected by any kind of devices we use for observing. It is obvious that visual sensors have great advantages in this aspect. Despite the necessity to illuminate targets when lighting condition is insufficient, visual sensors are passive sensors, and will not introduce extra disturbance. Compared with other devices that have to be attached to observation targets, remotely set camera does not introduce any load to the target animal, and it is possible to be used on animals of very smaller sizes. In addition, because of the nature of human, visual representation is always the richest source of information. Visual sensors can provide most human intuitive data, and they are nowadays preferred as tools for recording raw data. Visual sensors are extensively used in experiments including animal observation. However, to reliably measure, or quantify abstract information from visual data is difficult. Because of the complexity of vision problems, methodologies processing vision data typically adopt many human priors so as to simplify the problem under concern. Consequently, general   2 purpose vision applications can hardly be generalized to situations where those assumed priors are violated. So are situations involved with animals. It is worth noting that most animals typically live in environments significantly different from human custom. Many articulated “general purpose” vision techniques are not so “general” when put into contexts unfamiliar to our species. Animals may live in worlds of much different scales with things in quite different appearance. Many commonly used vision techniques become inapplicable. In this study, we develop vision based systems that apply various vision techniques onto specific systems for automatically measuring navigation behaviors of freely moving animals. These systems are tested within real situations and proved to be applicable for automating processes of measuring or quantifying information involved in animal navigation.  Chapter 2 Measuring behaviors of microorganism This chapter describes the microscope system developed for measuring behaviors of microorganisms. Since a motile organism is always moving in navigation, an electric stage is introduced to compensate for the displacement caused by the motion of the target animal. The stage, essentially, need to be controlled in real time according to the position of the target. Thus, it is required that the overall system is able to response within a given short time, which include, among all, time for the visual data to be processed, and that for the physical system to react. The system, accordingly, is a real time control system. It is critical in the system that the current position of an interested region can be estimated accurately within short time. The process of estimating the position is modeled as an image registration problem. An image patch need to be specified by the practitioner as the template image at the beginning. Then at every time stamp, algorithms try to search for the region in view that is most similar to the template. Because of the absence of distinct textures in tissue like appearances, it is difficult to robustly track some region over long time with real time constraint. I introduce a multiple tracker method to tackle the problem. The multiple tracker method adopts a fast tracker and a robust tracker simultaneously. The fast tracker searches locally for the minimized sum of squared error between the template and an image patch, but it may suffer from the problem of divergence for some cases. The robust tracker exhaustively searches for maximized phase correlation between the template and an image image patch. A schedule for updating the overall estimation is proposed so both estimation results can be fused properly. With this tracking method, the microscope system can robustly track tissue like patterns over long time. The proposed system has been successfully applied to measuring behaviors of C. elegans, which is a kind of worm like animal popularly studied in neural science. The neuronal activities of C. elegans can be measured using optogenetic techniques. Together with optogenetic equipment, the proposed base system is extended to multiple specialized systems for advanced research purposes, which already helped accelerate scientific discoveries in biological studies.  Chapter 3 Measuring behaviors of flying animals   3 This chapter describes the two view camera system developed for measuring behaviors of flying animals. The tracking system introduced in chapter 2 is mainly designed for measuring behaviors of microorganisms. Motion of microorganisms is approximately confined within 2D plane, imposed by the base observation device, the optical microscope. Therefore, visual tracking can be achieved by adopting appearance based methods, assuming the top view of a target does not change rapidly, which is reasonable, since the camera is positioned far away in the vertical direction of the plane under observation. However, for larger animals that move in a wider space, such view setting is often infeasible in practice given limited budget. Among them are flying animals that move in 3D space. The position in 3D space is significant information in their navigation and needs to be measured properly. To obtain information of these animals, different system setups and vision techniques are required. A vision system is developed for tracking flying animals and reconstructing their locations in 3D space. The system uses two cameras placed fixed at one side of the assumed space where target animals are flying, and it captures the motion of the targets with high frame rate. It is difficult to track flying animals within unknown environment in image sequence, because the appearance of any target will be always significantly deforming, caused by changes in pose, perspective, lighting condition, depth, focus, etc. General purpose will typically fail within a few frames in such situations. A novel algorithm is proposed for this tracking purpose, adopting sparse appearance representation and quick updating schedule while exploiting filtering techniques. The algorithm uses a set of sparsely distributed points as the main elements for representing a given target. These points are tracked respectively, before refining the estimation of tracking using a statistical model. With the results in this step, the point set is updated, removing bad points and adding new candidate points with careful schedule. Subsequently, Kalman filter is applied to void possible drifting problem, fusing flow information and positional information. The proposed tracker can track flying targets over very long duration, and it does not require strictly conditioned environment for targets to be measured. With the tracking results, 3D locations of targets can then be automatically reconstructed using the system.  Sets of experiments are conducted tracking bats, a typical kind of nocturnal animals, under low lighting condition using infrared radiation (IR) as illumination source. Image sequences are recorded simultaneously using the two cameras. With the recorded images, trajectories of the bats are successfully reconstructed using the system.  Chapter 4 Estimation and prediction of animal behavior The purpose of observing animals is to clarify the dynamics within these animals, such as their innate behavioral regulations, their responses against certain environmental factors, or the mutual interactions of certain internal states within their body. While previous chapters focus on automatic tools for measuring animal behaviors, this chapter describes methods for aiding the process of modeling motions of animals, using machine learning based techniques. To model animal behaviors, we can view the target animal as a general input output system.   4 When a model can be used to predict future behaviors of a target, the model is approximately correct. In such a system, the system output is the future event that we are to predict; the system input is the pre-defined elementary information that we must know so as to make the prediction; and the system states include any information that are not directly known but has any influences on the future event. Because of the complexity of biological systems, an animal is often viewed as a black box. In this case, neural networks are useful techniques for identifying unknown models. To model the dynamics of animal motion, long short term memory network is used. Measurement can be obtained easily using previously proposed systems. These data are pre-processed, removing noises, before being used to trained the neural network model. Experiments are conducted verifying the capability of using networks to predict future motions of animals. The results suggest that the trained network model can predict future motion states of the target animals. Especially, the non-intuitive motion pattern of C. elegans was successfully learned by a LSTM network model.  Chapter 5 Conclusions and future work Navigation is one of the most fundamental behaviors for moving animals including human. Various information involved in navigation need to be measure properly. To do this, vision system is considered as an optimal solution, compared with other sensing techniques. The difficulty to reliably measure or quantify desired information from these vision data is a main challenge in vision systems. Different systems are developed for such measuring purposes. The microscope system described in Chapter 2 provides a solution for real time tracking of microorganisms over long period, recording behaviors of the targets along with its environment. A key technology in the system is the real time visual tracking algorithm used for robustly locking on a specific region on target animal, whose appearance is usually tissue like. The system achieved great success in measuring behaviors of C. elegans, and it is further extended into multiple specialized systems for advanced research purposes, including calcium imaging of neural activities and optogenetic control. The two view camera system in Chapter 3 facilitates automatic reconstruction of 3D location of flying animals. A tracker algorithm is proposed to tackle this problem, robust against poor lighting conditions, deformation, perspective changes, etc. With this tracker, target animal can be robustly tracked within loosely conditioned environment. 3D locations of a flying target is estimated successfully in experiments conducted on flying bats in confined indoor space. Methodologies are discussed in Chapter 4 for modeling animal behaviors. Types of filtering techniques are first reviewed and experimented, with their characteristics compared respectively. This is followed by discussion on system identification problem, where estimation methods and neural network models are mainly introduced. Experiment on modeling navigation behavior of C. elegans using LSTM network is performed, and the success in learning the non-intuitive motion pattern indicates the applicability of modeling animal dynamics using neural networks. However, there are still many tasks remained as future work. For example, in the system of measuring behaviors of flying animals, further improvement is under consideration. One might   5 be to adapt the system to situations in open space, so animals in the wild can also be observed. Automatic extraction of other types of information, such as the pose changes of a target, are also challenging tasks. 

Vision Based System for Measuring and Estimating Navigation Activities of Freely Behaving Animals

https://tohoku.repo.nii.ac.jp/?action=repository_action_common_download&item_id=129941&item_no=1&attribute_id=18&file_no=1

非拘束状態動物のナビゲーション行動を計測及び推定するためのビジョンシステム

Abstract

Similar works

Full text

Available Versions

Tohoku University Repository (TOUR) / 東北大学機関リポジトリ