An Epipolar Line from a Single Pixel

Computing the epipolar geometry from feature points between cameras with very different viewpoints is often error prone, as an object's appearance can vary greatly between images. For such cases, it has been shown that using motion extracted from video can achieve much better results than using a static image. This paper extends these earlier works based on the scene dynamics. In this paper we propose a new method to compute the epipolar geometry from a video stream, by exploiting the following observation: For a pixel p in Image A, all pixels corresponding to p in Image B are on the same epipolar line. Equivalently, the image of the line going through camera A's center and p is an epipolar line in B. Therefore, when cameras A and B are synchronized, the momentary images of two objects projecting to the same pixel, p, in camera A at times t1 and t2, lie on an epipolar line in camera B. Based on this observation we achieve fast and precise computation of epipolar lines. Calibrating cameras based on our method of finding epipolar lines is much faster and more robust than previous methods.Comment: WACV 201

Object Tracking: Appearance Modeling And Feature Learning

Object tracking in real scenes is an important problem in computer vision due to increasing usage of tracking systems day in and day out in various applications such as surveillance, security, monitoring and robotic vision. Object tracking is the process of locating objects of interest in every frame of video frames. Many systems have been proposed to address the tracking problem where the major challenges come from handling appearance variation during tracking caused by changing scale, pose, rotation, illumination and occlusion. In this dissertation, we address these challenges by introducing several novel tracking techniques. First, we developed a multiple object tracking system that deals specially with occlusion issues. The system depends on our improved KLT tracker for accurate and robust tracking during partial occlusion. In full occlusion, we applied a Kalman filter to predict the object\u27s new location and connect the trajectory parts. Many tracking methods depend on a rectangle or an ellipse mask to segment and track objects. Typically, using a larger or smaller mask will lead to loss of tracked objects. Second, we present an object tracking system (SegTrack) that deals with partial and full occlusions by employing improved segmentation methods: mixture of Gaussians and a silhouette segmentation algorithm. For re-identification, one or more feature vectors for each tracked object are used after target reappearing. Third, we propose a novel Bayesian Hierarchical Appearance Model (BHAM) for robust object tracking. Our idea is to model the appearance of a target as combination of multiple appearance models, each covering the target appearance changes under a certain situation (e.g. view angle). In addition, we built an object tracking system by integrating BHAM with background subtraction and the KLT tracker for static camera videos. For moving camera videos, we applied BHAM to cluster negative and positive target instances. As tracking accuracy depends mainly on finding good discriminative features to estimate the target location, finally, we propose to learn good features for generic object tracking using online convolutional neural networks (OCNN). In order to learn discriminative and stable features for tracking, we propose a novel object function to train OCNN by penalizing the feature variations in consecutive frames, and the tracker is built by integrating OCNN with a color-based multi-appearance model. Our experimental results on real-world videos show that our tracking systems have superior performance when compared with several state-of-the-art trackers. In the feature, we plan to apply the Bayesian Hierarchical Appearance Model (BHAM) for multiple objects tracking

Pictures in Your Mind: Using Interactive Gesture-Controlled Reliefs to Explore Art

Tactile reliefs offer many benefits over the more classic raised line drawings or tactile diagrams, as depth, 3D shape, and surface textures are directly perceivable. Although often created for blind and visually impaired (BVI) people, a wider range of people may benefit from such multimodal material. However, some reliefs are still difficult to understand without proper guidance or accompanying verbal descriptions, hindering autonomous exploration. In this work, we present a gesture-controlled interactive audio guide (IAG) based on recent low-cost depth cameras that can be operated directly with the hands on relief surfaces during tactile exploration. The interactively explorable, location-dependent verbal and captioned descriptions promise rapid tactile accessibility to 2.5D spatial information in a home or education setting, to online resources, or as a kiosk installation at public places. We present a working prototype, discuss design decisions, and present the results of two evaluation studies: the first with 13 BVI test users and the second follow-up study with 14 test users across a wide range of people with differences and difficulties associated with perception, memory, cognition, and communication. The participant-led research method of this latter study prompted new, significant and innovative developments

딥러닝 기반 단일 거리 공간 내 GPCR 단백질군 계층 구조의 동시적 모델링 기법

학위논문(석사)--서울대학교 대학원 :공과대학 컴퓨터공학부,2019. 8. 김선.G 단밸질 연결 수용체(GPCR)은 계층 구조로 형성된 다양한 단백질군으로 구성된다. 단백질 서열을 통한 GPCR에 대한 계산적인 모델링은 군(family), 아군(subfamily), 준아군(sub-subfamily)의 각 계층에서 독립적으로 실행되는 방식으로 이루어져왔다. 하지만 이러한 접근 방식들은 단절된 모델들을 통하여 단백질 내의 정보를 처리하기 때문에 GPCR 종류 사이의 관계는 고려하지 못한다는 한계를 가지고 있다. 본 연구에서는 딥러닝을 이용하여 GPCR의 계층 구조에서 나타나는 특징들을 단일한 모델로 동시적으로 학습하는 방법을 제시한다. 또한 계층적인 관계들을 하나의 벡터 공간에 거리를 통해 표현할 수 있도록 하기 위한 손실함수도 제시한다. 이 연구는 GPCR 수용체들의 여러 계층에서 공통적으로 나타나는 특징들을 학습하고 표현할 수 있도록 하는 방법을 다루고 있다. 여러 심화적인 실험들을 통하여 우리는 기술적인 측면과 생물학적인 측면에서 단백질 간 계층적인 관계가 성공적으로 학습이 되었다는 것을 보였다. 첫번째로, 우리는 임베딩 벡터에 계층적 군집화(hierarchical clustering) 알고리즘을 적용함으로써 계통수(phylogenetic tree)를 만들었고, 군집 알고리즘과 실제 계층 구조와의 수치적인 비교를 통하여 임베딩 벡터를 통해 계통학적 특징에 대한 유추가 가능하다는 것을 보였다. 두번째로, 임베딩 벡터의 군집화 결과에 다중 서열 정렬(multiple sequence alignment)를 적용시킴으로써 생물학적으로 유의미한 서열적 특성들을 찾아낼 수 있다는 것을 보였다. 이는 임베딩 벡터 분석이 GPCR 단백질 연구에 있어 효율적인 첫걸음이 될 수 있다는 것을 보여준다. 이러한 결과는 여러 계층으로 이루어진 단백질군에 대한 동시적인 모델링이 가능하다는 것을 말하고 있다.G protein-coupled receptors (GPCRs) belong to diverse families of proteins that can be defined at multiple levels. Computational modeling of GPCR families from the sequences has been performed separately at each level of family, sub-family, and sub-subfamily. However, relationships between classes are ignored in these approaches as they process the information in the sequences with a group of disconnected models. In this work, we propose a deep learning network to simultaneously learn representations in the GPCR hierarchy with a unified model and a loss term to express hierarchical relations in terms of distances in a single embedding space. The model introduces a method to learn and construct shared representations across hierarchies of the protein family. In extensive experiments, we showed that hierarchical relations between sequences are successfully captured in our model in both of technical and biological aspect. First, we showed that phylogenetic information in the sequences can be inferred from the vectors by constructing phylogenetic tree using hierarchical clustering algorithm and by quantitatively analyzing the quality of clustering results compared to the real label information. Second, inspection on embedding vectors is demonstrated to be a effective first step to-ward an analysis of GPCR proteins by showing that biologically significant sequence features can be revealed from multiple sequence alignments on clustering results on embedding vectors. Our work showed that simultaneous modeling of protein families with multiple hierarchies is possible.Abstract i Chapter Ⅰ. Introduction 1 1.1 Background 1 1.2 Motivation 3 Chapter Ⅱ. Methods 7 2.1 Data Preparation 7 2.1.1 Dataset 7 2.1.2 Data representation 7 2.2 Model architecture 8 2.2.1 Feature extractor with CNN 8 2.2.2 Embedding layer 8 2.2.3 Output layer 9 2.3 Loss function 10 2.3.1 Softmax loss 10 2.3.2 Center loss 10 2.3.3 Overall loss 12 2.4 Training procedure 13 2.5 Evaluation metric 14 2.5.1 Silhouette score 14 2.5.2 Adjusted mutual information score 15 Chapter Ⅲ. Results 17 3.1 Evaluation on hierarchical structure 17 3.1.1 Preservation of distances 17 3.1.2 Phylogenetic tree reconstruction 20 3.1.3 Quantitative evaluation on clustering results 21 3.2 Sequence analysis with embedding vectors 26 3.2.1 Technical analysis 26 3.2.2 Biological analysis 28 3.3 Classification accuracy 30 Chapter Ⅳ. Conclusion 32 References 35Maste

Embedded System Object Tracking Using Webcam

The extensive availability of hardware devices and intensive expansion their computing power have been the catalyst behind the rapid development of computer vision. In this project, an implementation of object tracking in an inexpensive and small embedded system platform is presented. The tracking system comprised of two Raspberry Pis with two different cameras used: a webcam and Raspicam module. Three communication connection models of the system are discussed in this paper for establishing communication between the two Raspberry Pis. Data sharing between these two hardware platforms is the proposed solution for resolving the limited processing power each platform possesses. The SimpleCV, an open source framework that provides free computer vision libraries that is useful for object detection and tracking algorithm development

Which One is Me?: Identifying Oneself on Public Displays

While user representations are extensively used on public displays, it remains unclear how well users can recognize their own representation among those of surrounding users. We study the most widely used representations: abstract objects, skeletons, silhouettes and mirrors. In a prestudy (N=12), we identify five strategies that users follow to recognize themselves on public displays. In a second study (N=19), we quantify the users' recognition time and accuracy with respect to each representation type. Our findings suggest that there is a significant effect of (1) the representation type, (2) the strategies performed by users, and (3) the combination of both on recognition time and accuracy. We discuss the suitability of each representation for different settings and provide specific recommendations as to how user representations should be applied in multi-user scenarios. These recommendations guide practitioners and researchers in selecting the representation that optimizes the most for the deployment's requirements, and for the user strategies that are feasible in that environment

Vision-based traffic surveys in urban environments

This paper presents a state-of-the-art, vision-based vehicle detection and type classification to perform traffic surveys from a roadside closed-circuit television camera. Vehicles are detected using background subtraction based on a Gaussian mixture model that can cope with vehicles that become stationary over a significant period of time. Vehicle silhouettes are described using a combination of shape and appearance features using an intensity-based pyramid histogram of orientation gradients (HOG). Classification is performed using a support vector machine, which is trained on a small set of hand-labeled silhouette exemplars. These exemplars are identified using a model-based preclassifier that utilizes calibrated images mapped by Google Earth to provide accurately surveyed scene geometry matched to visible image landmarks. Kalman filters track the vehicles to enable classification by majority voting over several consecutive frames. The system counts vehicles and separates them into four categories: car, van, bus, and motorcycle (including bicycles). Experiments with real-world data have been undertaken to evaluate system performance and vehicle detection rates of 96.45% and classification accuracy of 95.70% have been achieved on this data.The authors gratefully acknowledge the Royal Borough of Kingston for providing the video data. S.A. Velastin is grateful to funding received from the Universidad Carlos III de Madrid, the European Union’s Seventh Framework Programme for research, technological development and demonstration under grant agreement nº 600371, el Ministerio de Economía y Competitividad (COFUND2013-51509) and Banco Santander