A Real-Time Face and Writing Tracking System Based on the Combination of a PTZ Camera and a Static Camera

It is obvious that the excellence of education is boosted by the rise of E-learning, which provides electronic learning material through the Internet. The online education benefits more people than traditional classroom education by enabling anyone to participate in the learning process, regardless of the identity, location and personal schedule. The goal of lifetime study can be achieved in this way. In early years, the online lectures are recorded with static cameras placed at a fixed position, and result in visual effects of low quality. Currently, many online education systems employ operators to control static cameras or PTZ cameras to record lectures. The wages of operators increase the system cost, and the complex algorithm raises the computational power and slows down the running of the system. This thesis proposes a real-time face, upper body and writing detection and tracking system for online education. The recording equipment includes a PTZ camera and a static camera. It saves the hardware cost by lowering the number of cameras involved in the system and eliminating the need for operators. The controlling software makes use of OpenCv library to simplify and accelerate the image processing and execute face detection. The proposed system implements an efficient motion detection algorithm to detect writing action and rigorous controlling logic to switch views between the instructor and the board. The proposed system is thought to be economical from the perspective of hardware and efficient and accurate from the perspective of software. The recording is capable of keeping pace with the instructor and switching the view of the instructor to the view of the content written on the board smoothly as if an actual operator controls the camera. The proposed system brings competitive results regarding detection and tracking performance compared with conventional systems

A Real-Time Face Tracking System Based On A Single PTZ Camera

It is evident that the effectiveness of education is strengthened from the assistance of distance learning which provides lectures online. The benefits of online education include enabling more distance courses based on local programs and increasing participation and interaction between the students and the instructor. Nowadays, conventional systems commonly implement a combination of static and PTZ cameras. However, such systems are not only costly but also require operators and high computational power in exchange. Thus, this thesis proposes a real-time face tracking system based on a single PTZ camera as a cost effective solution by minimizing hardware requirements and functioning automatically. The proposed system focuses on the delay possible to occur due to the movement of the PTZ camera and the network delay which varies the video frame rate which alters the performance from a software perspective. The main contributions include the low cost and flexibility regarding installation. Preliminaries are introduced as a basis of the proposed system such that hardware is maintained to be minimal and universal while software is retained to use less computational power. The proposed system minimizes the delays to maintain pace with the subject of interest, provides a smooth and natural movement of the camera as if an actual operator controls the camera, and produces competitive results regarding performance compared to conventional systems

Real time tracking using an active pan-tilt-zoom network camera

Abstract—We present here a real time active vision system on a PTZ network camera to track an object of interest. We address two critical issues in this paper. One is the control of the camera through network communication to follow a selected object. The other is to track an arbitrary type of object in real time under conditions of pose, viewpoint and illumination changes. We analyze the difficulties in the control through the network and propose a practical solution for tracking using a PTZ network camera. Moreover, we propose a robust real time tracking approach, which enhances the effectiveness by using complementary features under a two-stage particle filtering framework and a multi-scale mechanism. To improve time performance, the tracking algorithm is implemented as a multi-threaded process in OpenMP. Comparative experiments with state-of-the-art methods demonstrate the efficiency and robustness of our system in various applications such as pedestrian tracking, face tracking, and vehicle tracking. I

Real-Time, Multiple Pan/Tilt/Zoom Computer Vision Tracking and 3D Positioning System for Unmanned Aerial System Metrology

The study of structural characteristics of Unmanned Aerial Systems (UASs) continues to be an important field of research for developing state of the art nano/micro systems. Development of a metrology system using computer vision (CV) tracking and 3D point extraction would provide an avenue for making these theoretical developments. This work provides a portable, scalable system capable of real-time tracking, zooming, and 3D position estimation of a UAS using multiple cameras. Current state-of-the-art photogrammetry systems use retro-reflective markers or single point lasers to obtain object poses and/or positions over time. Using a CV pan/tilt/zoom (PTZ) system has the potential to circumvent their limitations. The system developed in this paper exploits parallel-processing and the GPU for CV-tracking, using optical flow and known camera motion, in order to capture a moving object using two PTU cameras. The parallel-processing technique developed in this work is versatile, allowing the ability to test other CV methods with a PTZ system using known camera motion. Utilizing known camera poses, the object\u27s 3D position is estimated and focal lengths are estimated for filling the image to a desired amount. This system is tested against truth data obtained using an industrial system

Face Recognition from Weakly Labeled Data

Recognizing the identity of a face or a person in the media usually requires lots of training data to design robust classifiers, which demands a great amount of human effort for annotation. Alternatively, the weakly labeled data is publicly available, but the labels can be ambiguous or noisy. For instance, names in the caption of a news photo provide possible candidates for faces appearing in the image. Names in the screenplays are only weakly associated with faces in the videos. Since weakly labeled data is not explicitly labeled by humans, robust learning methods that use weakly labeled data should suppress the impact of noisy instances or automatically resolve the ambiguities in noisy labels. We propose a method for character identification in a TV-series. The proposed method uses automatically extracted labels by associating the faces with names in the transcripts. Such weakly labeled data often has erroneous labels resulting from errors in detecting a face and synchronization. Our approach achieves robustness to noisy labeling by utilizing several features. We construct track nodes from face and person tracks and utilize information from facial and clothing appearances. We discover the video structure for effective inference by constructing a minimum-distance spanning tree (MST) from the track nodes. Hence, track nodes of similar appearance become adjacent to each other and are likely to have the same identity. The non-local cost aggregation step thus serves as a noise suppression step to reliably recognize the identity of the characters in the video. Another type of weakly labeled data results from labeling ambiguities. In other words, a training sample can have more than one label, and typically one of the labels is the true label. For instance, a news photo is usually accompanied by the captions, and the names provided in the captions can be used as the candidate labels for the faces appearing in the photo. Learning an effective subject classifier from the ambiguously labeled data is called ambiguously labeled learning. We propose a matrix completion framework for predicting the actual labels from the ambiguously labeled instances, and a standard supervised classifier that subsequently learns from the disambiguated labels to classify new data. We generalize this matrix completion framework to handle the issue of labeling imbalance that avoids domination by dominant labels. Besides, an iterative candidate elimination step is integrated with the proposed approach to improve the ambiguity resolution. Recently, video-based face recognition techniques have received significant attention since faces in a video provide diverse exemplars for constructing a robust representation of the target (i.e., subject of interest). Nevertheless, the target face in the video is usually annotated with minimum human effort (i.e., a single bounding box in a video frame). Although face tracking techniques can be utilized to associate faces in a single video shot, it is ineffective for associating faces across multiple video shots. To fully utilize faces of a target in multiples-shot videos, we propose a target face association (TFA) method to obtain a set of images of the target face, and these associated images are then utilized to construct a robust representation of the target for improving the performance of video-based face recognition task. One of the most important applications of video-based face recognition is outdoor video surveillance using a camera network. Face recognition in outdoor environment is a challenging task due to illumination changes, pose variations, and occlusions. We present the taxonomy of camera networks and discuss several techniques for continuous tracking of faces acquired by an outdoor camera network as well as a face matching algorithm. Finally, we demonstrate the real-time video surveillance system using pan-tilt-zoom (PTZ) cameras to perform pedestrian tracking, localization, face detection, and face recognition

Estudi de l’algorítmica en el seguiment d’objectes mòbils amb càmera activa

La capacitat d’identificar i seguir de forma ràpida, precisa i autònoma un objecte en temps real a partir de fotogrames capturats amb una càmera activa ha estat una àrea atractiva de recerca en els últims anys. Però no ha estat fins aquestes dues darreres dècades que, gràcies als avanços tecnològics quant a potència computacional i fàcil accés a hardware òptic comercial, aquesta capacitat s’ha convertit en una possibilitat real. Una plètora d’estratègies i mètodes amb enfocaments molt diferents per identificar i seguir objectes d’interès es poden trobar a la literatura. En aquest projecte de fi de grau, de bon començament, amb el propòsit de fer el seguiment d’un objecte mòbil amb un sistema de visió activa, es presenten els conceptes bàsics sobre els quals s’emmarcarà el treball; i, a continuació, s’examinen en detall alguns dels algoritmes que per la seva eficiència computacional han jugat un paper més rellevant en aquests tipus d’aplicacions de seguiment en temps real. Concretament s’estudien els detectors de moviment centrats en el modelatge de fons i el flux òptic; i l’algoritme de seguiment d’objectes CAMShift. En addició, s’analitzen com aquests se les enginyen en situacions de càmera mòbil i les contramesures basades en la compensació del moviment del fons que poden promulgar-se per adaptar-los en aquestes noves condicions. Altrament, amb la finalitat d’integrar els algoritmes de visió analitzats en un sistema de seguiment de visió per computador amb càmera activa, s’ha dut a terme una cerca de les diferents lleis de control basades en imatge i s’ha indagat sobre si la incorporació d’un filtre de Kalman produeix millores en l’estimació de la posició de l’objecte seguit. És més, s’ha detallat el desenvolupament d’un sistema de visió activa conformat per la combinació d’una càmera USB i una unitat pan/tilt que, juntament amb codi personalitzat, ha permès automatitzar la tasca d’apuntar i seguir de forma contínua el moviment d’un objecte. Finalment, s’ha construït un prototipus funcional demostrador que ha permès verificar el comportament del sistema via resultats experimental