Physical simulation for monocular 3D model based tracking

The problem of model-based object tracking in three dimensions is addressed. Most previous work on tracking assumes simple motion models, and consequently tracking typically fails in a variety of situations. Our insight is that incorporating physics models of object behaviour improves tracking performance in these cases. In particular it allows us to handle tracking in the face of rigid body interactions where there is also occlusion and fast object motion. We show how to incorporate rigid body physics simulation into a particle filter. We present two methods for this based on pose and force noise. The improvements are tested on four videos of a robot pushing an object, and results indicate that our approach performs considerably better than a plain particle filter tracker, with the force noise method producing the best results over the range of test videos

Bayesian inference application to burglary detection

Real time motion tracking is very important for video analytics. But very little research has been done in identifying the top-level plans behind the atomic activities evident in various surveillance footages [61]. Surveillance videos can contain high level plans in the form of complex activities [61]. These complex activities are usually a combination of various articulated activities like breaking windshield, digging, and non-articulated activities like walking, running. We have developed a Bayesian framework for recognizing complex activities like burglary. This framework (belief network) is based on an expectation propagation algorithm [8] for approximate Bayesian inference. We provide experimental results showing the application of our framework for automatically detecting burglary from surveillance videos in real time

Audio‐Visual Speaker Tracking

Target motion tracking found its application in interdisciplinary fields, including but not limited to surveillance and security, forensic science, intelligent transportation system, driving assistance, monitoring prohibited area, medical science, robotics, action and expression recognition, individual speaker discrimination in multi‐speaker environments and video conferencing in the fields of computer vision and signal processing. Among these applications, speaker tracking in enclosed spaces has been gaining relevance due to the widespread advances of devices and technologies and the necessity for seamless solutions in real‐time tracking and localization of speakers. However, speaker tracking is a challenging task in real‐life scenarios as several distinctive issues influence the tracking process, such as occlusions and an unknown number of speakers. One approach to overcome these issues is to use multi‐modal information, as it conveys complementary information about the state of the speakers compared to single‐modal tracking. To use multi‐modal information, several approaches have been proposed which can be classified into two categories, namely deterministic and stochastic. This chapter aims at providing multimedia researchers with a state‐of‐the‐art overview of tracking methods, which are used for combining multiple modalities to accomplish various multimedia analysis tasks, classifying them into different categories and listing new and future trends in this field

Design and Implementation of a Framework for the Interconnection of Cellular Automata in Software and Hardware

There has been a move recently in academia, industry, and the consumer space towards the use of unsupervised parallel computation and distributed networks (i.e., networks of computing elements working together to achieve a global outcome with only local knowledge). To fully understand the types of problems that these systems are applied to regularly, a representative member of this group of unsupervised parallel and distributed systems is needed to allow the development of generalizable results. Although not the only potential candidate, the field of cellular automata is an excellent choice for analyzing how these systems work as it is one of the simplest members of this group in terms of design specification. The current ability of the field of cellular automata to represent the realm of unsupervised parallel and distributed systems is limited to only a subset of the possible systems, which leads to the main goal of this work of finding a method of allowing cellular automata to represent a much larger range of systems. To achieve this goal, a conceptual framework has been developed that allows the definition of interconnected systems of cellular automata that can represent most, if not all, unsupervised parallel and distributed systems. The framework introduces the concept of allowing the boundary conditions of a cellular automaton to be defined by a separately specified system, which can be any system that is capable of producing the information needed, including another cellular automaton. Using this interconnection concept, two forms of computational simplification are enabled: the deconstruction of a large system into smaller, modular pieces; and the construction of a large system built from a heterogeneous set of smaller pieces. This framework is formally defined using an interconnection graph, where edges signify the flow of information from one node to the next and the nodes are the various systems involved. A library has been designed which implements the interconnection graphs defined by the framework for a subset of the possible nodes, primarily to allow an exploration of the field of cellular automata as a potential representational member of unsupervised parallel and distributed systems. This library has been developed with a number of criteria in mind that will allow it to be instantiated on both hardware and software using an open and extendable architecture to enable interaction with external systems and future expansion to take into account novel research. This extendability is discussed in terms of combining the library with genetic algorithms to find an interconnected system that will satisfy a specific computational goal. There are also a number of novel components of the library that further enhance the capabilities of potential research, including methods for automatically building interconnection graphs from sets of cellular automata and the ability to skip over static regions of a given cellular automaton in an intelligent way to reduce computation time. With a particular set of cellular automaton parameters, the use of this feature reduced the computation time by 75%. As a demonstration of the usefulness of both the library and the framework that it implements, a hardware application has been developed which makes use of many of the novel aspects that have been introduced to produce an interactive art installation named 'Aurora'. This application has a number of design requirements that are directly achieved through the use of library components and framework definitions. These design requirements included a lack of centralized control or data storage, a need for visibly dynamic behaviour in the installation, and the desire for the visitors to the installation to be able to affect the visible movement of patterns across the surface of the piece. The success of the library in this application was heavily dependent on its instantiation on a mixture of hardware and software, as well as the ability to extend the library to suit particular needs and aspects of the specific application requirements. The main goal of this thesis research, finding a method that allows cellular automata to represent a much larger range of unsupervised parallel and distributed systems, has been partially achieved in the creation of a novel framework which defines the concept of interconnection, and the design of an interconnection graph using this concept. This allows the field of cellular automata, in combination with the framework, to be an excellent representational member of an extended set of unsupervised parallel and distributed systems when compared to the field alone. A library has been developed that satisfies a broad set of design criteria that allow it to be used in any future research built on the use of cellular automata as this representational member. A hardware application was successfully created that makes use of a number of novel aspects of both the framework and the library to demonstrate their applicability in a real world situation

Systems and Algorithms for Automated Collaborative Observation using Networked Robotic Cameras

The development of telerobotic systems has evolved from Single Operator Single Robot (SOSR) systems to Multiple Operator Multiple Robot (MOMR) systems. The relationship between human operators and robots follows the master-slave control architecture and the requests for controlling robot actuation are completely generated by human operators. Recently, the fast evolving advances in network and computer technologies and decreasing size and cost of sensors and robots enable us to further extend the MOMR system architecture to incorporate heterogeneous components such as humans, robots, sensors, and automated agents. The requests for controlling robot actuation are generated by all the participants. We term it as the MOMR++ system. However, to reach the best potential and performance of the system, there are many technical challenges needing to be addressed. In this dissertation, we address two major challenges in the MOMR++ system development. We first address the robot coordination and planning issue in the application of an autonomous crowd surveillance system. The system consists of multiple robotic pan-tilt-zoom (PTZ) cameras assisted with a fixed wide-angle camera. The wide-angle camera provides an overview of the scene and detects moving objects, which are required for close-up views using the PTZ cameras. When applied to the pedestrian surveillance application and compared to a previous work, the system achieves increasing number of observed objects by over 210% in heavy traffic scenarios. The key issue here is given the limited number (e.g., p (p > 0)) of PTZ cameras and many more (e.g., n (n >> p)) observation requests, how to coordinate the cameras to best satisfy all the requests. We formulate this problem as a new camera resource allocation problem. Given p cameras, n observation requests, and [epsilon] being approximation bound, we develop an approximation algorithm running in O(n/[epsilon]³ + p²/[epsilon]⁶) time, and an exact algorithm, when p = 2, running in O(n³) time. We then address the automatic object content analysis and recognition issue in the application of an autonomous rare bird species detection system. We set up the system in the forest near Brinkley, Arkansas. The camera monitors the sky, detects motions, and preserves video data for only those targeted bird species. During the one-year search, the system reduces the raw video data of 29.41TB to only 146.7MB (reduction rate 99.9995%). The key issue here is to automatically recognize the flying bird species. We verify the bird body axis dynamic information by an extended Kalman filter (EKF) and compare the bird dynamic state with the prior knowledge of the targeted bird species. We quantify the uncertainty in recognition due to the measurement uncertainty and develop a novel Probable Observation Data Set (PODS)-based EKF method. In experiments with real video data, the algorithm achieves 95% area under the receiver operating characteristic (ROC) curve. Through the exploration of the two MOMR++ systems, we conclude that the new MOMR++ system architecture enables much wider range of participants, enhances the collaboration and interaction between participants so that information can be exchanged in between, suppresses the chance of any individual bias or mistakes in the observation process, and further frees humans from the control/observation process by providing automatic control/observation. The new MOMR++ system architecture is a promising direction for future telerobtics advances