Human extremity detection and its applications in action detection and recognition

textIt is proven that locations of internal body joints are sufficient visual cues to characterize human motion. In this dissertation I propose that locations of human extremities including heads, hands and feet provide powerful approximation to internal body motion. I propose detection of precise extremities from contours obtained from image segmentation or contour tracking. Junctions of medial axis of contours are selected as stars. Contour points with a local maximum distance to various stars are chosen as candidate extremities. All the candidates are filtered by cues including proximity to other candidates, visibility to stars and robustness to noise smoothing parameters. I present my applications of using precise extremities for fast human action detection and recognition. Environment specific features are built from precise extremities and feed into a block based Hidden Markov Model to decode the fence climbing action from continuous videos. Precise extremities are grouped into stable contacts if the same extremity does not move for a certain duration. Such stable contacts are utilized to decompose a long continuous video into shorter pieces. Each piece is associated with certain motion features to form primitive motion units. In this way the sequence is abstracted into more meaningful segments and a searching strategy is used to detect the fence climbing action. Moreover, I propose the histogram of extremities as a general posture descriptor. It is tested in a Hidden Markov Model based framework for action recognition. I further propose detection of probable extremities from raw images without any segmentation. Modeling the extremity as an image patch instead of a single point on the contour helps overcome the segmentation difficulty and increase the detection robustness. I represent the extremity patches with Histograms of Oriented Gradients. The detection is achieved by window based image scanning. In order to reduce computation load, I adopt the integral histograms technique without sacrificing accuracy. The result is a probability map where each pixel denotes probability of the patch forming the specific class of extremities. With a probable extremity map, I propose the histogram of probable extremities as another general posture descriptor. It is tested on several data sets and the results are compared with that of precise extremities to show the superiority of probable extremities.Electrical and Computer Engineerin

Workshop on Emerging Surveillance Capabilities & Requirements

On 5-6 July 2011, the Joint Research Center (JRC) of the European Commission organized a workshop on “Emerging Surveillance Capabilities and Requirements”. Around 40 R&D managers, researchers and practitioners from key European RTD institutes or academia, operators and leading system / equipment providers attended the workshop. The workshop was hosted by the Institute for the Protection and Security of the Citizen (IPSC) at JRC (http://ipsc.jrc.ec.europa.eu/), in Ispra, Italy. The purpose of the workshop was to provide a forum where key scientists and practitioners could meet to address the following issues: (a) Review the technology state of the art and the operational requirements; (b) Anticipate technology trends and future needs, including testing and standards; (c) Consider legal, regulatory and ethical aspects. Some key-elements which arose during these sessions are worth being highlighted here: The overview of current European projects and national initiatives showed some common concerns in terms of issues being addressed and including the following main topics standardization, testing and performance assessment under increasingly complex scenarios (365/24/7 video analytics service level, multiple image/video sources and sensors), impact of societal requirements/issues, such as privacy and security, in the planning, design and implementation phase of surveillance systems. European networking and multi-disciplinary cooperation were often mentioned as relevant means to improve surveillance capabilities as well as to significantly enhance awareness of the societal values of concern. More precisely, the following points were cited: - Surveillance capabilities need to be federated and coordinated at the European level. - A mutual interest for cooperation for surveillance has been identified with the main aims of sharing best practices and common standards; the exchange of scientific staff as well as the use of national capabilities by other countries are some of the potential means to achieve it. The organization of workshops on the regular yearly basis was also suggested. - Trend to pervasive observation / monitoring rendering data protection and privacy even more important issues which it is suggested to address in a future workshop. Some new challenges for surveillance to be addressed in a very short-term future were introduced by several participants. A European certification framework for smart surveillance, the development of common privacy and performance testing methodologies, need for reference datasets, are some of these challenges. This report summarizes the workshop’s contents and the main findings agreed during the closing session. It also contains all the PowerPoint format documents presented except the ones for which a confidentiality clause was required.JRC.G.7-Digital Citizen Securit