Cognitive visual tracking and camera control

Cognitive visual tracking is the process of observing and understanding the behaviour of a moving person. This paper presents an efficient solution to extract, in real-time, high-level information from an observed scene, and generate the most appropriate commands for a set of pan-tilt-zoom (PTZ) cameras in a surveillance scenario. Such a high-level feedback control loop, which is the main novelty of our work, will serve to reduce uncertainties in the observed scene and to maximize the amount of information extracted from it. It is implemented with a distributed camera system using SQL tables as virtual communication channels, and Situation Graph Trees for knowledge representation, inference and high-level camera control. A set of experiments in a surveillance scenario show the effectiveness of our approach and its potential for real applications of cognitive vision

Combining Multiple Sensors for Event Detection of Older People

International audienceWe herein present a hierarchical model-based framework for event detection using multiple sensors. Event models combine a priori knowledge of the scene (3D geometric and semantic information, such as contextual zones and equipment) with moving objects (e.g., a Person) detected by a video monitoring system. The event models follow a generic ontology based on natural language, which allows domain experts to easily adapt them. The framework novelty lies on combining multiple sensors at decision (event) level, and handling their conflict using a proba-bilistic approach. The event conflict handling consists of computing the reliability of each sensor before their fusion using an alternative combination rule for Dempster-Shafer Theory. The framework evaluation is performed on multisensor recording of instrumental activities of daily living (e.g., watching TV, writing a check, preparing tea, organizing week intake of prescribed medication) of participants of a clinical trial for Alzheimer's disease study. Two fusion cases are presented: the combination of events (or activities) from heterogeneous sensors (RGB ambient camera and a wearable inertial sensor) following a deterministic fashion, and the combination of conflicting events from video cameras with partially overlapped field of view (a RGB-and a RGB-D-camera, Kinect). Results showed the framework improves the event detection rate in both cases

Visual Analysis of Extremely Dense Crowded Scenes

Visual analysis of dense crowds is particularly challenging due to large number of individuals, occlusions, clutter, and fewer pixels per person which rarely occur in ordinary surveillance scenarios. This dissertation aims to address these challenges in images and videos of extremely dense crowds containing hundreds to thousands of humans. The goal is to tackle the fundamental problems of counting, detecting and tracking people in such images and videos using visual and contextual cues that are automatically derived from the crowded scenes. For counting in an image of extremely dense crowd, we propose to leverage multiple sources of information to compute an estimate of the number of individuals present in the image. Our approach relies on sources such as low confidence head detections, repetition of texture elements (using SIFT), and frequency-domain analysis to estimate counts, along with confidence associated with observing individuals, in an image region. Furthermore, we employ a global consistency constraint on counts using Markov Random Field which caters for disparity in counts in local neighborhoods and across scales. We tested this approach on crowd images with the head counts ranging from 94 to 4543 and obtained encouraging results. Through this approach, we are able to count people in images of high-density crowds unlike previous methods which are only applicable to videos of low to medium density crowded scenes. However, the counting procedure just outputs a single number for a large patch or an entire image. With just the counts, it becomes difficult to measure the counting error for a query image with unknown number of people. For this, we propose to localize humans by finding repetitive patterns in the crowd image. Starting with detections from an underlying head detector, we correlate them within the image after their selection through several criteria: in a pre-defined grid, locally, or at multiple scales by automatically finding the patches that are most representative of recurring patterns in the crowd image. Finally, the set of generated hypotheses is selected using binary integer quadratic programming with Special Ordered Set (SOS) Type 1 constraints. Human Detection is another important problem in the analysis of crowded scenes where the goal is to place a bounding box on visible parts of individuals. Primarily applicable to images depicting medium to high density crowds containing several hundred humans, it is a crucial pre-requisite for many other visual tasks, such as tracking, action recognition or detection of anomalous behaviors, exhibited by individuals in a dense crowd. For detecting humans, we explore context in dense crowds in the form of locally-consistent scale prior which captures the similarity in scale in local neighborhoods with smooth variation over the image. Using the scale and confidence of detections obtained from an underlying human detector, we infer scale and confidence priors using Markov Random Field. In an iterative mechanism, the confidences of detections are modified to reflect consistency with the inferred priors, and the priors are updated based on the new detections. The final set of detections obtained are then reasoned for occlusion using Binary Integer Programming where overlaps and relations between parts of individuals are encoded as linear constraints. Both human detection and occlusion reasoning in this approach are solved with local neighbor-dependent constraints, thereby respecting the inter-dependence between individuals characteristic to dense crowd analysis. In addition, we propose a mechanism to detect different combinations of body parts without requiring annotations for individual combinations. Once human detection and localization is performed, we then use it for tracking people in dense crowds. Similar to the use of context as scale prior for human detection, we exploit it in the form of motion concurrence for tracking individuals in dense crowds. The proposed method for tracking provides an alternative and complementary approach to methods that require modeling of crowd flow. Simultaneously, it is less likely to fail in the case of dynamic crowd flows and anomalies by minimally relying on previous frames. The approach begins with the automatic identification of prominent individuals from the crowd that are easy to track. Then, we use Neighborhood Motion Concurrence to model the behavior of individuals in a dense crowd, this predicts the position of an individual based on the motion of its neighbors. When the individual moves with the crowd flow, we use Neighborhood Motion Concurrence to predict motion while leveraging five-frame instantaneous flow in case of dynamically changing flow and anomalies. All these aspects are then embedded in a framework which imposes hierarchy on the order in which positions of individuals are updated. The results are reported on eight sequences of medium to high density crowds and our approach performs on par with existing approaches without learning or modeling patterns of crowd flow. We experimentally demonstrate the efficacy and reliability of our algorithms by quantifying the performance of counting, localization, as well as human detection and tracking on new and challenging datasets containing hundreds to thousands of humans in a given scene

Gait analysis advancements: rehabilitation value and new perspectives from forensic application

The clinical and rehabilitation value of gait analysis is remarkable and indisputable and poised to grow as technological advancements unfold. This article aims to shed light on the advances in how gait is assessed, enabling those who have suffered an injury impairing their motor skills to be diagnosed more accurately and efficiently as well as to compare the hallmarks of rehabilitative and forensic gait analysis. The authors have conducted an analysis of relevant papers (published between 1967 and 2020) from a medicolegal perspective, cited in PubMed, MEDLINE, Cochrane Library, EMBASE, and available recommendations for the legal application of such techniques. Moreover, considering the use of gait analysis as a forensic tool, this study broadens the scope of research by including search engines, legal databases, and court filings (DeJure, Lexis Nexis, Justia) between 2000 and 2022. The instrumental assessment of movement (Gait Analysis) has come to constitute an essential analytical tool for the biomedical sector to objectively and accurately assess human movement and posture. The article is also aimed at elaborating differences and similarities between clinical and forensic gait analysis. When it comes to the forensic applicability of gait analysis and its evidentiary value, however, there is a pressing need for a review of its scientific basis. Therefore, it is necessary to conduct a thorough evaluation of its use in legal practice, as stressed in scientific literature and surveys. It is of utmost importance to highlight the procedural and assessment standards currently applied to forensic gait analysis, to evaluate its strengths and weaknesses, and to achieve standardized guidelines based on broad scientific consensus

FUSION FRAMEWORK FOR VIDEO EVENT RECOGNITION

International audienceThis paper presents a multisensor fusion framework for video activities recognition based on statistical reasoning and D-S evidence theory. Precisely, the framework consists in the combination of the events' uncertainty computation with the trained database and the fusion method based on the conflict management of evidences. Our framework aims to build Multisensor fusion architecture for event recognition by combining sensors, dealing with conflicting recognition, and improving their performance. According to a complex event's hierarchy, Primitive state is chosen as our target event in the framework. A RGB camera and a RGB-D camera are used to recognise a person's basic activities in the scene. The main convenience of the proposed framework is that it firstly allows adding easily more possible events into the system with a complete structure for handling uncertainty. And secondly, the inference of Dempster-Shafer theory resembles human perception and fits for uncertainty and conflict management with incomplete information. The cross-validation of real-world data (10 persons) is carried out using the proposed framework, and the evaluation shows promising results that the fusion approach has an average sensitivity of 93.31% and an average precision of 86.7%. These results are better than the ones when only one camera is used, encouraging further research focusing on the combination of more sensors with more events, as well as the optimization of the parameters in the framework for improvements

A REAL-TIME TRAFFIC CONDITION ASSESSMENT AND PREDICTION FRAMEWORK USING VEHICLE-INFRASTRUCTURE INTEGRATION (VII) WITH COMPUTATIONAL INTELLIGENCE

This research developed a real-time traffic condition assessment and prediction framework using Vehicle-Infrastructure Integration (VII) with computational intelligence to improve the existing traffic surveillance system. Due to the prohibited expenses and complexity involved for the field experiment of such a system, this study adopted state-of-the-art simulation tools as an efficient alternative. This work developed an integrated traffic and communication simulation platform to facilitate the design and evaluation of a wide range of online traffic surveillance and management system in both traffic and communication domain. Using the integrated simulator, the author evaluated the performance of different combination of communication medium and architecture. This evaluation led to the development of a hybrid VII framework exemplified by hierarchical architecture, which is expected to eliminate single point failures, enhance scalability and easy integration of control functions for traffic condition assessment and prediction. In the proposed VII framework, the vehicle on-board equipments and roadside units (RSUs) work collaboratively, based on an intelligent paradigm known as \u27Support Vector Machine (SVM),\u27 to determine the occurrence and characteristics of an incident with the kinetics data generated by vehicles. In addition to incident detection, this research also integrated the computational intelligence paradigm called \u27Support Vector Regression (SVR)\u27 within the hybrid VII framework for improving the travel time prediction capabilities, and supporting on-line leaning functions to improve its performance over time. Two simulation models that fully implemented the functionalities of real-time traffic surveillance were developed on calibrated and validated simulation network for study sites in Greenville and Spartanburg, South Carolina. The simulation models\u27 encouraging performance on traffic condition assessment and prediction justifies further research on field experiment of such a system to address various research issues in the areas covered by this work, such as availability and accuracy of vehicle kinetic and maneuver data, reliability of wireless communication, maintenance of RSUs and wireless repeaters. The impact of this research will provide a reliable alternative to traditional traffic sensors to assess and predict the condition of the transportation system. The integrated simulation methodology and open source software will provide a tool for design and evaluation of any real-time traffic surveillance and management systems. Additionally, the developed VII simulation models will be made available for use by future researchers and designers of other similar VII systems. Future implementation of the research in the private and public sector will result in new VII related equipment in vehicles, greater control of traffic loading, faster incident detection, improved safety, mitigated congestion, and reduced emissions and fuel consumption

Framework for integrated oil pipeline monitoring and incident mitigation systems

Wireless Sensor Nodes (motes) have witnessed rapid development in the last two decades. Though the design considerations for Wireless Sensor Networks (WSNs) have been widely discussed in the literature, limited investigation has been done for their application in pipeline surveillance. Given the increasing number of pipeline incidents across the globe, there is an urgent need for innovative and effective solutions for deterring the incessant pipeline incidents and attacks. WSN pose as a suitable candidate for such solutions, since they can be used to measure, detect and provide actionable information on pipeline physical characteristics such as temperature, pressure, video, oil and gas motion and environmental parameters. This paper presents specifications of motes for pipeline surveillance based on integrated systems architecture. The proposed architecture utilizes a Multi-Agent System (MAS) for the realization of an Integrated Oil Pipeline Monitoring and Incident Mitigation System (IOPMIMS) that can effectively monitor and provide actionable information for pipelines. The requirements and components of motes, different threats to pipelines and ways of detecting such threats presented in this paper will enable better deployment of pipeline surveillance systems for incident mitigation. It was identified that the shortcomings of the existing wireless sensor nodes as regards their application to pipeline surveillance are not effective for surveillance systems. The resulting specifications provide a framework for designing a cost-effective system, cognizant of the design considerations for wireless sensor motes used in pipeline surveillance

Trust and Suspicion as a Function of Cyber Security in Human Machine Team (HMT) of Unmanned Systems

The research focuses on cyber-attacks on cyber-physical systems of the unmanned vehicles that are characteristically used in the military, particularly the Air Force. Unmanned systems are exposed to various risks as the capacity of cyber attackers continue to expand, raising the need for speedy and immediate responses. The advances in military technologies form the basis of the research that explores the challenges faced in the timely detection and response to cyber-attacks. The purpose of the research is to study the connections between operator suspicion and the detection and response to cyber-attacks alongside the identification of theory of suspicion as the theoretical framework. The paper further presents the experiment used and the interview questions that offer the basis for the recommendations and importance of the research while answering the research questions. The conclusion from the literature review, interview, and experiment indicates the need for training among operators in the Air Force to reinforce their capacity in the detection and response to cyber-attacks and other adverse events that could compromise the execution of the mission established for unmanned systems. The research offers recommendations that can be implemented by the Royal Saudi Air Force (RSAF) in enhancing the security measures of unmanned systems