A generic framework for video understanding applied to group behavior recognition

This paper presents an approach to detect and track groups of people in video-surveillance applications, and to automatically recognize their behavior. This method keeps track of individuals moving together by maintaining a spacial and temporal group coherence. First, people are individually detected and tracked. Second, their trajectories are analyzed over a temporal window and clustered using the Mean-Shift algorithm. A coherence value describes how well a set of people can be described as a group. Furthermore, we propose a formal event description language. The group events recognition approach is successfully validated on 4 camera views from 3 datasets: an airport, a subway, a shopping center corridor and an entrance hall.Comment: (20/03/2012

Automatic object classification for surveillance videos.

PhDThe recent popularity of surveillance video systems, specially located in urban scenarios, demands the development of visual techniques for monitoring purposes. A primary step towards intelligent surveillance video systems consists on automatic object classification, which still remains an open research problem and the keystone for the development of more specific applications. Typically, object representation is based on the inherent visual features. However, psychological studies have demonstrated that human beings can routinely categorise objects according to their behaviour. The existing gap in the understanding between the features automatically extracted by a computer, such as appearance-based features, and the concepts unconsciously perceived by human beings but unattainable for machines, or the behaviour features, is most commonly known as semantic gap. Consequently, this thesis proposes to narrow the semantic gap and bring together machine and human understanding towards object classification. Thus, a Surveillance Media Management is proposed to automatically detect and classify objects by analysing the physical properties inherent in their appearance (machine understanding) and the behaviour patterns which require a higher level of understanding (human understanding). Finally, a probabilistic multimodal fusion algorithm bridges the gap performing an automatic classification considering both machine and human understanding. The performance of the proposed Surveillance Media Management framework has been thoroughly evaluated on outdoor surveillance datasets. The experiments conducted demonstrated that the combination of machine and human understanding substantially enhanced the object classification performance. Finally, the inclusion of human reasoning and understanding provides the essential information to bridge the semantic gap towards smart surveillance video systems

Ground truth annotation of traffic video data

This paper presents a software application to generate ground-truth data on video files from traffic surveillance cameras used for Intelligent Transportation Systems (IT systems). The computer vision system to be evaluated counts the number of vehicles that cross a line per time unit intensity-, the average speed and the occupancy. The main goal of the visual interface presented in this paper is to be easy to use without the requirement of any specific hardware. It is based on a standard laptop or desktop computer and a Jog shuttle wheel. The setup is efficient and comfortable because one hand of the annotating person is almost all the time on the space key of the keyboard while the other hand is on the jog shuttle wheel. The mean time required to annotate a video file ranges from 1 to 5 times its duration (per lane) depending on the content. Compared to general purpose annotation tool a time factor gain of about 7 times is achieved.This work was funded by the Spanish Government project MARTA under the CENIT program and CICYT contract TEC2009-09146.Mossi García, JM.; Albiol Colomer, AJ.; Albiol Colomer, A.; Oliver Moll, J. (2014). Ground truth annotation of traffic video data. Multimedia Tools and Applications. 1-14. https://doi.org/10.1007/s11042-013-1396-xS114Albiol A et al (2011) Detection of parked vehicles using spatiotemporal maps. IEEE Trans Intell Transport Syst 12(4):1277–1291Blunsden SJ, Fisher R (2010) The BEHAVE video dataset: ground truthed video for multi-person behavior classification. Annal British Mach Vis Assoc 4:1–12Bradski G, Kaehler A (2008) Learning OpenCV: Computer vision with the OpenCV library. O'Reilly Media, IncorporatedBrooke J. SUS: a “quick and dirty” usability scale. Usability evaluation in industry. Taylor and FrancisBrostow GJ et al (2009) Semantic object classes in video: a high-definition ground truth database. Pattern Recognit Lett 30(2):88–97Buch N et al (2011) A review of computer vision techniques for the analysis of urban traffic. IEEE Trans Intell Transp Syst 12(3):920–939D’Orazio T et al. (2009) A semi-automatic system for ground truth generation of soccer video sequences. Advanced Video and Signal Based Surveillance, 2009. AVSS’09. Sixth IEEE International Conference on (Sep. 2009), 559–564Dollar P et al (2012) Pedestrian detection: an evaluation of the state of the art. IEEE Trans Pattern Anal Mach Intell 34(4):743–761Faro A et al (2011) Adaptive background modeling integrated with luminosity sensors and occlusion processing for reliable vehicle detection. IEEE Trans Intell Transport Syst 12(4):1398–1412Giro-i-Nieto X et al (2010) GAT: a graphical annotation tool for semantic regions. Multimed Tool Appl 46(2–3):155–174i-LIDS. Image Library for Intelligent Detection Systems: www.ilids.co.uk . Home Office Scientific Development Branch, United Kingdom. Last Accessed February 2013Kasturi R et al (2009) Framework for performance evaluation of face, text, and vehicle detection and tracking in video: data, metrics, and protocol. IEEE Trans Pattern Anal Mach Intell 31(2):319–336Laganière R (2011) OpenCV 2 computer vision application programming cookbook. Packt Pub LimitedLorist MM et al (2000) Mental fatigue and task control: planning and preparation. Psychophysiology 37(5):614–625Russell B et al (2008) LabelMe: a database and web-based tool for image annotation. Int J Comput Vis 77(1):157–173Serrano M, Gracía J, Patricio M, Molina J (2010). Interactive video annotation tool. Distributed Computing and Artificial Intelligence, 325–332Traffic City Cameras. Ajuntament de València, Spain. http://camaras.valencia.es . Last Accessed February 2013TREC video retrieval evaluation. http://www-nlpir.nist.gov/projects/trecvid/Vezzani R, Cucchiara R (2010) Video Surveillance Online Repository (ViSOR): an integrated framework. Multimed Tool Appl 50(2):359–380ViPER: the video performance evaluation resource: http://viper-toolkit.sourceforge.net/Volkmer T et al. (2005) A web-based system for collaborative annotation of large image and video collections: an evaluation and user study. Proceedings of the 13th annual ACM international conference on Multimedia (New York, NY, USA, 2005), 892–901Zhang HB, Li SA, Chen SY, Su SZ, Duh DJ, Li SZ (2012) Adaptive photograph retrieval method. Multimedia Tools and Applications, Published online September 2012.Zou Y et al (2011) Traffic incident classification at intersections based on image sequences by HMM/SVM classifiers. Multimed Tool Appl 52(1):133–14

A review of the internet of floods : near real-time detection of a flood event and its impact

Worldwide, flood events frequently have a dramatic impact on urban societies. Time is key during a flood event in order to evacuate vulnerable people at risk, minimize the socio-economic, ecologic and cultural impact of the event and restore a society from this hazard as quickly as possible. Therefore, detecting a flood in near real-time and assessing the risks relating to these flood events on the fly is of great importance. Therefore, there is a need to search for the optimal way to collect data in order to detect floods in real time. Internet of Things (IoT) is the ideal method to bring together data of sensing equipment or identifying tools with networking and processing capabilities, allow them to communicate with one another and with other devices and services over the Internet to accomplish the detection of floods in near real-time. The main objective of this paper is to report on the current state of research on the IoT in the domain of flood detection. Current trends in IoT are identified, and academic literature is examined. The integration of IoT would greatly enhance disaster management and, therefore, will be of greater importance into the future

Developing a system for health and safety enhancement and automation in construction sites

The construction industry forms an important element within the economic activities and is known to be challenging and dangerous. Erroneously construction site accidents were accepted as unavoidable. The existing work health and safety protocols goals were not to cut risk but to provide risk assessment by understanding the types of risks associated with various activities and setting out rules and procedures to manage them and cut their impact. This study attempts a proactive approach to construction site health and safety by anticipating the hazards associated with a planned daily work activity and providing on site the relevant training and safety instructions. This was achieved by integrating the project’s digital design with site images processing and analysis. Digital image processing applies signal processing algorithms to images and videos resulting in extracting useful information from them. An essential and critical issue in the field of computer vision is the object’s recognition methods which should be capable of finding the partial occlusion of objects. Knowledge management systems archive and locate the required information and make it available to the relevant destination quickly and efficiently. It can also provide access to information in other construction sites and to the design team. This management system helps to save the gained experience and make it available to the project or other similar projects. The Building Information System was introduced as a system in which the objectives of this study can be incorporated leaving the door open to incorporate other project management activities. The possible solutions for the identified health and safety business problem were analysed in order to arrive at the best solution suitable to the objectives of the study. The end users ‘needs obtained from the distributed questionnaire and the project’s functional requirements were considered in order to create a model that will achieve their goals in an efficient manner. An activity diagram and a user case diagram based on the UML language were generated. Based on them a computerized model (CONSTRUCTION AUTOMATA) was developed to identify risks associated with specific work activities and provide the relevant safety instructions and training to mitigate them. The model automatically produces safety reports to record and serve as a knowledge management base for future reference thus eliminating possible human errors. The computer program was tested with available site images from an existing project and it proved to deliver its outputs according to its design. The developed model was then demonstrated to a selected group of relevant professionals and was seen to score well with ease of use mark of (6.17) and effectiveness as a health and safety tool mark of (6.37) out of a total mark of (10)

Novel statistical modeling methods for traffic video analysis

Video analysis is an active and rapidly expanding research area in computer vision and artificial intelligence due to its broad applications in modern society. Many methods have been proposed to analyze the videos, but many challenging factors remain untackled. In this dissertation, four statistical modeling methods are proposed to address some challenging traffic video analysis problems under adverse illumination and weather conditions. First, a new foreground detection method is presented to detect the foreground objects in videos. A novel Global Foreground Modeling (GFM) method, which estimates a global probability density function for the foreground and applies the Bayes decision rule for model selection, is proposed to model the foreground globally. A Local Background Modeling (LBM) method is applied by choosing the most significant Gaussian density in the Gaussian mixture model to model the background locally for each pixel. In addition, to mitigate the correlation effects of the Red, Green, and Blue (RGB) color space on the independence assumption among the color component images, some other color spaces are investigated for feature extraction. To further enhance the discriminatory power of the input feature vector, the horizontal and vertical Haar wavelet features and the temporal information are integrated into the color features to define a new 12-dimensional feature vector space. Finally, the Bayes classifier is applied for the classification of the foreground and the background pixels. Second, a novel moving cast shadow detection method is presented to detect and remove the cast shadows from the foreground. Specifically, a set of new chromatic criteria is presented to detect the candidate shadow pixels in the Hue, Saturation, and Value (HSV) color space. A new shadow region detection method is then proposed to cluster the candidate shadow pixels into shadow regions. A statistical shadow model, which uses a single Gaussian distribution to model the shadow class, is presented to classify shadow pixels. Additionally, an aggregated shadow detection strategy is presented to integrate the shadow detection results and remove the shadows from the foreground. Third, a novel statistical modeling method is presented to solve the automated road recognition problem for the Region of Interest (RoI) detection in traffic video analysis. A temporal feature guided statistical modeling method is proposed for road modeling. Additionally, a model pruning strategy is applied to estimate the road model. Then, a new road region detection method is presented to detect the road regions in the video. The method applies discriminant functions to classify each pixel in the estimated background image into a road class or a non-road class, respectively. The proposed method provides an intra-cognitive communication mode between the RoI selection and video analysis systems. Fourth, a novel anomalous driving detection method in videos, which can detect unsafe anomalous driving behaviors is introduced. A new Multiple Object Tracking (MOT) method is proposed to extract the velocities and trajectories of moving foreground objects in video. The new MOT method is a motion-based tracking method, which integrates the temporal and spatial features. Then, a novel Gaussian Local Velocity (GLV) modeling method is presented to model the normal moving behavior in traffic videos. The GLV model is built for every location in the video frame, and updated online. Finally, a discriminant function is proposed to detect anomalous driving behaviors. To assess the feasibility of the proposed statistical modeling methods, several popular public video datasets, as well as the real traffic videos from the New Jersey Department of Transportation (NJDOT) are applied. The experimental results show the effectiveness and feasibility of the proposed methods

Semiotic Annotation of Narrative Video Commercials: Bridging the Gap between Artifacts and Ontologies

Drawing on semiotic theories, the paper proposes a new concept of annotation \u2013 called semiotic annotation \u2013 whose goal is to describe the multilayered articulation of meaning inscribed within narrative video commercials by their designers. The approach exploits the use of a meta-model of the narrative video genre providing the conceptualizations and the vocabulary for analysis and annotation. By explicating design knowledge embodied in the video, semiotic annotation plays the role of intermediate level knowledge between the meta-model (an informal ontology) and practice (the concrete video artifact). In order to assess the feasibility of the approach, a test bed is presented and results are reported. A final discussion about the potential contribution of semiotic annotation in the fields of Research Through Design, Technological Mediation, and Interface Criticism concludes the study

INQUIRIES IN INTELLIGENT INFORMATION SYSTEMS: NEW TRAJECTORIES AND PARADIGMS

Rapid Digital transformation drives organizations to continually revitalize their business models so organizations can excel in such aggressive global competition. Intelligent Information Systems (IIS) have enabled organizations to achieve many strategic and market leverages. Despite the increasing intelligence competencies offered by IIS, they are still limited in many cognitive functions. Elevating the cognitive competencies offered by IIS would impact the organizational strategic positions. With the advent of Deep Learning (DL), IoT, and Edge Computing, IISs has witnessed a leap in their intelligence competencies. DL has been applied to many business areas and many industries such as real estate and manufacturing. Moreover, despite the complexity of DL models, many research dedicated efforts to apply DL to limited computational devices, such as IoTs. Applying deep learning for IoTs will turn everyday devices into intelligent interactive assistants. IISs suffer from many challenges that affect their service quality, process quality, and information quality. These challenges affected, in turn, user acceptance in terms of satisfaction, use, and trust. Moreover, Information Systems (IS) has conducted very little research on IIS development and the foreseeable contribution for the new paradigms to address IIS challenges. Therefore, this research aims to investigate how the employment of new AI paradigms would enhance the overall quality and consequently user acceptance of IIS. This research employs different AI paradigms to develop two different IIS. The first system uses deep learning, edge computing, and IoT to develop scene-aware ridesharing mentoring. The first developed system enhances the efficiency, privacy, and responsiveness of current ridesharing monitoring solutions. The second system aims to enhance the real estate searching process by formulating the search problem as a Multi-criteria decision. The system also allows users to filter properties based on their degree of damage, where a deep learning network allocates damages in 12 each real estate image. The system enhances real-estate website service quality by enhancing flexibility, relevancy, and efficiency. The research contributes to the Information Systems research by developing two Design Science artifacts. Both artifacts are adding to the IS knowledge base in terms of integrating different components, measurements, and techniques coherently and logically to effectively address important issues in IIS. The research also adds to the IS environment by addressing important business requirements that current methodologies and paradigms are not fulfilled. The research also highlights that most IIS overlook important design guidelines due to the lack of relevant evaluation metrics for different business problems