PETS 2017: dataset and challenge

This paper indicates the dataset and challenges evaluated under PETS2017. In this edition PETS continues the evaluation theme of on-board surveillance systems for protection of mobile critical assets as set in PETS 2016. The datasets include (1) the ARENA Dataset; an RGB camera dataset, as used for PETS2014 to PETS 2016, which addresses protection of trucks; and (2) the IPATCH Dataset; a multi sensor dataset, as used in PETS2016, addressing the application of multi sensor surveillance to protect a vessel at sea from piracy. The datasets allow for performance evaluation of tracking in low-density scenarios and detection of various surveillance events ranging from innocuous abnormalities to dangerous and criminal situations. Training data for tracking algorithms is released with the dataset; tracking data is also available for authors addressing only surveillance event detection challenges but not working on tracking

Evaluating deep semantic segmentation networks for object detection in maritime surveillance

Maritime surveillance is important for applications in safety and security, but the visual detection of objects in maritime scenes remains challenging due to the diverse and unconstrained nature of such environments, and the need to operate in near real-time. Recent work on deep neural networks for semantic segmentation has achieved good performance in the road/urban scene parsing task. Driven by the potential application in autonomous vehicle navigation, many of the architectures are designed to be fast and lightweight. In this paper, we evaluate semantic segmentation networks in the context of an object detection system for maritime surveillance. Using data from the ADE20k scene parsing dataset, we train a selection of recent semantic segmentation network architectures to compare their performance on a number of publicly available maritime surveillance datasets

NovelCraft: A Dataset for Novelty Detection and Discovery in Open Worlds

In order for artificial agents to successfully perform tasks in changing environments, they must be able to both detect and adapt to novelty. However, visual novelty detection research often only evaluates on repurposed datasets such as CIFAR-10 originally intended for object classification, where images focus on one distinct, well-centered object. New benchmarks are needed to represent the challenges of navigating the complex scenes of an open world. Our new NovelCraft dataset contains multimodal episodic data of the images and symbolic world-states seen by an agent completing a pogo stick assembly task within a modified Minecraft environment. In some episodes, we insert novel objects of varying size within the complex 3D scene that may impact gameplay. Our visual novelty detection benchmark finds that methods that rank best on popular area-under-the-curve metrics may be outperformed by simpler alternatives when controlling false positives matters most. Further multimodal novelty detection experiments suggest that methods that fuse both visual and symbolic information can improve time until detection as well as overall discrimination. Finally, our evaluation of recent generalized category discovery methods suggests that adapting to new imbalanced categories in complex scenes remains an exciting open problem.Comment: Published in Transactions on Machine Learning Research (03/2023

Seleção de faces para reconhecimento facial em videovigilância

Orientador: Prof. Dr. Luciano SilvaDissertação (mestrado) - Universidade Federal do Paraná, Setor de Ciências Exatas, Programa de Pós-Graduação em Informática. Defesa : Curitiba, 17/08/2022Inclui referências: p. 61-66Área de concentração: Ciência da ComputaçãoResumo: A proposta do trabalho é o desenvolvimento de um módulo, dedicado à seleção da imagem da face mais representativa da identidade de pessoas, para fins de reconhecimento facial em sistemas de vigilância por vídeo de ambientes sem restrições. Em um sistema ideal de vigilância por vídeo com reconhecimento facial, uma das etapas fundamentais é a Seleção de Faces. A Seleção de Faces combina detecção, rastreamento e aferimento de qualidade de faces para encontrar, agrupar e filtrar, respectivamente, as faces nas sequências de vídeo de acordo com métricas de qualidade, como por exemplo: orientação do rosto e nitidez da imagem. Objetiva-se que o módulo proposto seja robusto aos desafios presentes nas sequências de vídeo obtidas por sistemas de vigilância, como: múltiplas faces, baixa resolução de captura, iluminação irregular e oclusões. Para a concretização da proposta, trabalhos similares foram estudados, criou-se um dataset complementar de vídeos com múltiplas faces anotadas em ambientes não controlados, e o sistema proposto por Barquero et al. (2021) foi designado como baseline. Diante de experimentação com diferentes modelos para detecção de faces, utilização da resolução como medida de qualidade, substituição da medida de nitidez e exploração aleatória de parâmetros, obteve-se um aumento de 10.1% na métrica de precisão multi objetos (MOTP) e 9% a mais na métrica de identificação IDF1.Abstract: This work’s proposal is the development of a module, dedicated to the selection of the most representative face image of people’s identities, for the purposes of facial recognition in video surveillance systems in unrestricted environments. In an ideal video surveillance system with facial recognition, one of the fundamental steps is Face Selection. Face Selection combines face detection, tracking and quality assessment to respectively find, group and filter faces in video sequences according to quality metrics such as face orientation and image sharpness. The goal is for the proposed module to be robust to the challenges present in video sequences obtained by surveillance systems, such as: multiple faces, low resolution, irregular lighting and occlusions. To carry out the proposal, similar works were studied, an additional video dataset with multiple annotated faces in uncontrolled environments was created, and the pipeline proposed by (Barquero et al., 2021) was chosen as a baseline. By means of experimentation with different face detection models, face resolution as an added quality measure, replacement of the baseline’s sharpness measurement approach and random parameter search, it was achieved an increase of 10.1% in Multiple Object Tracking Precision (MOTP) metric and 9% more in the IDF1 identification metric

Siamese networks for surveillance and security

This thesis investigates the usage of Siamese networks across three surveillance and security tasks for land border security. Siamese networks (also known as a twin-pair network) are a layout of neural networks that contain a segment that contains duplicated architecture and configuration parameters for feature extraction of two inputs, combining the outputs into one vector for comparison in a final set of layers to produce a similarity score. The effectiveness of multiple architectures of Siamese networks crafted from multiple generations of Convolutional Neural Networks and Residual Neural Networks are examined for side-profile vehicle classification and Differential Morphing Attack Detection (D-MAD), and with a novel architecture for trajectory similarity analysis. The challenging domain of automated vehicle classification from pole-mounted roadway cameras from side-profile views is evaluated. Three Siamese networks based on existing non-Siamese architectures are proposed and compared against five existing methods on a novel and published dataset. The evaluation undertaken shows that the residual based Siamese network is able to outperform other state of the art methods on datasets with a small number of classes. An end-to-end Siamese trajectory network framework is proposed for the purpose of trajectory similarity analysis in surveillance tasks. A deep feature auto-encoding network is used as part of a discriminative Siamese architecture to perform trajectory similarity analysis. The effectiveness of this method is evaluated on four challenging public real-world datasets containing both vehicle and pedestrian targets, and compared with five existing methods. The proposed method outperforms the existing methods on three of the four datasets. Face morphing attacks pose an increasingly severe threat to automatic face recognition systems in border control environments. Three Siamese architectures built up from multiple generations of non-Siamese Convolutional and Residual Neural Networks for D-MAD are proposed, showing the effectiveness of these networks against a pre-established Convolutional architecture for Single-image Morphing Attack Detection (S-MAD). The residual network based architecture outperforms representative convolutional architectures from the literature, with the Siamese D-MAD architecture able to outperform its S-MAD variant

Emerging Technologies in Fisheries Science: A Transdisciplinary Report

The Pacific Coast Groundfish Fishery harvests a diverse and large grouping of fishes, but it did not become heavily fished until around WWII. This makes the groundfish fishery a comparatively young fishery. Despite its youth, it is one of the largest and most lucrative fisheries in Oregon—with a current harvest value of approximately $48 million per year, which is exceeded only by the Dungeness crab fishery. Northeastern Pacific Coast Groundfish species are also important for recreational and tribal purposes, although it is difficult to compare these to the commercial industry. With over 90 different species to consider, this commercial fishery is complex, and there are many different stakeholder groups involved, each with their own goals, values, and perspectives. Fishing regulations greatly impact local stakeholders, some of whom rely on the fishery for their livelihoods. These local stakeholders are dependent on accurate stock assessment surveys and models so that the fishing regulations are appropriate. Some stakeholders feel that regulations tend to be overly cautious to compensate for the large amount of uncertainty involved with managing a fishery and estimating a fish population. To reduce this uncertainty and the need to err so heavily on the side of caution, stock assessment surveys could include innovative technologies and novel datasets. For example, these stock assessments do not currently use automated video surveillance on their bottom trawl surveys, an emerging form of machine learning. As understood by the NSF-funded National Research Traineeship (NRT) training, there are three interwoven core concepts: 1) Big Data (BD), 2) Coupled Natural-Human (CNH) systems, and 3) Risk and Uncertainty (R&U) analysis and communication. Big Data refers to any high volume of data with high throughput. Coupled Natural-Human systems are the biological and human worlds, as well as their overlap and interaction. Risk is the potential and likelihood of an unfavorable event, and uncertainty refers to the unknowns of a likelihood, process, or analysis. This project chose to investigate these three concepts within the framework of emerging technologies and fisheries science. Emerging technologies are those dealing with BD, since this is a relatively new area of study, and this project specifically focused on computer vision within machine learning. This technology was applied to the realm of fisheries science and ultimately management, which is the study of a coupled natural-human system. Changing oceans conditions mean that Northeastern Pacific groundfish are at risk and their future is uncertain. Therefore, this project set out to determine how the influence of big data, machine learning, ecological inference, and environmental decision making overlap. The story of the life and study of these fishes in a newly Americanized sea is ready for a closer examination. It is for these reasons combined that Pacific coast groundfish fishery science provides a robust platform in which to explore the autonomous capacity of technology and data production at the intersection of environmental science and decision making. More specifically, to what extent are large, ecological datasets informing the production and application of emerging technologies in fisheries science, and how are these new technologies and sampling methods being integrated into fisheries management frameworks? A case study in which to explore this concept can be found in the testimony of a flatfish, or rather, the complex, ecologically and economically important assemblages of numerous groundfish species in the northeastern Pacific Ocean where flatfish are found