Detecção de eventos complexos em vídeos baseada em ritmos visuais

Orientador: Hélio PedriniDissertação (mestrado) - Universidade Estadual de Campinas, Instituto de ComputaçãoResumo: O reconhecimento de eventos complexos em vídeos possui várias aplicações práticas relevantes, alavancadas pela grande disponibilidade de câmeras digitais instaladas em aeroportos, estações de ônibus e trens, centros de compras, estádios, hospitais, escolas, prédios, estradas, entre vários outros locais. Avanços na tecnologia digital têm aumentado as capacidades dos sistemas em reconhecer eventos em vídeos por meio do desenvolvimento de dispositivos com alta resolução, dimensões físicas pequenas e altas taxas de amostragem. Muitos trabalhos disponíveis na literatura têm explorado o tema a partir de diferentes pontos de vista. Este trabalho apresenta e avalia uma metodologia para extrair características dos ritmos visuais no contexto de detecção de eventos em vídeos. Um ritmo visual pode ser visto com a projeção de um vídeo em uma imagem, tal que a tarefa de análise de vídeos é reduzida a um problema de análise de imagens, beneficiando-se de seu baixo custo de processamento em termos de tempo e complexidade. Para demonstrar o potencial do ritmo visual na análise de vídeos complexos, três problemas da área de visão computacional são selecionados: detecção de eventos anômalos, classificação de ações humanas e reconhecimento de gestos. No primeiro problema, um modelo e? aprendido com situações de normalidade a partir dos rastros deixados pelas pessoas ao andar, enquanto padro?es representativos das ações são extraídos nos outros dois problemas. Nossa hipo?tese e? de que vídeos similares produzem padro?es semelhantes, tal que o problema de classificação de ações pode ser reduzido a uma tarefa de classificação de imagens. Experimentos realizados em bases públicas de dados demonstram que o método proposto produz resultados promissores com baixo custo de processamento, tornando-o possível aplicar em tempo real. Embora os padro?es dos ritmos visuais sejam extrai?dos como histograma de gradientes, algumas tentativas para adicionar características do fluxo o?tico são discutidas, além de estratégias para obter ritmos visuais alternativosAbstract: The recognition of complex events in videos has currently several important applications, particularly due to the wide availability of digital cameras in environments such as airports, train and bus stations, shopping centers, stadiums, hospitals, schools, buildings, roads, among others. Moreover, advances in digital technology have enhanced the capabilities for detection of video events through the development of devices with high resolution, small physical size, and high sampling rates. Many works available in the literature have explored the subject from different perspectives. This work presents and evaluates a methodology for extracting a feature descriptor from visual rhythms of video sequences in order to address the video event detection problem. A visual rhythm can be seen as the projection of a video onto an image, such that the video analysis task can be reduced into an image analysis problem, benefiting from its low processing cost in terms of time and complexity. To demonstrate the potential of the visual rhythm in the analysis of complex videos, three computer vision problems are selected in this work: abnormal event detection, human action classification, and gesture recognition. The former problem learns a normalcy model from the traces that people leave when they walk, whereas the other two problems extract representative patterns from actions. Our hypothesis is that similar videos produce similar patterns, therefore, the action classification problem is reduced into an image classification task. Experiments conducted on well-known public datasets demonstrate that the method produces promising results at high processing rates, making it possible to work in real time. Even though the visual rhythm features are mainly extracted as histogram of gradients, some attempts for adding optical flow features are discussed, as well as strategies for obtaining alternative visual rhythmsMestradoCiência da ComputaçãoMestre em Ciência da Computação1570507, 1406910, 1374943CAPE

Acoustic complexity indices reveal the acoustic communities of the old-growth Mediterranean forest of Sasso Fratino Integral Natural Reserve (Central Italy)

The Sasso Fratino Integral Natural Reserve (Central Italy), a rare example of climax Mediterranean forest, provides an extraordinary opportunity to create an important soundscape reference of old-growth forest. In this study, we describe the soundscape of three localities (Lama, Sasso 950, Sasso 1400) representative of a gradient of variety and complexity of habitats, recorded during the period 10 May to 9 June 2017. Our results reveal temporal partitioning into acoustically homogeneous periods across 24 h suggesting that soniferous species (mainly birds) adopt ecological routines in which their acoustic activity is organized according to specific transient physiological needs. We processed multi-temporal aggregates of 1, 5, 10, and 15 s recordings and calculated the Acoustic Signature (AS) with four new indices: Ecoacoustic Events (EE), Acoustic Signature Dissimilarity (ASD), and their fractal dimensions (DEE and DASD), derived from the Acoustic Complexity Index (ACI). The use of the EE and ASD greatly improved the AS interpretation, adding further details such as the emergence of a clear sequence of patterns consistent with the daily evolution of the overall soundscape. DEE and DASD confirm the patterns observed using the AS, but provide more clarity and detail about the great acoustic complexity that exists across temporal scales in this old-growth forest. The temporal turnover of different acoustic communities occurs as a result of a gradual shift of different homogenous acoustic properties. We conclude that soniferous species use distinct, species-specific temporal resolutions according to their physiological and ecological needs and that the fractal approach used here provides a novel tool to overcome the difficulties associated with describing multi-temporal acoustic patterns

Seasonal feeding and movement responses of resident sculpin in the Canadian Arctic

Environments change across space and time, requiring organisms to adapt behaviorally and/or physiologically. In the Arctic, where productivity varies greatly among seasons, many marine species exploit a resource pulse associated with annual ice melt. Melting sea-ice releases zooplankton on and within the ice into open water causing many consumers to migrate with the receding ice; however, resident species may too rely on the ephemeral prey source. Fourhorn Sculpin (Myoxocephalus quadricornis) and Slimy Sculpin (Cottus cognatus) are residents of Tremblay Sound, Nunavut Canada which is known as a seasonally productive system in the Arctic. I characterized the seasonal behavior of sculpin by collecting and identifying stomach contents from 175 sculpin during the ice-free periods of 2017-2019. Other individuals were released with acoustic transmitters (n=119) for year-round monitoring of movements throughout Tremblay Sound. Consumption rates were high, with 33% of individuals consuming \u3e5% of their body mass (mean 4.32%; SE 0.25). Diet composition was dominated by amphipods and krill but both traditional and multivariate analyses showed diets shifted and broadened over the short (~6 weeks) ice-free season and between years. Namely, indicator species analysis revealed significant associations with distinct amphipod genera between years (Gammaracanthus sp. in 2017, Onisimus sp. in 2018). Feeding exploitation corresponded with high rates of activity as several metrics of movement and home range consistently revealed more activity during the ice-free period. Further analyses revealed qualitatively that sculpin showed high site fidelity and were residents within Tremblay Sound year-round, though acceleration and pressure sensors indicate individuals may not be entirely dormant under the ice. Together, showing high levels of activity to feed at high levels while maintaining flexibility in diet composition indicates that sculpin are adapted to exploit ephemeral resources. These physiological and behavioral adaptations are likely important to annual energy budgets and have potential to affect the resilience of sculpin to ongoing climate change in marine Arctic ecosystems

Componentes e pontos de quebra em séries temporais na análise de imagens de sensoriamento remoto

Orientador: Ricardo da Silva TorresDissertação (mestrado) - Universidade Estadual de Campinas, Instituto de ComputaçãoResumo: A detecção e caracterização de mudanças temporais são indicadores cruciais no processo de compreensão da maneira como mecanismos complexos funcionam e evoluem. Técnicas e imagens de sensoriamento remoto têm sido amplamente empregadas nas últimas décadas com objetivo de detectar e investigar mudanças temporais na superfície terrestre. Tal detecção em dados de séries temporais é passível de ser refinada ainda mais isolando-se as componentes aditivas de tendência e sazonalidade do ruído subjacente. Este trabalho investiga, em particular, o método Breaks For Additive Season and Trend (BFAST) para a análise, decomposição e detecção de pontos de quebra em séries temporais associadas a dados de sensoriamento remoto. Os outputs do método são, então, utilizados em três distintas ¿ mas altamente interconectadas ¿ linhas de pesquisa: em uma melhor compreensão de fenômenos climáticos; na correlação com dados de distúrbios antropológicos; e em problemas de classificação usando funções de dissimilaridade descobertas por um framework evolucionário baseado em Programação Genética (GP). Experimentos realizados demonstram que a decomposição e pontos de quebra produziram resultados efetivos quando aplicados aos estudos com dados ecológicos, mas não foram capazes de melhorar os resultados de classificação quando comparados ao uso das séries brutas. As realizações nesses três contextos também culminaram na criação de duas ferramentas de análise de séries temporais com código aberto baseadas na web, sendo que uma delas foi tão bem aceita pela comunidade-alvo, que atualmente encontra-se integrada em uma plataforma privada de computação em nuvemAbstract: Detecting and characterizing temporal changes are crucial indicators in the process of understanding how complex mechanisms work and evolve. The use of remote sensing images and techniques has been broadly employed over the past decades in order to detect and investigate temporal changes on the Earth surface. Such change detection in time series data may be even further refined by isolating the additive long-term (trend) and cyclical (seasonal) components from the underlying noise. This work investigates the particular Breaks For Additive Season and Trend (BFAST) method for the analysis, decomposition, and breakpoint detection of time series associated with remote sensing data. The derived outputs from that method are, then, used in three distinct ¿ but highly interconnected ¿ research venues: in a better comprehension of climatic phenomena; in the correlation to human-induced disturbances data; and in data classification problems using time series dissimilarity functions discovered by a Genetic-Programming-(GP)-based evolutionary framework. Performed experiments show that decomposition and breakpoints produced insightful and effective results when applied to the ecological data studies, but could not further improve the classification results when compared to its raw time series counterpart. The achievements in those three contexts also led to the creation of two open-source web-based time series analysis tools. One of those tools was so well received by the target community, that it is currently integrated into a private cloud computing platformMestradoCiência da ComputaçãoMestre em Ciência da Computação132847/2015-92015/02105-0CNPQFAPES

A Fortran Kernel Generation Framework for Scientific Legacy Code

Quality assurance procedure is very important for software development. The complexity of modules and structure in software impedes the testing procedure and further development. For complex and poorly designed scientific software, module developers and software testers need to put a lot of extra efforts to monitor not related modules\u27 impacts and to test the whole system\u27s constraints. In addition, widely used benchmarks cannot help programmers with accurate and program specific system performance evaluation. In this situation, the generated kernels could provide considerable insight into better performance tuning. Therefore, in order to greatly improve the productivity of various scientific software engineering tasks such as performance tuning, debugging, and verification of simulation results, we developed an automatic compute kernel extraction prototype platform for complex legacy scientific code. In addition, considering that scientific research and experiment require long-term simulation procedure and the huge size of data transfer, we apply message passing based parallelization and I/O behavior optimization to highly improve the performance of the kernel extractor framework and then use profiling tools to give guidance for parallel distribution. Abnormal event detection is another important aspect for scientific research; dealing with huge observational datasets combined with simulation results it becomes not only essential but also extremely difficult. In this dissertation, for the sake of detecting high frequency event and low frequency events, we reconfigured this framework equipped with in-situ data transfer infrastructure. Through the method of combining signal processing data preprocess(decimation) with machine learning detection model to train the stream data, our framework can significantly decrease the amount of transferred data demand for concurrent data analysis (between distributed computing CPU/GPU nodes). Finally, the dissertation presents the implementation of the framework and a case study of the ACME Land Model (ALM) for demonstration. It turns out that the generated compute kernel with lower cost can be used in performance tuning experiments and quality assurance, which include debugging legacy code, verification of simulation results through single point and multiple points of variables tracking, collaborating with compiler vendors, and generating custom benchmark tests

Ocean surface provinces off Southwest Iberia based on satellite remote sensing

This thesis aimed to partition the complex surface marine domain off Southwest Iberia Peninsula (SWIP), using satellite remote sensing, and use it to assess phytoplankton variability patterns and underlying environmental drivers (1997 – 2015). Three unsupervised partition strategies, based on distinct input databases and temporal representations, detected a variable number of partition units (regions, provinces) of singular environmental and phytoplankton patterns within SWIP. An abiotic-based partition delineated 12 dynamic Environmental Provinces (EPs) that alternated coverage dominance along the annual cycle. EP patterns were in general related to phytoplankton biomass, indicated by satellite chlorophyll-a concentration (Chl-a), and productivity, thus supporting the biological relevance of this abiotic-based partition. A static partition, based on the main variability modes of Chl-a, derived 9 Chl-a regions. Moreover, a static partition strategy synthesised phytoplankton phenological patterns over SWIP into 5 phenoregions, with coherent patterns of timing, magnitude and duration of blooms. The spatial distribution of EPs, Chl-a regions and phenoregions shared similarities, which can be considered the main spatial patterns of SWIP ocean surface. In general, the spatial arrangement of the partition units showed a separation between coastal and open ocean, a latitudinal division (ca. 36.5oN) over the open ocean and, over the coast and slope, the influence of coastal upwelling along the west Portuguese coast and Cape São Vicente, and of river discharge along the northeastern Gulf of Cadiz. The environmental drivers of phytoplankton varied across partition units. Water column stratification, riverine discharge and upwelling intensity were the most influential modulators, and large scale climate indices usually showed minor effects. Environmental variables, Chl-a and phenology showed significant seasonal variability patterns, varying across regions. Interannual patterns were more complex, and significant trends were mostly detected within the Gulf of Cadiz. Linkages between environmental variability and phytoplankton support their use as an indicator of ecosystem status and change.O oceano superficial é um domínio extremamente complexo e dinâmico, onde as interações com a atmosfera e o continente modulam a distribuição e atividade dos organismos marinhos e o clima da Terra. O fitoplâncton, principal produtor primário marinho, é fortemente influenciado pelos processos atuantes no oceano superficial, constituindo um importante indicador do estado e variabilidade dos ecossistemas marinhos. Assim, a organização espacial horizontal do oceano superficial, função da variabilidade das propriedades abióticas e comunidades biológicas (incluindo o fitoplâncton), apresenta uma série de unidades funcionais distintas (regiões ou províncias), com atributos e padrões de variabilidade específicos. A partição ou regionalização do oceano, com identificação e delimitação destas unidades funcionais, simplifica a complexidade do oceano superficial e representa uma ferramenta para avaliar e compreender o funcionamento do oceano superficial, apresentando diversas aplicações ao nível do estudo, gestão e conservação dos ecossistemas marinhos. A deteção remota por satélite constitui uma fonte valiosa de dados para a partição do oceano superficial, pois disponibiliza campos sinóticos de várias variáveis oceanográficas e atmosféricas, em escalas espacial e temporal pertinentes, abrangendo períodos de várias décadas. A presente tese pretende particionar o complexo domínio marinho superficial do sudoeste da Península Ibérica (Southwest Iberia Peninsula, SWIP), com base em deteção remota por satélite, e avaliar a variabilidade do fitoplâncton e forçadores ambientais associados em regiões específicas (unidades funcionais) da área de estudo. Para atingir os objectivos principais foi inicialmente efetuada uma revisão do conhecimento científico sobre as estratégias de partição do oceano superficial baseadas em deteção remota por satélite (Capítulo 2) e, posteriormente, foram aplicadas diversas estratégias de partição nãosupervisionadas à área de estudo (Capítulos 3 - 5). Tais estratégias permitiram particionar a área de estudo com base em diferentes caraterísticas do oceano superficial (propriedades abióticas, variação da concentração de clorofila-a e índices fenológicos do fitoplâncton) e diferentes abordagens metodológicas (métodos de partição e resolução temporal). As diferentes partições do SWIP foram utilizadas para avaliar os padrões de variabilidade da biomassa e fenologia do fitoplâncton e suas relações com diferentes forçantes ambientais. No contexto deste estudo, as variáveis ambientais avaliadas incluíram variáveis locais indicadoras do ambiente físico, químico e ótico, variáveis hidrológicas indicadoras de processos costeiros (descarga dos rios e intensidade do afloramento costeiro) e indicadores climáticos de larga escala.This thesis was supported by Science without Borders Programme from the Brazilian National Council for Scientific and Technological Development (237998/2012‐2

A light in the dark: using cabled seafloor observatories to study abundance and behaviour of seafloor megafauna in response to environmental change

Surface primary productivity forms the base of most marine food webs and contributes significantly to global carbon cycling, providing a key link from surface to seafloor. High seasonal primary productivity along temperate latitude coastlines provides crucial nutrients for seafloor communities, driving spatiotemporal patterns in abundance, behaviour, biodiversity, and distribution of benthic megafauna. Many factors, including ocean warming, deoxygenation, and increasing frequency and duration of marine heatwaves (MHW), may alter the dynamics governing primary production, threatening benthic organisms that depend on the seasonal input of phytodetritus for food. The focus of this thesis is to make use of two seafloor observatories, NEPTUNE near Vancouver Island, British Columbia, and a new platform in Conception Bay, NL, to examine variability in abundance, behaviour, and composition of seafloor communities in response to environmental change across temporal scales. First, the response of the deep-sea pink urchin Strongylocentrotus fragilis to a recent MHW on the continental margin off the coast of Vancouver Island was investigated using a combination of benthic trawls (2004-2018) and seafloor observatory data (2013-2020). Sea urchin density declined during the MHW, likely in response to reduced kelp subsidies from coastal waters. Next, the new Holyrood Underwater Observatory in Conception Bay was used to study benthic community response to the spring phytoplankton bloom. High-frequency variability in seafloor environmental dynamics was documented during the winter-spring transition, and the unexpected emergence of >200 sea cucumbers (Psolus sp.) coinciding with the arrival of phytodetritus at the seafloor was observed. These data will provide a baseline against which to monitor changes in phenology as climate change progresses. This work comes at a critical point in ocean observing as we approach global climate tipping points. Now more than ever, it is essential to document the current state of marine communities to understand and predict community responses to changing ocean conditions, and to sustainably manage ocean resources

N-Eared listening: transdisciplinary ecoacoustics at human-environment interfaces

Understanding, managing and improving human-environment interactions is amongst the most pressing chal- lenges of our time. However, this is a wicked problem that we need to approach from multiple perspectives. I will outline a series of case-studies in inter- and transdisciplinary soundscape research: from validating ecoacoustic metrics in temperate UK woodlands, through wilderness mapping in the Swedish Arctic and community-led reef restoration in Indonesia to indigenous-led cultural heritage projects in the Ecuadorian Amazon. In doing so I aim to illustrate the value of integrating different forms of knowledge across disciplines, everyday and indigenous practices and speculate that triangulating ecological theory and computational methods with human experience might point to valuable new approaches to ecoaoustic analysis and even inspire fresh consideration of core con- servation imperatives, in order to better align the anthroposphere and biosphere for the benefit of all

The Spatial and Temporal Ecology of Seed Dispersal by Gorillas in Lopé National Park, Gabon: Linking Patterns of Disperser Behavior and Recruitment in an Afrotropical Forest

Western lowland gorillas: Gorilla g. gorilla) consume large quantities of fruit and disperse a great number of seeds. The majority these seeds are dispersed intact and viable in the dung. Dung is often deposited around the rim of a night nest or at a nest-site. Gorillas often construct nests in areas that have a sparse canopy, flattening the ground vegetation. These locations can be beneficial to the growth and survival of the seed species they disperse. Thus, not only are gorillas effective in terms of depositing seeds great distances from parent plants, away from the highest seed rain densities, they are also effective in directing seeds to potentially beneficial microsites. The objective of this research was to develop an understanding of the spatial and temporal patterns in fruit availability, seed deposition, and adult plants, and to test whether these patterns relate to the ecology of seed dispersal by gorillas. Results suggest that gorilla foraging and nesting behavior in particular, impose both spatial and temporal limitations to the distribution of dispersed seeds. In addition, temporal variation in the gorilla diet and factors that affect defecation rates and locations promote variation in the combinations: composition and abundance) of the seed species dispersed to different microsites. The clustered distribution of nest-sites leads to clumped and spatially restricted seed deposition patterns. Recruitment in gorilla-dispersed seed species corresponds with the aggregated: clumped) distribution of nest-sites. Gorillas have a long-lasting effect on the spatial structure and floristic composition of the forests they inhabit, particularly in large-seeded species