From media crossing to media mining

This paper reviews how the concept of Media Crossing has contributed to the advancement of the application domain of information access and explores directions for a future research agenda. These will include themes that could help to broaden the scope and to incorporate the concept of medium-crossing in a more general approach that not only uses combinations of medium-specific processing, but that also exploits more abstract medium-independent representations, partly based on the foundational work on statistical language models for information retrieval. Three examples of successful applications of media crossing will be presented, with a focus on the aspects that could be considered a first step towards a generalized form of media mining

Assessment of Driver\u27s Attention to Traffic Signs through Analysis of Gaze and Driving Sequences

A driver’s behavior is one of the most significant factors in Advance Driver Assistance Systems. One area that has received little study is just how observant drivers are in seeing and recognizing traffic signs. In this contribution, we present a system considering the location where a driver is looking (points of gaze) as a factor to determine that whether the driver has seen a sign. Our system detects and classifies traffic signs inside the driver’s attentional visual field to identify whether the driver has seen the traffic signs or not. Based on the results obtained from this stage which provides quantitative information, our system is able to determine how observant of traffic signs that drivers are. We take advantage of the combination of Maximally Stable Extremal Regions algorithm and Color information in addition to a binary linear Support Vector Machine classifier and Histogram of Oriented Gradients as features detector for detection. In classification stage, we use a multi class Support Vector Machine for classifier also Histogram of Oriented Gradients for features. In addition to the detection and recognition of traffic signs, our system is capable of determining if the sign is inside the attentional visual field of the drivers. It means the driver has kept his gaze on traffic signs and sees the sign, while if the sign is not inside this area, the driver did not look at the sign and sign has been missed

Reconhecimento automático de sinalização vertical de trânsito a partir de dados vídeo de um sistema de mapeamento móvel

Tese de mestrado em Engenharia Geográfica, apresentada à Universidade de Lisboa, através da Faculdade de Ciências, 2013Este trabalho consiste no estudo e desenvolvimento de um método de Reconhecimento Automático de Sinalização Vertical de Trânsito (RASVT), nomeadamente da sinalização de perigo e alguma da sinalização de regulamentação (cedência de passagem, proibição e obrigação), toda de cor vermelha ou azul. O ponto de partida para o seu desenvolvimento são os dados vídeo de um Sistema de Mapeamento Móvel, obtidos em condições reais (não controladas). A metodologia desenvolvida de reconhecimento de sinalização vertical de trânsito funciona em modo pós-aquisição, ou seja, após aquisição e processamento dos dados vídeo em bruto (raw) e dos dados de posicionamento (GPS e IMU). O método desenvolvido é constituído por três fases. A fase de detecção, que consiste em detectar Regiões de Interesse da imagem (RdI) que possam conter sinalização. Esta detecção é feita utilizando um processo de segmentação por cor (vermelha e azul) e utilizando cinco critérios de selecção: dimensões da RdI, rácio altura-comprimento da RdI, proximidade da RdI aos limites da imagem, posicionamento relativo entre o centróide da RdI e centróide da área segmentada e rácio de preenchimento da RdI. A fase de classificação, que tem por base as RdI obtidas na fase anterior e consiste no reconhecimento da forma geométrica de cada região, bem como na agregação, por classes, de cada uma dessas RdI analisadas. As classes utilizadas baseiam-se na cor e forma geométrica. A fase de reconhecimento, consiste na identificação da sinalização que tenha sido detectada e classificada e baseia-se na correspondência dos pictogramas presentes nos sinais. Através de um processo de segmentação por cor é extraído o pictograma para de seguida ser feito o seu reconhecimento, através da correspondência entre o pictograma extraído e pictogramas template. Esta correspondência é realizada utilizando a correlação simples. A fase de detecção apresentou uma taxa de sucesso de 32 % (que excluindo os resultados falsos positivos, apresenta uma taxa de sucesso de 89 %), a fase de classificação teve uma taxa de sucesso de 93 % e a fase de reconhecimento teve uma taxa de sucesso de 91 %. A taxa de sucesso global obtido pelo método RASVT implementado é de 81 %, ou seja, da sinalização presente na amostra analisada, 81 % foi correctamente detectada, classificada e reconhecida.This work aims to develop a method for Automatic Traffic Sign Recognition, namely, danger signs and some regulation signs (giving way, prohibition and obligation signs, all of them red or blue). The starting point for developing the method was a Mobile Mapping System’s video data, obtained in real conditions (uncontrolled conditions). The Automatic Traffic Sign Recognition method performs data post-processing (not real time processing) and comprises three stages. The detection stage consists in detecting the image Regions Of Interest (ROI) that may contain signs. This detection is performed using a color segmentation process (red and blue) and using five selection criteria (ROI dimensions, length-height ratio of the ROI, ROI distance to the limits of the image, the relative positioning between ROI’s centroid and a targeted area’s centroid and finally the filling ratio of ROI). The step of classification is based on the ROI obtained in the previous stage and consists in recognizing each region’s geometric shape, as well as the aggregation per classes of each one of these ROI. The classes used are based on color and geometric shapes. The recognition stage consists in the identification of the traffic signs that has been detected and classified, and is based on the correspondence of the pictograms present in the signals. Through a process of color segmentation the pictogram is extracted from the ROI and then its recognition is done by matching with template pictograms. This matching is performed using simple correlation. The detection stage showed a success rate of 32 % (if we exclude false positive results, the success rate increases to 89 %), the classification stage had a success rate of 93 % and the final recognition stage had a success rate of 91 %. The overall success rate obtained by the implemented method is 81 %, i.e., from the totality of traffic signs present in the sample, 81 % were correctly detected, classified and recognized

Detection of dynamic form in faces and fire

Moving natural scenes pose a challenge to the human visual system, containing diverse objects, clutter, and backgrounds. Well-known models of object recognition do not fully explain natural scene perception, ignoring segmentation or the recognition of dynamic objects. In this thesis, we use a familiar natural stimulus, moving flames, to evaluate the human visual system’s ability to match and search for complex examples of dynamic form. What can analysis in the image domain tell us about dynamic flame? Using image statistics, Fourier analysis and motion evaluation algorithms, we analysed a highresolution dataset typical of moving flame. We characterise it as a motion-rich stimulus with an exponential power spectrum and few long-range spatial or temporal correlations. Are observers able to effectively encode and recognise dynamic flame stimuli? What visual features play an important role in matching? To investigate, we set observers matching tasks using clips from the same dataset. Colour changes do not affect matching on short clips, but inversion and reversal do. We show that dynamic edges are a key component of flame representations. Can observers search well for flame stimuli? Can they detect targets (short flame clips) in equally-sized longer clips? Using temporal search tasks, we show that observers’ accuracy drops quickly as the search space grows; there is no pop-out. Accuracy is not so strongly affected by a blank ISI, however, showing that search difficulties, rather than representational decay, are to blame. In conclusion, we find that the human visual system is capable of matching the complex motion patterns of dynamic flame, but finds search much harder. We find no evidence of category orientation specialisation. Combining several experimental results, we suggest that the representation of dynamic flame is neither snapshot-based nor dedicated and high-level, but relies on the encoding of sparse, local spatiotemporal features