Automatically generated interactive weather reports based on webcam images

Most weather reports are either based on data from dedicated weather stations, satellite images, manual measurements or forecasts. In this paper a system that automatically generates weather reports using the contents on webcam images are proposed. There are thousands of openly available webcams on the Internet that provide images in real time. A webcam image can reveal much about the weather conditions at a particular site and this study demonstrates a strategy for automatically classifying a webcam scene into cloudy, partially cloudy, sunny, foggy and night. The system has been run for several months collecting 60 Gb of image data from webcams across the world. The reports are available through an interactive web-based interface. A selection of benchmark images was manually tagged to assess the accuracy of the weather classification which reached a success rate of 67.3%

Automated Image Classification for Post-Earthquake Reconnaissance Images

In the aftermath of earthquake events, many reconnaissanceteams are dispatched to collect as much data as possible, movingquickly to capture the damages and failures on our built environments before they are recovered. Unfortunately, only a tiny portionof these images are shared, curated, and utilized. There is a pressing need for a viable visual data organizing or categorizing tool witha minimal manual effort. In this study, we aim to build a system toautomate classifying and analyzing a large volume of post-disastervisual data. Our system called Automated Reconnaissance ImageOrganizer (ARIO) is a web-based tool to automatically categorizing reconnaissance images using a deep convolutional neural net-work and generate a summary report combined with useful metadata. Automated classifiers trained using our ground-truth visualdatabase classify images into various categories that are useful toreadily analyze and document reconnaissance images from post-disaster buildings in the field

Mining user activity as a context source for search and retrieval

Nowadays in information retrieval it is generally accepted that if we can better understand the context of users then this could help the search process, either at indexing time by including more metadata or at retrieval time by better modelling the user context. In this work we explore how activity recognition from tri-axial accelerometers can be employed to model a user's activity as a means of enabling context-aware information retrieval. In this paper we discuss how we can gather user activity automatically as a context source from a wearable mobile device and we evaluate the accuracy of our proposed user activity recognition algorithm. Our technique can recognise four kinds of activities which can be used to model part of an individual's current context. We discuss promising experimental results, possible approaches to improve our algorithms, and the impact of this work in modelling user context toward enhanced search and retrieval

Picasso: A Modular Framework for Visualizing the Learning Process of Neural Network Image Classifiers

Picasso is a free open-source (Eclipse Public License) web application written in Python for rendering standard visualizations useful for analyzing convolutional neural networks. Picasso ships with occlusion maps and saliency maps, two visualizations which help reveal issues that evaluation metrics like loss and accuracy might hide: for example, learning a proxy classification task. Picasso works with the Tensorflow deep learning framework, and Keras (when the model can be loaded into the Tensorflow backend). Picasso can be used with minimal configuration by deep learning researchers and engineers alike across various neural network architectures. Adding new visualizations is simple: the user can specify their visualization code and HTML template separately from the application code.Comment: 9 pages, submission to the Journal of Open Research Software, github.com/merantix/picass

Facial Expression Recognition from World Wild Web

Recognizing facial expression in a wild setting has remained a challenging task in computer vision. The World Wide Web is a good source of facial images which most of them are captured in uncontrolled conditions. In fact, the Internet is a Word Wild Web of facial images with expressions. This paper presents the results of a new study on collecting, annotating, and analyzing wild facial expressions from the web. Three search engines were queried using 1250 emotion related keywords in six different languages and the retrieved images were mapped by two annotators to six basic expressions and neutral. Deep neural networks and noise modeling were used in three different training scenarios to find how accurately facial expressions can be recognized when trained on noisy images collected from the web using query terms (e.g. happy face, laughing man, etc)? The results of our experiments show that deep neural networks can recognize wild facial expressions with an accuracy of 82.12%

Multiple Evidence Combination in Image retrieval: Diogenes Searches for People on the Web

Abstract In this work, we examine evidence combination mechAnisms for classifying multimedia information. In particular, we examine linear and Dempster-Shafer methods of evidence combination in the context of identifying personal images on the World Wide Web. An automatic web search engine named Diogenes 1 searches the web for personal images and combines different pieces of evidence for identification. The sources of evidence consist of input from face detection/recognition and text/HTML analysis modules. A degree of uncertainty is involved with both of these sources. Diogenes automatically determines the uncertainty locally for each retrieval and uses this information to set a relative significance for each evidence. To our knowledge, Diogenes is the first image search engine using Dempster-Shafer evidence combination based on automatic object recognition and dynamic local uncertainty assessment. In our experiments Diogenes comfortably outperformed some well known commercial and research prototype image search engines for celebrity image queries