DeeSIL: Deep-Shallow Incremental Learning

Incremental Learning (IL) is an interesting AI problem when the algorithm is assumed to work on a budget. This is especially true when IL is modeled using a deep learning approach, where two com- plex challenges arise due to limited memory, which induces catastrophic forgetting and delays related to the retraining needed in order to incorpo- rate new classes. Here we introduce DeeSIL, an adaptation of a known transfer learning scheme that combines a fixed deep representation used as feature extractor and learning independent shallow classifiers to in- crease recognition capacity. This scheme tackles the two aforementioned challenges since it works well with a limited memory budget and each new concept can be added within a minute. Moreover, since no deep re- training is needed when the model is incremented, DeeSIL can integrate larger amounts of initial data that provide more transferable features. Performance is evaluated on ImageNet LSVRC 2012 against three state of the art algorithms. Results show that, at scale, DeeSIL performance is 23 and 33 points higher than the best baseline when using the same and more initial data respectively

The lifecycle of geotagged data

The world is a big place. At any given instant something is happening somewhere, but even when nothing in particular is going on people still find ways to generate data, such as posting on s

“When Was This Picture Taken?” – Image Date Estimation in the Wild

The problem of automatically estimating the creation date of photos has been addressed rarely in the past. In this paper, we introduce a novel dataset Date Estimation in the Wild for the task of predicting the acquisition year of images captured in the period from 1930 to 1999. In contrast to previous work, the dataset is neither restricted to color photography nor to specific visual concepts. The dataset consists of more than one million images crawled from Flickr and contains a large number of different motives. In addition, we propose two baseline approaches for regression and classification, respectively, relying on state-of-the-art deep convolutional neural networks. Experimental results demonstrate that these baselines are already superior to annotations of untrained humans

The network structure of visited locations according to geotagged social media photos

Businesses, tourism attractions, public transportation hubs and other points of interest are not isolated but part of a collaborative system. Making such collaborative network surface is not always an easy task. The existence of data-rich environments can assist in the reconstruction of collaborative networks. They shed light into how their members operate and reveal a potential for value creation via collaborative approaches. Social media data are an example of a means to accomplish this task. In this paper, we reconstruct a network of tourist locations using fine-grained data from Flickr, an online community for photo sharing. We have used a publicly available set of Flickr data provided by Yahoo! Labs. To analyse the complex structure of tourism systems, we have reconstructed a network of visited locations in Europe, resulting in around 180,000 vertices and over 32 million edges. An analysis of the resulting network properties reveals its complex structure.Comment: 8 pages, 3 figure

Shonan Rotation Averaging: Global Optimality by Surfing SO(p)nSO(p)^n

Shonan Rotation Averaging is a fast, simple, and elegant rotation averaging algorithm that is guaranteed to recover globally optimal solutions under mild assumptions on the measurement noise. Our method employs semidefinite relaxation in order to recover provably globally optimal solutions of the rotation averaging problem. In contrast to prior work, we show how to solve large-scale instances of these relaxations using manifold minimization on (only slightly) higher-dimensional rotation manifolds, re-using existing high-performance (but local) structure-from-motion pipelines. Our method thus preserves the speed and scalability of current SFM methods, while recovering globally optimal solutions.Comment: 30 pages (paper + supplementary material). To appear at the European Conference on Computer Vision (ECCV) 202

Re-ranking Permutation-Based Candidate Sets with the n-Simplex Projection

In the realm of metric search, the permutation-based approaches have shown very good performance in indexing and supporting approximate search on large databases. These methods embed the metric objects into a permutation space where candidate results to a given query can be efficiently identified. Typically, to achieve high effectiveness, the permutation-based result set is refined by directly comparing each candidate object to the query one. Therefore, one drawback of these approaches is that the original dataset needs to be stored and then accessed during the refining step. We propose a refining approach based on a metric embedding, called n-Simplex projection, that can be used on metric spaces meeting the n-point property. The n-Simplex projection provides upper- and lower-bounds of the actual distance, derived using the distances between the data objects and a finite set of pivots. We propose to reuse the distances computed for building the data permutations to derive these bounds and we show how to use them to improve the permutation-based results. Our approach is particularly advantageous for all the cases in which the traditional refining step is too costly, e.g. very large dataset or very expensive metric function

Rethinking summarization and storytelling for modern social multimedia

Traditional summarization initiatives have been focused on specific types of documents such as articles, reviews, videos, image feeds, or tweets, a practice which may result in pigeonholing the summarization task in the context of modern, content-rich multimedia collections. Consequently, much of the research to date has revolved around mostly toy problems in narrow domains and working on single-source media types. We argue that summarization and story generation systems need to re-focus the problem space in order to meet the information needs in the age of user-generated content in different formats and languages. Here we create a framework for flexible multimedia storytelling. Narratives, stories, and summaries carry a set of challenges in big data and dynamic multi-source media that give rise to new research in spatial-temporal representation, viewpoint generation, and explanatio

Detecting natural disasters, damage, and incidents in the wild

Responding to natural disasters, such as earthquakes, floods, and wildfires, is a laborious task performed by on-the-ground emergency responders and analysts. Social media has emerged as a low-latency data source to quickly understand disaster situations. While most studies on social media are limited to text, images offer more information for understanding disaster and incident scenes. However, no large-scale image datasets for incident detection exists. In this work, we present the Incidents Dataset, which contains 446,684 images annotated by humans that cover 43 incidents across a variety of scenes. We employ a baseline classification model that mitigates false-positive errors and we perform image filtering experiments on millions of social media images from Flickr and Twitter. Through these experiments, we show how the Incidents Dataset can be used to detect images with incidents in the wild. Code, data, and models are available online at http://incidentsdataset.csail.mit.edu.Comment: ECCV 202

Knee disorders in primary care: design and patient selection of the HONEUR knee cohort.

BACKGROUND: Knee complaints are a frequent reason for consultation in general practice. These patients constitute a specific population compared to secondary care patients. However, information to base treatment decisions on is generally derived from specialistic settings. Our cohort study is aimed at collecting knowledge about prognosis and prognostic factors of knee complaints presented in a primary care setting. This paper describes the methods used for data collection, and discusses potential selectiveness of patient recruitment. METHODS: This is a descriptive prospective cohort study with one-year follow-up. 40 Dutch GPs recruited consecutive patients with incident knee complaints aged 12 years and above from October 2001 to October 2003. Patients were assessed with questionnaires and standardised physical examinations. Additional measurements of subgroups included MRI for recent knee traumas and device assessed function measurements for non-traumatic patients. After the inclusion period we retrospectively searched the computerized medical files of participating GPs to obtain a sample to determine possible selective recruitment. We assessed differences in proportions of gender, traumatic onset of injury and age groups between participants and non-participants using Odds Ratios (OR) and 95% confidence intervals. RESULTS: We recruited 1068 patients. In a sample of 310 patients visiting the GP, we detected some selective recruitment, indicating an underrepresentation of patients aged 12 to 35 years (OR 1.70; 1.15-2.77), especially among men (OR 2.16; 1.12-4.18). The underrepresentation of patients with traumatic onset of injury was not statistically significant. CONCLUSION: This cohort is unique in its size, setting, and its range of both age and type of knee complaints. We believe the detected selective recruitment is unlikely to introduce significant bias, as the cohort will be divided into subgroups according to age group or traumatic onset of injury for future analyses. However, the underrepresentation of men in the age group of 12 to 35 years of age warrants caution. Based on the available data, we believe our cohort is an acceptable representation of patients with new knee complaints consulting the GP, and we expect no problems with extrapolation of the results to the general Dutch population