Visual Landmark Recognition from Internet Photo Collections: A Large-Scale Evaluation

The task of a visual landmark recognition system is to identify photographed buildings or objects in query photos and to provide the user with relevant information on them. With their increasing coverage of the world's landmark buildings and objects, Internet photo collections are now being used as a source for building such systems in a fully automatic fashion. This process typically consists of three steps: clustering large amounts of images by the objects they depict; determining object names from user-provided tags; and building a robust, compact, and efficient recognition index. To this date, however, there is little empirical information on how well current approaches for those steps perform in a large-scale open-set mining and recognition task. Furthermore, there is little empirical information on how recognition performance varies for different types of landmark objects and where there is still potential for improvement. With this paper, we intend to fill these gaps. Using a dataset of 500k images from Paris, we analyze each component of the landmark recognition pipeline in order to answer the following questions: How many and what kinds of objects can be discovered automatically? How can we best use the resulting image clusters to recognize the object in a query? How can the object be efficiently represented in memory for recognition? How reliably can semantic information be extracted? And finally: What are the limiting factors in the resulting pipeline from query to semantics? We evaluate how different choices of methods and parameters for the individual pipeline steps affect overall system performance and examine their effects for different query categories such as buildings, paintings or sculptures

Intimate Nevada: Artists Respond

Creative Works Winner Most of us know Nevada beyond the Strip. It’s a place of houses, of shopping plazas, of movie theaters, and grocery stores. A place of hotels that are also places of work. A place of basins, ranges, vistas, and nature. A place of personal history. For Intimate Nevada: Artists Respond, curators Lauren Paljusaj (ENG BA ‘20) and Anne Savage (CFA BA ‘22), draw on photographs found in UNLV Special Collections to uncover the intimate visuality of a Nevada of past centuries. The exhibition focuses on how the imaged built landscape of early 20th century Southern Nevada (Paljusaj) and candids and personal snapshots of 1910s Las Vegas (Savage) allow us to interpret the past in light of who we are today. It also shows how artists utilize research archives and the bottomless fascination of material memory to respond to historical artifacts

Data Driven Discovery in Astrophysics

We review some aspects of the current state of data-intensive astronomy, its methods, and some outstanding data analysis challenges. Astronomy is at the forefront of "big data" science, with exponentially growing data volumes and data rates, and an ever-increasing complexity, now entering the Petascale regime. Telescopes and observatories from both ground and space, covering a full range of wavelengths, feed the data via processing pipelines into dedicated archives, where they can be accessed for scientific analysis. Most of the large archives are connected through the Virtual Observatory framework, that provides interoperability standards and services, and effectively constitutes a global data grid of astronomy. Making discoveries in this overabundance of data requires applications of novel, machine learning tools. We describe some of the recent examples of such applications.Comment: Keynote talk in the proceedings of ESA-ESRIN Conference: Big Data from Space 2014, Frascati, Italy, November 12-14, 2014, 8 pages, 2 figure

PlaNet - Photo Geolocation with Convolutional Neural Networks

Is it possible to build a system to determine the location where a photo was taken using just its pixels? In general, the problem seems exceptionally difficult: it is trivial to construct situations where no location can be inferred. Yet images often contain informative cues such as landmarks, weather patterns, vegetation, road markings, and architectural details, which in combination may allow one to determine an approximate location and occasionally an exact location. Websites such as GeoGuessr and View from your Window suggest that humans are relatively good at integrating these cues to geolocate images, especially en-masse. In computer vision, the photo geolocation problem is usually approached using image retrieval methods. In contrast, we pose the problem as one of classification by subdividing the surface of the earth into thousands of multi-scale geographic cells, and train a deep network using millions of geotagged images. While previous approaches only recognize landmarks or perform approximate matching using global image descriptors, our model is able to use and integrate multiple visible cues. We show that the resulting model, called PlaNet, outperforms previous approaches and even attains superhuman levels of accuracy in some cases. Moreover, we extend our model to photo albums by combining it with a long short-term memory (LSTM) architecture. By learning to exploit temporal coherence to geolocate uncertain photos, we demonstrate that this model achieves a 50% performance improvement over the single-image model

Smartphone picture organization: a hierarchical approach

We live in a society where the large majority of the population has a camera-equipped smartphone. In addition, hard drives and cloud storage are getting cheaper and cheaper, leading to a tremendous growth in stored personal photos. Unlike photo collections captured by a digital camera, which typically are pre-processed by the user who organizes them into event-related folders, smartphone pictures are automatically stored in the cloud. As a consequence, photo collections captured by a smartphone are highly unstructured and because smartphones are ubiquitous, they present a larger variability compared to pictures captured by a digital camera. To solve the need of organizing large smartphone photo collections automatically, we propose here a new methodology for hierarchical photo organization into topics and topic-related categories. Our approach successfully estimates latent topics in the pictures by applying probabilistic Latent Semantic Analysis, and automatically assigns a name to each topic by relying on a lexical database. Topic-related categories are then estimated by using a set of topic-specific Convolutional Neuronal Networks. To validate our approach, we ensemble and make public a large dataset of more than 8,000 smartphone pictures from 40 persons. Experimental results demonstrate major user satisfaction with respect to state of the art solutions in terms of organization.Peer ReviewedPreprin

Automatic Synchronization of Multi-User Photo Galleries

In this paper we address the issue of photo galleries synchronization, where pictures related to the same event are collected by different users. Existing solutions to address the problem are usually based on unrealistic assumptions, like time consistency across photo galleries, and often heavily rely on heuristics, limiting therefore the applicability to real-world scenarios. We propose a solution that achieves better generalization performance for the synchronization task compared to the available literature. The method is characterized by three stages: at first, deep convolutional neural network features are used to assess the visual similarity among the photos; then, pairs of similar photos are detected across different galleries and used to construct a graph; eventually, a probabilistic graphical model is used to estimate the temporal offset of each pair of galleries, by traversing the minimum spanning tree extracted from this graph. The experimental evaluation is conducted on four publicly available datasets covering different types of events, demonstrating the strength of our proposed method. A thorough discussion of the obtained results is provided for a critical assessment of the quality in synchronization.Comment: ACCEPTED to IEEE Transactions on Multimedi

Excavating the Archive: Heritage-making Practices in Cornwall’s Clay Country.

In 1999 English China Clays (the then principal china clay producer in Cornwall and west Devon) was acquired by the multinational industrial minerals company Imerys. Shortly after, a group of concerned clay workers and local historians came together in a salvage mission to recover historical documents which had been deemed expendable during the business takeover. Together they ransacked offices and emptied filing cabinets collecting historic documentation about the industry. In the eighteen years that have followed, the china clay industry and its associated landscape have undergone immense change and transformation. Meanwhile, that small band of individuals has grown into the China Clay History Society (CCHS). CCHS is now in the process of formalising their salvaged collection, with curatorial expertise from the Wheal Martyn Museum (of which the CCHS is a component part). In this thesis, the CCHS archive and its associated community relationships are examined in relation to experiences of past loss, present instability, and the hope of future renewal. Over an extended period of participant observation working alongside the caretakers of the archive, I explored the different practices of collecting, sorting, and valuing which are making and remaking china clay heritage in mid-Cornwall. Drawing on heritage studies and past studies of collecting, as well as professional museum and archival scholarship, this thesis emphasises the role that practice and material relationships play in the assembling of heritage (Macdonald 2009). Two distinct modes of ordering (Law 1994; 2004) – ‘Passion’ and ‘Purpose’ – are identified as central to this research, which aims to show how different practices of collecting and valuing have profound implications for the ways china clay heritage may be performed in the future.Arts and Humanities Research Council (AHRC

The Care and Preservation of an Island Mountain Archaeological Textile: A Collections Management Project

Like many archaeological collections worldwide, the University of Nevada, Reno, Anthropology Museum’s Island Mountain collection has been impacted by the far-reaching curation crisis. This thesis discusses the variety of preservation needs for archaeological collections to remain viable sources of future scholarship, focusing on the curation crisis, preventative conservation methods, and best practices for archaeologists and students when dealing with delicate artifacts such as archaeological textiles. These concepts are applied to an at-risk archaeological textile from the Island Mountain collection, documenting the remedying measures taken and the analysis of the textile