Building and Tracking Hierarchical Geographical & Temporal Partitions for Image Collection Management on Mobile Devices

International audienceUsage of mobile devices (phones, digital cameras) raises the need for organizing large personal image collections. In accordance with studies on user needs, we propose a statistical criterion and an associated optimization technique, relying on geo-temporal image metadata, for building and tracking a hierarchical structure on the image collection. In a mixture model framework, particularities of the application and typical data sets are taken into account in the design of the scheme (incrementality, ability to cope with non-Gaussian data, with both small and large samples). Results are reported on real data sets

Organisation statistique spatio-temporelle d'une collection d'images acquises d'un terminal mobile géolocalisé

International audienceNous présentons une technique automatique d'organisation de collection d'images personnelles, pour répondre aux besoins particuliers émergents des téléphones portables équipés d'appareil photographique. Après avoir examiné ce qui fait la particularit é de ce contexte, nous proposons une technique de structuration de collection d'image basée sur la date et le lieu de prise de vue des images. L'objectif est formalisé comme un problème de classification non-supervisée, temporelle et spatiale. Le critère statistique de vraisemblance complétée intégrée (ICL) est retenu, car il fournit une solution efficace pour déterminer la complexité du modèle et un bon niveau de séparabilité de ses composantes, tout en limitant le caractère arbitraire de la paramétrisation. La fiabilité des classifications obtenues est ensuite évaluée, afin d'en sélectionner la plus pertinente, pour fournir une structure utilisable avec une interface de type calendrier électronique permettant d'explorer la collection

Investigating the use of Photo Collection Structures for Photo Searching

In this paper, we investigate the use of photo collection structures in aiding the rapid retrieval of events, single photographs and sets of photographs sharing a common property. As these structures encode a great deal of useful contextual information, we advocate for the need for photo searching interfaces to exploit and expose this information. In our study of people’s photo collection structures, we found that people organise their photographs into event folders. Providing rapid access to events, singles and properties equates to locating event folders quickly. When event names are well known, we advocate for the use keyword based searches. Temporal based navigation becomes increasing important when event folders are less well known. We found a significant amount of data showing that people do organise and structure their photo collections more than previous literature suggests. The number of different property folders we found, illustrates the range of different tasks people perform when structuring and organizing their photo collections. In concluding this paper, we also make a number of recommendations for photo searching interfaces

IMAGE MANAGEMENT USING PATTERN RECOGNITION SYSTEMS

With the popular usage of personal image devices and the continued increase of computing power, casual users need to handle a large number of images on computers. Image management is challenging because in addition to searching and browsing textual metadata, we also need to address two additional challenges. First, thumbnails, which are representative forms of original images, require significant screen space to be represented meaningfully. Second, while image metadata is crucial for managing images, creating metadata for images is expensive. My research on these issues is composed of three components which address these problems. First, I explore a new way of browsing a large number of images. I redesign and implement a zoomable image browser, PhotoMesa, which is capable of showing thousands of images clustered by metadata. Combined with its simple navigation strategy, the zoomable image environment allows users to scale up the size of an image collection they can comfortably browse. Second, I examine tradeoffs of displaying thumbnails in limited screen space. While bigger thumbnails use more screen space, smaller thumbnails are hard to recognize. I introduce an automatic thumbnail cropping algorithm based on a computer vision saliency model. The cropped thumbnails keep the core informative part and remove the less informative periphery. My user study shows that users performed visual searches more than 18% faster with cropped thumbnails. Finally, I explore semi-automatic annotation techniques to help users make accurate annotations with low effort. Automatic metadata extraction is typically fast but inaccurate while manual annotation is slow but accurate. I investigate techniques to combine these two approaches. My semi-automatic annotation prototype, SAPHARI, generates image clusters which facilitate efficient bulk annotation. For automatic clustering, I present hierarchical event clustering and clothing based human recognition. Experimental results demonstrate the effectiveness of the semi-automatic annotation when applied on personal photo collections. Users were able to make annotation 49% and 6% faster with the semi-automatic annotation interface on event and face tasks, respectively

Käyttäjäkeskeisen metatiedon suunnittelu digitaaliseen näppäilykuvaukseen

With digital cameras, camera phones, and camcorders connected to personal computers and the Internet, people are capturing, sharing, and storing more and more media created by themselves. This has created the organizing challenge called media management problem: how to browse and find media from the growing personal media collections. Metadata is seen as one of the most promising technologies to address this problem. Previous work on metadata design has focused on public or commercially produced media – not on media created by consumers for themselves or their social networks. Taking snapshot photography as a starting point, this dissertation looks into the media management problem from a user-centric design perspective. In this work, three consecutive systems were designed: MMM-1, MobShare, and PhotosToFriends. Each of these systems was evaluated in extensive user trials. The objective of the user trials was to provide a better understanding of the uses people have for mobile photos and metadata. The results gained from constructing the systems and the associated user trials are divided into three. First, they contribute to understanding social uses for mobile photographs: what was photographed, with whom were the photos shared, and what kind of social activity emerged. Second, the results can be applied into designing systems for photo sharing: the use of galleries, control over sharing, support for discussions, and social awareness and notifications. Third, what are the implications for metadata that the social uses and system design have. The main implications are the inherent problem of personal photo information being contextual, dynamic, and highly semantic, and the strong coupling of metadata and its application. To address these problems we propose the concept of social metadata, which takes advantage of the social activity in photo sharing systems. The user-centric design approach also brings forth that content metadata generation should be approached from several angles – social metadata, user tagging and contextual information – in addition to the more traditional content-based analysis.Digitaalikameroiden, kamerapuhelimien ja digitaalisten videokameroiden ollessa nykyään yhdistettynä kotitietokoneiseen ja sitä kautta Internetiin ihmiset luovat, jakavat ja tallentavat entistä enemmän itse tekemäänsä mediaa. Tämän seurauksena on syntynyt organisointihaaste, jota kutsutaan henkilökohtaisen median hallintaongelmaksi: kuinka selata mediaa tai löytää sieltä haluamansa media? Metatietoa pidetään yhtenä lupaavimmista teknologisista ratkaisuista tähän ongelmaan. Aikaisempi tutkimus metatiedon suunnittelussa on keskittynyt julkiseen tai kaupalliseen mediaan – ei mediaan, jonka kuluttajat luovat itselleen tai sosiaaliselle verkostolleen. Tämä väitöskirja tutkii median hallintaongelmaa käyttäjäkeskeisestä näkökulmasta, jossa lähtökohtana on näppäilyvalokuvaus. Tässä työssä rakennettiin kolme peräkkäistä järjestelmää: MMM-1, MobShare ja PhotosToFriends. Kukin näistä järjestelmistä evaluoitiin kattavilla käyttäjäkokeilla, joiden tavoitteena oli laajentaa ymmärrystä ihmisten mobiilikuvien käytöstä sekä metatiedosta. Järjestelmien rakentamisesta ja niihin liittyvistä käyttäjäkokeista saadut tulokset voidaan jakaa kolmeen osaan. Ensiksi, tulosten kautta voidaan paremmin ymmärtää mobiilikuvien sosiaalista käyttöä: mitä kuvattiin, keille kuvia jaettiin ja minkälaista sosiaalista toimintaa syntyi. Toiseksi, tuloksia voidaan käyttää kuvienjakojärjestelmien suunnittelussa: gallerioiden käyttö, jakamisen kontrollointi, keskustelun tukeminen sekä tietoisuus ja tiedottaminen sosiaalisesta aktiivisuudesta. Kolmanneksi, mitä seuraamuksia kahdella edellisellä kohdalla on metatiedon suunnittelulle. Tärkeimmät seuraamukset ovat se, että henkilökohtainen tieto on kontekstuaalista, dynaamista ja semanttista, ja metatiedon sekä sitä käyttävien sovellusten välillä on vahva keskinäinen liitos. Näiden ongelmien ratkaisemiseksi ehdotamme sosiaalisen metatiedon käsitettä, joka käyttää hyväksi kuvienjakojärjestelmien sosiaalista aktiivisuutta. Käyttäjäkeskeinen lähestyminen suunnittelussa nostaa esille myös, että sisältöä kuvaavan metatiedon luomista pitäisi lähestyä useasta eri näkökulmasta – sosiaalinen metatieto, käyttäjien "tägit" ja konteksti-informaatio – perinteisen sisältöpohjaisen analyysin lisäksi.reviewe

Flexible photo retrieval (FlexPhoReS) : a prototype for multimodel personal digital photo retrieval

Digital photo technology is developing rapidly and is motivating more people to have large personal collections of digital photos. However, effective and fast retrieval of digital photos is not always easy, especially when the collections grow into thousands. World Wide Web (WWW) is one of the platforms that allows digital photo users to publish a collection of photos in a centralised and organised way. Users typically find their photos by searching or browsing uSing a keyboard and mouse. Also in development at the moment are alternative user interfaces such as graphical user interfaces with speech (S/GUI) and other multimodal user interfaces which offer more flexibility to users. The aim of this research was to design and evaluate a flexible user interface for a web based personal digital photo retrieval system. A model of a flexible photo retrieval system (FlexPhoReS) was developed based on a review of the literature and a small-scale user study. A prototype, based on the model, was built using MATLAB and WWW technology. FlexPhoReS is a web based personal digital photo retrieval prototype that enables digital photo users to . accomplish photo retrieval tasks (browsing, keyword and visual example searching (CBI)) using either mouse and keyboard input modalities or mouse and speech input modalities. An evaluation with 20 digital photo users was conducted using usability testing methods. The result showed that there was a significant difference in search performance between using mouse and keyboard input modalities and using mouse and speech input modalities. On average, the reduction in search performance time due to using mouse and speech input modalities was 37.31%. Participants were also significantly more satisfied with mouse and speech input modalities than with mouse and keyboard input modalities although they felt that both were complementary. This research demonstrated that the prototype was successful in providing a flexible model of the photo retrieval process by offering alternative input modalities through a multimodal user interface in the World Wide Web environment.EThOS - Electronic Theses Online ServiceGBUnited Kingdo

Photo searching on small screen devices

Word processed copy.Includes bibliographical references (p. 221-239).The aim of this thesis is to improve HCI (Human-Computer Interaction) knowledge in the design of the next generation of photo search tools on small display devices. Today, these devices have all the ingredients for a truly mobile photo collection, such as large storage, multiple networking capabilities and high resolution screens. However, they lack the tools for searching through large collections of photographs. This is particularly important as users have expressed a desire to store images on mobile devices in the long term. No substantial research has looked at addressing users searching needs. Few researchers have considered the importance of supporting both searching and browsing to cater for user needs. None that we could find have assessed the potential impact of adding desktop-based annotation capabilities. Consequently, this thesis seeks to address these challenges and provide an empirical foundation for the design of photo search tools. To achieve these objectives an iterative user-centered design methodology was employed. The end practical result was a single photo search interface that incorporates the best traits of a variety of tools to support search. The thesis reflects on each cycle in the iterative design process. The first major area of contribution to the field of HCI improves existing knowledge on photo searching behavior by providing a number of empirically grounded findings about searching behavior. It identifies some of the core factors that influence search strategies and outlines a conceptual framework to guide the design of future systems. The second area of contribution is a single photo searching tool for small display devices that is based on iterative studies of various user interface designs. It integrates multiple search methods within a single user interface. In contrast to previous research in this area, the design is centered on locating events rather than individual images as we found that people naturally associate photographs with events when searching