Large-scale interactive exploratory visual search

Large scale visual search has been one of the challenging issues in the era of big data. It demands techniques that are not only highly effective and efficient but also allow users conveniently express their information needs and refine their intents. In this thesis, we focus on developing an exploratory framework for large scale visual search. We also develop a number of enabling techniques in this thesis, including compact visual content representation for scalable search, near duplicate video shot detection, and action based event detection. We propose a novel scheme for extremely low bit rate visual search, which sends compressed visual words consisting of vocabulary tree histogram and descriptor orientations rather than descriptors. Compact representation of video data is achieved through identifying keyframes of a video which can also help users comprehend visual content efficiently. We propose a novel Bag-of-Importance model for static video summarization. Near duplicate detection is one of the key issues for large scale visual search, since there exist a large number nearly identical images and videos. We propose an improved near-duplicate video shot detection approach for more effective shot representation. Event detection has been one of the solutions for bridging the semantic gap in visual search. We particular focus on human action centred event detection. We propose an enhanced sparse coding scheme to model human actions. Our proposed approach is able to significantly reduce computational cost while achieving recognition accuracy highly comparable to the state-of-the-art methods. At last, we propose an integrated solution for addressing the prime challenges raised from large-scale interactive visual search. The proposed system is also one of the first attempts for exploratory visual search. It provides users more robust results to satisfy their exploring experiences

CHORUS Deliverable 2.1: State of the Art on Multimedia Search Engines

Based on the information provided by European projects and national initiatives related to multimedia search as well as domains experts that participated in the CHORUS Think-thanks and workshops, this document reports on the state of the art related to multimedia content search from, a technical, and socio-economic perspective. The technical perspective includes an up to date view on content based indexing and retrieval technologies, multimedia search in the context of mobile devices and peer-to-peer networks, and an overview of current evaluation and benchmark inititiatives to measure the performance of multimedia search engines. From a socio-economic perspective we inventorize the impact and legal consequences of these technical advances and point out future directions of research

Enhancing Virtual City with Collective Memory to Support Urban Design Learning

This research investigates the pedagogical effects of collective memory enhanced virtual city models as an e-learning environment for urban design in an Indonesian university context. In this study, collective memory refers to expressions people made when remembering buildings, spaces and places of the past and present city living. A city's collective memory is either elicited from or produced directly by the city's residents or visitors. The thesis presents the outcomes of designing an experimental collective memory enhanced virtual city (CREATI) platform and applying it to the urban design course at the Department of Architecture, Atma Jaya Yogyakarta University, Indonesia. The Jalan Malioboro area in Yogyakarta was chosen as the case study site for constructing 30 virtual city models and gathering records of collective memory associated with the street and city. Student participants were invited to use CREATI while undertaking the urban design course. The usability and effects of CREATI on the students' learning outcomes are analysed according to the learning experiences reported by the students as well as the urban design proposals they produced for the course. The study finds that students appear more knowledgeable about the urban contextual issues because of accessing and sharing the resources hosted on CREATI. Students also reported that they felt better supported in developing more context­ sensitive design proposals by working with the collective memory enhanced virtual city models

CHORUS Deliverable 3.3: Vision Document - Intermediate version

The goal of the CHORUS vision document is to create a high level vision on audio-visual search engines in order to give guidance to the future R&D work in this area (in line with the mandate of CHORUS as a Coordination Action). This current intermediate draft of the CHORUS vision document (D3.3) is based on the previous CHORUS vision documents D3.1 to D3.2 and on the results of the six CHORUS Think-Tank meetings held in March, September and November 2007 as well as in April, July and October 2008, and on the feedback from other CHORUS events. The outcome of the six Think-Thank meetings will not just be to the benefit of the participants which are stakeholders and experts from academia and industry – CHORUS, as a coordination action of the EC, will feed back the findings (see Summary) to the projects under its purview and, via its website, to the whole community working in the domain of AV content search. A few subjections of this deliverable are to be completed after the eights (and presumably last) Think-Tank meeting in spring 2009

Design and Evaluation of User Interfaces for Mobile Web Search

Mobiili tiedonhaku on jatkuvasti kasvava ja monimuotoistuva osa jokapäiväistä tiedonhankintaa. Aikaisemman tutkimuksen mukaan tarvitaan kuitenkin parempia käyttöliittymäratkaisuja tukemaan mobiililaitteilla tapahtuvaa verkkotiedonhakua. Väitöskirjatutkimuksessa suunniteltiin ja toteutettiin kaksi uutta hakukäyttöliittymää, joita arvioitiin käyttäjätutkimuksissa. Ensimmäinen käyttöliittymä perustuu siihen, että hakutulokset luokitellaan ryhmiin niissä esiintyvien avainsanojen perusteella. Käyttäjätutkimusten tulokset osoittavat, että luokittelulla voidaan tukea mobiilikäyttäjien tutkivaa tiedonhakua. Toinen käyttöliittymä antaa hakutulosten yhteydessä yleiskuvan hakulauseen sijaintikohdista tulosdokumenteissa. Vaikkakin menetelmän käyttö vaatii opettelua, käyttäjäarviot osoittavat että se voi auttaa sivuuttamaan huonot hakutulokset, etenkin silloin kun muut hakutulosta kuvaavat tiedot ovat epäselviä. Lisäksi väitöskirjassa tutkittiin aktiivisten mobiili-Internetin käyttäjien tiedontarpeita verkkotiedonhaun käytön ymmärtämiseksi. Tutkimustulosten mukaan hakujen tekeminen ja verkon selaaminen ovat näiden käyttäjien tärkeimpiä tiedonhankintatapoja. Niillä pyritään vastaamaan tiedontarpeisiin heti niiden ilmaantuessa, olipa käyttäjä sitten kotona, liikkeessä tai sosiaalisessa vuorovaikutustilanteessa. Mobiili tiedonhankinta on vahvasti sidoksissa käyttötilanteeseen, mikä tulee huomioida hakukäyttöliittymien suunnittelussa. Tulevaisuuden hakukäyttöliittymät voivat esimerkiksi tukea tiedonhankintaa hyödyntämällä tietoa käyttäjän sijainnista ja aktiviteeteista. Myös epämuodollisten ja tutkivien tiedontarpeiden kasvava rooli asettaa uusia haasteita vuorovaikutuksen suunnittelulle.Mobile Web search is a rapidly growing information seeking activity employed across different locations, situations, and activities. Current mobile search interfaces are based on the ranked result list, dominant in desktop interfaces. Research suggests that new paradigms are needed for better support of mobile searchers. For this dissertation, two such novel search interface techniques were designed, implemented, and evaluated. The first method, a clustering search interface that presents a category- based overview of the results, was studied both in a task-based experiment in a laboratory setting and in a longitudinal field study wherein it was used to address real information needs. The results indicate that clustering can support exploratory search needs when the searcher has trouble defining the information need, requires an overview of the search topic, or is interested in multiple results related to the same topic. The findings informed design guidelines for category-based search interfaces. How and when categorization is presented in the search interface needs to be carefully considered. Categorization methods should be improved, for better response to diverse information needs. Hybrid approaches employing contextually informed clustering, classification, and faceted browsing may offer the best match for user needs. The second presentation method, a visualization of the occurrences of the user s query phrase in a result document, can be incorporated into the ranked result list as an additional, unobtrusive result descriptor. It allows the searcher to see how often the query phrase appears in the result document, enabling the use of various evaluation strategies to assess the relevance of the results. Several iterations of the visualization were studied with users to form an understanding of the potential of this approach. The results suggest that a novel visualization can be useful in ruling out non-relevant results and can assist when the other result descriptors do not provide for a conclusive relevance assessment. However, users familiarity with well-established result descriptors means that users have to learn how to integrate the visualization into their search strategies and reconcile situations in which the visualization is in conflict with other metadata. In addition, the contextual triggers and information behaviors of mobile Internet users were studied, for understanding of the role of Web search as a mobile information seeking activity. The results from this study show that mobile Web search and browsing are important information seeking activities. They are engaged in to resolve emerging information needs as they appear, whether at home, on the go, or in social situations

Using contextual information to understand searching and browsing behavior

There is great imbalance in the richness of information on the web and the succinctness and poverty of search requests of web users, making their queries only a partial description of the underlying complex information needs. Finding ways to better leverage contextual information and make search context-aware holds the promise to dramatically improve the search experience of users. We conducted a series of studies to discover, model and utilize contextual information in order to understand and improve users' searching and browsing behavior on the web. Our results capture important aspects of context under the realistic conditions of different online search services, aiming to ensure that our scientific insights and solutions transfer to the operational settings of real world applications