A DOMAIN-CENTRIC APPROACH TO DESIGNING USER INTERFACES OF VIDEO RETRIEVAL SYSTEMS

Thesis (PhD) - Indiana University, Information Science, 2007User- and task-centric efforts in video information retrieval (IR) research are needed because current experiments are showing few significant results. It is our belief that unsatisfactory results in video IR can be partially attributed to the overemphasis on technologically-driven approaches to interface development and system evaluation. This study explored variables that have been consistently overlooked in video retrieval efforts, including those related to domain and search tasks. The underlying goal of this study is to promote alternative means for evaluating video retrieval systems, and to make progress toward developing new design principles and a video seeking model. A series of interactive search runs were conducted using a video retrieval system called ViewFinder. ViewFinder was implemented to search and browse the NASA K - 16 Science Education Programs. The system includes new design features that take into account the unique characteristics of the domain and associated tasks. Users with a background in Science Education, including teachers and academic majors, were recruited to perform a number of search tasks. Results from the search experiments were collected and analyzed using both objective and subjective measures. From these results, researchers gained further knowledge about domain-centric video search tasks, including how textual, visual, and hybrid tasks were all deemed important by science educators. Further analysis of experimental results also revealed associations between search tasks, user interaction, interface features and functions, and system effectiveness. The evaluation of individual interface features and functions exhibited that keyword searching was significant for retrieving Science Education video. However, these experiments also produced positive results for various visual search features. Unlike keyword searching, which was consistent and effective across many task types, the use and effectiveness of visual search and browse features were shown to be task dependent. Overall, the results from this study highlight the importance of user- and task-centric methods in video retrieval, as they provided researchers with additional understanding of the influences of domain-specific search tasks on user interaction with video systems. In addition, the experimental methodology employed for this study encourages future foundations for developing and evaluating video search interfaces designed for specific domains and search tasks

The Evolution of First Person Vision Methods: A Survey

The emergence of new wearable technologies such as action cameras and smart-glasses has increased the interest of computer vision scientists in the First Person perspective. Nowadays, this field is attracting attention and investments of companies aiming to develop commercial devices with First Person Vision recording capabilities. Due to this interest, an increasing demand of methods to process these videos, possibly in real-time, is expected. Current approaches present a particular combinations of different image features and quantitative methods to accomplish specific objectives like object detection, activity recognition, user machine interaction and so on. This paper summarizes the evolution of the state of the art in First Person Vision video analysis between 1997 and 2014, highlighting, among others, most commonly used features, methods, challenges and opportunities within the field.Comment: First Person Vision, Egocentric Vision, Wearable Devices, Smart Glasses, Computer Vision, Video Analytics, Human-machine Interactio

Homm-sw Networks-of-stories to value tangible and intangible heritage in museums

Social inclusion, lifelong learning and the regeneration of competence networks are key processes which foster innovation. Museums may play an important role in these processes and ICT can strongly support the effectiveness of the interventions required. Among them, digital tools used to tell stories are becoming increasingly popular. Narratives in new dimensions enable the formation of personal and community identities, and the construction of meanings [1] [2 ]. Hommsw supports multimedia narratives which enhance tangible and intangible heritage in education and cultural mediation, as well as in tourism. Developed by Officina Emilia (Italy) and Crafts Museum (India), the application is online at www.homm-museums-software.org. So far, it implements the engine for creating and managing the activity ‘networks-of-stories’, to create a nonlinear and open multimedia narration. [3],[4] In the last decade, digital storytelling has spread rapidly due to the growth and possibilities offered by new ICT devices [5] [6]. Together with crowd sourcing it can enrich the understanding of tangible and intangible heritage. Digital storytelling still faces some critical challenges: creation of content on tangible and intangible heritage, classification and re-use of existing documents and clips, cooperative and coordinated production of new content. Moreover, for effective exploratory paths and a more analytical approach to browsing material, contents must be set in the overall perspective of the narrations, to ensure narration is coherent. Finally, validation and dissemination of related outcomes must respect scientific standards. Homm-sw has tools that: support educators, also in contrasting learning difficulties, in developing inclusive and collaborative educational practices; support curators; facilitate crowd sourcing; create a personal web repository of contents and connections; share contents to be published, if approved by the administrator; create a network of contents and applications, at different levels for different users and specific needs. Homm-sw extends the museum experience. Before the visit: users can have a general look at museum contents and note, in their online personal desktop, what they are interested in. During the visit: users can retrieve their notes and add what is available on the museum’s exhibits and augmented reality, hands-on activities and multimedia contents, living laboratories, demonstration programs. Homm-sw in the onsite-mode allows only notes and memos, to fully enjoy what the museum uniquely offers. After the visit: users access their online workspace to retrieve and explore their notes, and any other content, as much as they wish. Beyond those common to other ICT tools in use in museums, Homm-sw has two key innovative functions. First, recording and retrieval of users' activities: during the visit in a museum, the visitor accessing her account may browse and take notes in her personal workspace and then retrieve and explore them, and many more, as much as she likes after the visit. Second, narratives presented through a set of related clips (videos, albums, texts) are easily seen in the conceptual map proposed by the curators. Homm-sw can be shared with non-profit organizations which intend to develop new features and share the upgrade with previous users and with new users, on the same conditions. Other engines will be implemented according to the demands of users, together with multi-channel and multilingual extension. As a tool for digital storytelling, integration with other software applications is welcome

Concept discovery and argument bundles in the web of experiences

Milions de persones interactuen i comparteixen informació cada dia a la Web. Des de converses intranscendents fins a comentaris sobre productes en tendes online, el contingut generat per les persones a la Web és enorme i divers. I entre aquests continguts n’hi ha un particularment interessant: les experiències. La gent escolta, llegeix i considera les experiències d’altri abans de prendre decisions, com per exemple comprar-se una càmera digital o anar de viatge a algun lloc exòtic. I quan aquestes experiències estan guardades a la Web, són accessibles per un gran nombre de persones. Malauradament, aquest contingut no és fàcilment accessible: Una persona interessada en anar-se’n de vacances a un hotel probablement llegirà unes quantes experiències d’altra gent que ha anat prèviament a aquest hotel per descobrir que n’opinen, però segurament no podrà llegir-les totes. D’aquesta manera ignorarà informació i experiències d’altra gent que li haurien estat profitoses de cara al seu viatge. Així doncs, considerem que hi ha una clara necessitat d’un anàlisis més profund d’aquesta informació continguda en les experiències de persones, per facilitar-ne el seu ús. El nostre enfocament es troba en el context de la Web de les Experiències, i es basa en extreure i adquirir coneixement pràctic d’experiències individuals respecte entitats del món real expressades en forma textual. A més a més, aquest coneixement han de ser tractat i representat de manera que permeti la seva reutilització per altres persones amb diferents interessos i preferències. Aquest procés està dividit en tres parts: Primer, extraiem les paraules més important utilitzades en les experiències de les diferents persones per descriure opinions sobre entitats. Seguidament, utilitzant el conjunt de paraules extretes, les agrupem en conceptes i creem un vocabulari de conceptes, que ens ajuda a descobrir els aspectes més importants de cada entitat segons les experiències viscudes per cada individu. Finalment, utilitzant el vocabulari de conceptes, creem els aplecs d’arguments de cada entitat. Els aplecs d’arguments caracteritzen els pros i els cons de cada entitat, i agreguen el coneixement pràctic extret de les experiències escrites per cada individu. A més amés, demostrem que els aplecs d’arguments, a part de ser útils per a representar el coneixement pràctic de les experiències, permeten fer inferència sobre quina entitat és la més adequada per cada individual, considerant el conjunt de preferències de cada individu. En aquesta tesi avaluem els aplecs d’arguments amb les puntuacions dels productes d’Amazon i les caracteritzacions de les càmeres de Dpreview, una web especialitzada en càmeres digitals. Demostrem com els arguments pro i con dels nostres aplecs d’arguments són molt semblants als presentats a Dpreview, fet que demostra la qualitat dels aplecs creats. Finalment, demostrem que les classificacions (rankings) de productes obtinguts amb la nostra implementació i els de Dpreview són molt semblants, mentre que la classificació donada per Amazon no s’assembla a cap de les altres dues.Millions of people interact and share interesting information every day in the Social Web. From daily conversations to comments about products in e-commerce sites, the content generated by people in these sites is huge and diverse. Among the wide diversity of user-contributed content on the web, there is a particular kind that has the potential of being put to good use by intelligent systems: human experiences. People very often use other people's experiences before making decisions, and when these kind of human experiences are expressed and recorded on the web, they can be shared with by large number of people. Nevertheless sometimes this content is not easily accessible, so a person trying to book a hotel may read a few reviews over a few hotels - but cannot possibly read them all. There is a clear need for an in-depth analysis of this kind of information, based on textual expressions of human particular experiences. Our approach, in the framework of the Web of Experiences, aims at acquiring practical knowledge from individual experiences with entities in the real world expressed in textual form. Moreover, this knowledge has to be represented in a way that facilitates the reuse of the experiential knowledge by other individuals with different preferences. Our approach has three stages: First, we extract the most salient set of aspects used by the individuals to describe their experiences with the entities in a domain. Second, using the set of extracted aspects, we group them in concepts to create a concept vocabulary that models the set of issues addressed in the reviews. Third, using the vocabulary of concepts, we create a bundle of arguments for each entity. An argument bundle characterizes the pros and cons of an entity, aggregating practical knowledge from judgments written by individuals with different biases and preferences. Moreover, we show how argument bundles allow us to define the notions of user query and the satisfaction degree of a bundle by a user query, proving that argument bundles are not only capable of representing practical knowledge but they are also useful to perform inference given a set of user preferences specified in a query. We evaluate the argument bundles of our approach with the Amazon score ratings and the camera characterizations of Dpreview. We show that pro and con arguments are very close to those listed in Dpreview. Evaluating entity rankings, we show that Dpreview and our approach give congruent rankings, while Amazon's is not congruent neither with Dpreview's or ours

Concept discovery and argument bundles in the web of experiences /

Milions de persones interactuen i comparteixen informació cada dia a la Web. Des de converses intranscendents fins a comentaris sobre productes en tendes online, el contingut generat per les persones a la Web és enorme i divers. I entre aquests continguts n'hi ha un particularment interessant: les experiències. La gent escolta, llegeix i considera les experiències d'altri abans de prendre decisions, com per exemple comprar-se una càmera digital o anar de viatge a algun lloc exòtic. I quan aquestes experiències estan guardades a la Web, són accessibles per un gran nombre de persones. Malauradament, aquest contingut no és fàcilment accessible: Una persona interessada en anar-se'n de vacances a un hotel probablement llegirà unes quantes experiències d'altra gent que ha anat prèviament a aquest hotel per descobrir que n'opinen, però segurament no podrà llegir-les totes. D'aquesta manera ignorarà informació i experiències d'altra gent que li haurien estat profitoses de cara al seu viatge. Així doncs, considerem que hi ha una clara necessitat d'un anàlisis més profund d'aquesta informació continguda en les experiències de persones, per facilitar-ne el seu ús. El nostre enfocament es troba en el context de la Web de les Experiències, i es basa en extreure i adquirir coneixement pràctic d'experiències individuals respecte entitats del món real expressades en forma textual. A més a més, aquest coneixement han de ser tractat i representat de manera que permeti la seva reutilització per altres persones amb diferents interessos i preferències. Aquest procés està dividit en tres parts: Primer, extraiem les paraules més important utilitzades en les experiències de les diferents persones per descriure opinions sobre entitats. Seguidament, utilitzant el conjunt de paraules extretes, les agrupem en conceptes i creem un vocabulari de conceptes, que ens ajuda a descobrir els aspectes més importants de cada entitat segons les experiències viscudes per cada individu. Finalment, utilitzant el vocabulari de conceptes, creem els aplecs d'arguments de cada entitat. Els aplecs d'arguments caracteritzen els pros i els cons de cada entitat, i agreguen el coneixement pràctic extret de les experiències escrites per cada individu. A més amés, demostrem que els aplecs d'arguments, a part de ser útils per a representar el coneixement pràctic de les experiències, permeten fer inferència sobre quina entitat és la més adequada per cada individual, considerant el conjunt de preferències de cada individu. En aquesta tesi avaluem els aplecs d'arguments amb les puntuacions dels productes d'Amazon i les caracteritzacions de les càmeres de Dpreview, una web especialitzada en càmeres digitals. Demostrem com els arguments pro i con dels nostres aplecs d'arguments són molt semblants als presentats a Dpreview, fet que demostra la qualitat dels aplecs creats. Finalment, demostrem que les classificacions (rankings) de productes obtinguts amb la nostra implementació i els de Dpreview són molt semblants, mentre que la classificació donada per Amazon no s'assembla a cap de les altres dues.Millions of people interact and share interesting information every day in the Social Web. From daily conversations to comments about products in e-commerce sites, the content generated by people in these sites is huge and diverse. Among the wide diversity of user-contributed content on the web, there is a particular kind that has the potential of being put to good use by intelligent systems: human experiences. People very often use other people's experiences before making decisions, and when these kind of human experiences are expressed and recorded on the web, they can be shared with by large number of people. Nevertheless sometimes this content is not easily accessible, so a person trying to book a hotel may read a few reviews over a few hotels - but cannot possibly read them all. There is a clear need for an in-depth analysis of this kind of information, based on textual expressions of human particular experiences. Our approach, in the framework of the Web of Experiences, aims at acquiring practical knowledge from individual experiences with entities in the real world expressed in textual form. Moreover, this knowledge has to be represented in a way that facilitates the reuse of the experiential knowledge by other individuals with different preferences. Our approach has three stages: First, we extract the most salient set of aspects used by the individuals to describe their experiences with the entities in a domain. Second, using the set of extracted aspects, we group them in concepts to create a concept vocabulary that models the set of issues addressed in the reviews. Third, using the vocabulary of concepts, we create a bundle of arguments for each entity. An argument bundle characterizes the pros and cons of an entity, aggregating practical knowledge from judgments written by individuals with different biases and preferences. Moreover, we show how argument bundles allow us to define the notions of user query and the satisfaction degree of a bundle by a user query, proving that argument bundles are not only capable of representing practical knowledge but they are also useful to perform inference given a set of user preferences specified in a query. We evaluate the argument bundles of our approach with the Amazon score ratings and the camera characterizations of Dpreview. We show that pro and con arguments are very close to those listed in Dpreview. Evaluating entity rankings, we show that Dpreview and our approach give congruent rankings, while Amazon's is not congruent neither with Dpreview's or ours

Computational Re-Photography

Rephotographers aim to recapture an existing photograph from the same viewpoint. A historical photograph paired with a well-aligned modern rephotograph can serve as a remarkable visualization of the passage of time. However, the task of rephotography is tedious and often imprecise, because reproducing the viewpoint of the original photograph is challenging. The rephotographer must disambiguate between the six degrees of freedom of 3D translation and rotation, and the confounding similarity between the effects of camera zoom and dolly. We present a real-time estimation and visualization technique for rephotography that helps users reach a desired viewpoint during capture. The input to our technique is a reference image taken from the desired viewpoint. The user moves through the scene with a camera and follows our visualization to reach the desired viewpoint. We employ computer vision techniques to compute the relative viewpoint difference. We guide 3D movement using two 2D arrows. We demonstrate the success of our technique by rephotographing historical images and conducting user studies

Masks: Maintaining Anonymity by Sequestering Key Statistics

High-resolution digital cameras are becoming ever-larger parts of our daily lives, whether as part of closed-circuit surveillance systems or as part of portable digital devices that many of us carry around with us. Combining the broadening reach of these cameras with automatic face recognition technology creates a sensor network that is ripe for abuse: our every action could be recorded and tagged with our identities, the date, and our location as if we each had an investigator tasked only with keeping each of us under constant surveillance. Adding the continually falling cost of data storage to this mix, and we are left with a situation where the privacy abuses don\u27t need to happen today: the stored imagery can be mined and re-mined forever, while the sophistication of automatic analysis continues to grow. The MASKS project takes the first steps toward addressing this problem. If we would like to be able to de-identify faces before the images are shared with others, we cannot do so with ad hoc techniques applied identically to all faces. Since each face is unique, the method of disguising that face must be equally unique. In order to hide or reduce those critical identifying characteristics, we are delivering the following foundational contributions toward characterizing the nature of facial information: - We have created a new pose-controlled, high-resolution database of facial images. - The most prominent anatomical markers on each face have been marked for position and shape, establishing a new gold standard for facial segmentation. - A parameterized model of the diversity of our subject population was built based on statistical analysis of the annotations. The model was validated by comparison with the performance of a standard set of artificial disguises