A Study of User's Performance and Satisfaction on the Web Based Photo Annotation with Speech Interaction

This paper reports on empirical evaluation study of users' performance and satisfaction with prototype of Web Based speech photo annotation with speech interaction. Participants involved consist of Johor Bahru citizens from various background. They have completed two parts of annotation task; part A involving PhotoASys; photo annotation system with proposed speech interaction and part B involving Microsoft Microsoft Vista Speech Interaction style. They have completed eight tasks for each part including system login and selection of album and photos. Users' performance was recorded using computer screen recording software. Data were captured on the task completion time and subjective satisfaction. Participants need to complete a questionnaire on the subjective satisfaction when the task was completed. The performance data show the comparison between proposed speech interaction and Microsoft Vista Speech interaction applied in photo annotation system, PhotoASys. On average, the reduction in annotation performance time due to using proposed speech interaction style was 64.72% rather than using speech interaction Microsoft Vista style. Data analysis were showed in different statistical significant in annotation performance and subjective satisfaction for both styles of interaction. These results could be used for the next design in related software which involves personal belonging management.Comment: IEEE Publication Format, https://sites.google.com/site/journalofcomputing

Photo annotation and retrieval through speech

Thesis (M. Eng.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2007.Includes bibliographical references (p. 36).In this thesis I describe the development of a speech-based annotation and retrieval system for digital photographs. The system uses a client/server architecture which allows photographs to be captured and annotated on various clients, such as mobile camera phones or the web, and then processed, indexed and stored on networked servers. For speech-based retrieval we have developed a mixed grammar recognition approach which allows the speech recognition system to construct a single finite-state network combining context-free grammars for recognizing and parsing query carrier phrases and metadata phrases, with an unconstrained statistical n-gram model for recognizing free-form search terms. Experiments demonstrating successful retrieval of photographs using purely speech-based annotation and retrieval are presented.by Brennan P. Sherry.M.Eng

An Improved Method for Image Retrieval using Speech Annotation

In this paper, we present a system for the image indexing and retrieval using speech annotations based on a pre-defined structured syntax. In addition to the introduction of N-best lists for index generation, a query expansion technique is explored to enhance the query terms and to improve retrieval effectiveness. By adding the most probable substitutions for the query terms, more relevant images are distinguished from the data collection. This approach is particularly helpful to deal with those less frequently used words, including out-of-vocabulary (OOV) words, which are very common for names of people and places. Experiments on a collection of 1,200 photos show that the retrieval effectiveness is increased considerably for segment of individual domain on People, Location and Event. With this method, the average value of precision versus recall over a combination of segments has improved significantly, from 50% to 72.4%