Contextual Media Retrieval Using Natural Language Queries

The widespread integration of cameras in hand-held and head-worn devices as well as the ability to share content online enables a large and diverse visual capture of the world that millions of users build up collectively every day. We envision these images as well as associated meta information, such as GPS coordinates and timestamps, to form a collective visual memory that can be queried while automatically taking the ever-changing context of mobile users into account. As a first step towards this vision, in this work we present Xplore-M-Ego: a novel media retrieval system that allows users to query a dynamic database of images and videos using spatio-temporal natural language queries. We evaluate our system using a new dataset of real user queries as well as through a usability study. One key finding is that there is a considerable amount of inter-user variability, for example in the resolution of spatial relations in natural language utterances. We show that our retrieval system can cope with this variability using personalisation through an online learning-based retrieval formulation.Comment: 8 pages, 9 figures, 1 tabl

An embodied conversational agent for intelligent web interaction on pandemic crisis communication

In times of crisis, an effective communication mechanism is paramount in providing accurate and timely information to the community. In this paper we study the use of an intelligent embodied conversational agent (EGA) as the front end interface with the public for a Crisis Communication Network Portal (CCNet). The proposed system, CCNet, is an integration of the intelligent conversation agent, AINI, and an Automated Knowledge Extraction Agent (AKEA). AKEA retrieves first hand information from relevant sources such as government departments and news channels. In this paper, we compare the interaction of AINI against two popular search engines, two question answering systems and two conversational systems

Evaluating SMS parsing using automated testing software

Mobile phones are ubiquitous with millions of users acquiring them every day for personal, business and social usage or communication. Its enormous pervasiveness has created a great advantage for its use as a technological tool applicable to overcome the challenges of information dissemination regarding burning issues, advertisement, and health related matters. Short message services (SMS), an integral functional part of cell phones, can be turned into a major tool for accessing databases of information on HIV/AIDS as appreciable percentage of the youth embrace the technology. The common features by the users of the unique language are the un-grammatical structure, convenience of spelling, homophony of words and alphanumeric mix up of the arrangement of words. This proves it to be difficult to serve as query in the search engine architecture. In this work SMS query was used for information accessing in Frequently Asked Question FAQ system under a specified medical domain. Finally, when the developed system was measured in terms of proximity to the answer retrieved remarkable results were observed

Text Messaging a tool in e-Health services

HIV/AIDS continues to be a menace to the global community, especially in sub-Saharan Africa and South Africa is not an exception. The infection rate is continues to grow, in particular, among the young adults. Cell phones have been identified as one of the tools that can be used to overcome the challenge of information dissemination regarding HIV/AIDS among young adults because of its acceptability within this age group. Access to appropriate information can be a powerful for prevention and management of many chronic illnesses, including hypertension, diabetes and HIV/AIDS. Within the young adults age group information access by use of Short Messaging Services (SMS) becomes particularly appealing. In this regard, it is proposed to provide access to carefully screened information on HIV/AIDS within the context of frequently asked questions (FAQ) system. However, automating SMS-based information search and retrieval poses significant challenges because of the inherent noise in SMS communications. In the paper, a special corpus of SMS messages was collected based on a standardised question-answer collection. The SMS messages were then analysed, transcribed and classified, with the aim of building a dictionary of SMS-speak to English translations, with reference to HIV/AIDS