Ontology driven voice-based interaction in mobile environment

The paper deals with a new approach for spoken dialogue handling in mobile environment. The goal of our project is to allow the user to retrieve information from a knowledge base defined by ontology, using speech in a mobile environment. This environment has specific features that should be taken into account when the speech recognition and synthesis is performed. First of all, it limits the size of the language that can be understood by speech recognizers. On the other hand, it allows us to use information about user context. Our approach is to use the knowledge and user context to allow the user to speak freely to the system. Our research has been performed in the framework of an EU funded project MUMMY. This project is targeted to the use of mobile devices on building sites. This fact determines the approach to the solution of the problem. The main issue is user context in which the interaction takes place. As the application (construction site) is rather specific it is possible to use the knowledge related to this particular application during the speech recognition process. Up-to now the voice based user interfaces are based on various techniques that usually contain various constraints which limit the communication context to strictly predefined application domain. The main idea behind our solution is usage of ontology that represents the knowledge related to our particular application in specific user context. The knowledge acquired from ontology allows the user to communicate in mobile environment as the user input analysis is heavily simplified. The crucial step in our solution was the design of proper system architecture that allows the system to access the knowledge in ontology and use it to enhance the recognition process. The model of environment in which the recognition process is performed has several parts: - Domain ontology (construction sites in general) - instance of the domain ontology (specific construction site) - conversation history + specific user context (location, type of mobile device etc.). The key part of the model is the access mechanism that allows to extract particular knowledge in specific context. This access mechanism is controlled by means of dialogue automaton that controls the course of dialogue. The acquired knowledge is used in the speech recognizer for generation of a specific grammar that defines the possible speech inputs in a particular moment of the dialogue - in the next state another access into ontology in different context is done resulting in generation of a grammar that defines new possible inputs. The same access mechanism is also used to produce a response to user\u27s input in natural language. There exists a pilot implementation of the voice based user interface system, which has been tested in various situations and the results obtained are very encouraging

A FRAMEWORK FOR INTELLIGENT VOICE-ENABLED E-EDUCATION SYSTEMS

Although the Internet has received significant attention in recent years, voice is still the most convenient and natural way of communicating between human to human or human to computer. In voice applications, users may have different needs which will require the ability of the system to reason, make decisions, be flexible and adapt to requests during interaction. These needs have placed new requirements in voice application development such as use of advanced models, techniques and methodologies which take into account the needs of different users and environments. The ability of a system to behave close to human reasoning is often mentioned as one of the major requirements for the development of voice applications. In this paper, we present a framework for an intelligent voice-enabled e-Education application and an adaptation of the framework for the development of a prototype Course Registration and Examination (CourseRegExamOnline) module. This study is a preliminary report of an ongoing e-Education project containing the following modules: enrollment, course registration and examination, enquiries/information, messaging/collaboration, e-Learning and library. The CourseRegExamOnline module was developed using VoiceXML for the voice user interface(VUI), PHP for the web user interface (WUI), Apache as the middle-ware and MySQL database as back-end. The system would offer dual access modes using the VUI and WUI. The framework would serve as a reference model for developing voice-based e-Education applications. The e-Education system when fully developed would meet the needs of students who are normal users and those with certain forms of disabilities such as visual impairment, repetitive strain injury (RSI), etc, that make reading and writing difficult

Development of Telephone-based e-Learning Portal

The proliferation of mobile phones in Nigeria, particularly among the student community, has continued to inspire the development and delivery of e-Learning applications. Most of the existing web-based e-Learning applications do not support nomadic voice-based learning (i.e. learning on the move through voice), and consequently do not provide a speedy access to information or enquiries on demand. Internet access is required to get every bit of information from most school portal system, which is not directly available to everyone. Lack of provision for voice in the existing web applications excludes support for people with limited capabilities such as the visually impaired and physical disabilities. In this paper, we present a design and development of a prototype telephone-based e-Learning portal that will be used for course registration and examination. This study is part of an ongoing e-Learning project involving the following modules: enrollment, course registration and examination, enquiries/information, messaging/collaboration, e-Learning and library. The prototype application was developed using VoiceXML for the voice user interface(VUI), PHP for database queries, Apache as the middle-ware and MySQL database as back-end. A unified modelling language (UML) was used to model and design the application. The proposed e-Learning system will compliment the web-based system in other to meet the needs of students with a range of disabilities such as visual impairment, repetitive strain injury, etc, that make reading and writing difficult. It also makes multiple platforms available to all users as well as boosting access to education for the physically challenged, particularly the sight impaired in the developing countries of the world. In institutions where students are not allowed to use mobile phones or where cost is an issue, then the alternative is the use of PC-phone

Multimodal agent interfaces and system architectures for health and fitness companions

Multimodal conversational spoken dialogues using physical and virtual agents provide a potential interface to motivate and support users in the domain of health and fitness. In this paper we present how such multimodal conversational Companions can be implemented to support their owners in various pervasive and mobile settings. In particular, we focus on different forms of multimodality and system architectures for such interfaces

Development and Deployment of VoiceXML-Based Banking Applications

In recent times, the financial sector has become one of the most vibrant sectors of the Nigerian economy with about twenty five banks after the bank consolidation / merger exercise. This sector presents huge business investments in the area of Information and Communication Technology (ICT). It is also plausible to say that the sector today is the largest body of ICT services and products users. It is no gainsaying the fact that so many Nigerians now carry mobile phones across the different parts of the country. However, applications that provide voice access to real-time banking transactions from anywhere, anytime via telephone are still at their very low stage of adoption across the Nigerian banking and financial sector. A versatile speech-enabled mobile banking application has been developed using VXML, PHP, Apache and MySQL. The developed application provides real-time access to banking services, thus improving corporate bottom-line and Quality of Service (QoS) for customer satisfaction

Mobile Audiovisual Terminal: System Design and Subjective Testing in DECT and UMTS networks

It is anticipated that there will shortly be a requirement for multimedia terminals that operate via mobile communications systems. This paper presents a functional specification for such a terminal operating at 32 kb/s in a digital European cordless telecommunications (DECT) and universal mobile telecommunications system (UMTS) radio network. A terminal has been built, based on a PC with digital signal processor (DSP) boards for audio and video coding and decoding. Speech coding is by a phonetically driven code-excited linear prediction (CELP) speech coder and video coding by a block-oriented hybrid discrete cosine transform (DCT) coder. Separate channel coding is provided for the audio and video data. The paper describes the techniques used for audio and video coding, channel coding, and synchronization. Methods of subjective testing in a DECT network and in a UMTS network are also described. These consisted of subjective tests of first impressions of the mobile audio–visual terminal (MAVT) quality, interactive tests, and the completion of an exit questionnaire. The test results showed that the quality of the audio was sufficiently good for comprehension and the video was sufficiently good for following and repeating simple mechanical tasks. However, the quality of the MAVT was not good enough for general use where high-quality audio and video was needed, especially when transmission was in a noisy radio environment